Mercurial > repos > shellac > sam_consensus_v3
comparison env/lib/python3.9/site-packages/cwltool/schemas/v1.2.0-dev3/CommandLineTool.yml @ 0:4f3585e2f14b draft default tip
"planemo upload commit 60cee0fc7c0cda8592644e1aad72851dec82c959"
author | shellac |
---|---|
date | Mon, 22 Mar 2021 18:12:50 +0000 |
parents | |
children |
comparison
equal
deleted
inserted
replaced
-1:000000000000 | 0:4f3585e2f14b |
---|---|
1 saladVersion: v1.1 | |
2 $base: "https://w3id.org/cwl/cwl#" | |
3 | |
4 $namespaces: | |
5 cwl: "https://w3id.org/cwl/cwl#" | |
6 | |
7 $graph: | |
8 | |
9 - name: CommandLineToolDoc | |
10 type: documentation | |
11 doc: | |
12 - | | |
13 # Common Workflow Language (CWL) Command Line Tool Description, v1.2.0-dev3 | |
14 | |
15 This version: | |
16 * https://w3id.org/cwl/v1.2.0-dev3/ | |
17 | |
18 Current version: | |
19 * https://w3id.org/cwl/ | |
20 - "\n\n" | |
21 - {$include: contrib.md} | |
22 - "\n\n" | |
23 - | | |
24 # Abstract | |
25 | |
26 A Command Line Tool is a non-interactive executable program that reads | |
27 some input, performs a computation, and terminates after producing some | |
28 output. Command line programs are a flexible unit of code sharing and | |
29 reuse, unfortunately the syntax and input/output semantics among command | |
30 line programs is extremely heterogeneous. A common layer for describing | |
31 the syntax and semantics of programs can reduce this incidental | |
32 complexity by providing a consistent way to connect programs together. | |
33 This specification defines the Common Workflow Language (CWL) Command | |
34 Line Tool Description, a vendor-neutral standard for describing the | |
35 syntax and input/output semantics of command line programs. | |
36 | |
37 - {$include: intro.md} | |
38 | |
39 - | | |
40 ## Introduction to the CWL Command Line Tool standard v1.2.0-dev3 | |
41 | |
42 This specification represents the latest development release from the | |
43 CWL group. Since the v1.1 release, v1.2.0-dev3 introduces the | |
44 following updates to the CWL Command Line Tool standard. | |
45 Documents should use `cwlVersion: v1.2.0-dev3` to make use of new | |
46 syntax and features introduced in v1.2.0-dev3. Existing v1.1 documents | |
47 should be trivially updatable by changing `cwlVersion`, however | |
48 CWL documents that relied on previously undefined or | |
49 underspecified behavior may have slightly different behavior in | |
50 v1.2.0-dev3. | |
51 | |
52 ## Changelog | |
53 | |
54 * `coresMin` and `coresMax` of | |
55 [ResourceRequirement](#ResourceRequirement) may now request fractional CPUs. | |
56 * (CommandLineTool)[#CommandLineTool] can now express `intent` | |
57 with an identifier for the type of computational operation. | |
58 * Added conformance tests for order of operations evaluating `secondaryFiles` on input | |
59 and ensure that input and output secondaryFiles expressions can return a File object. | |
60 | |
61 See also the [CWL Workflow Description, v1.2.0-dev3 changelog](Workflow.html#Changelog). | |
62 | |
63 ## Purpose | |
64 | |
65 Standalone programs are a flexible and interoperable form of code reuse. | |
66 Unlike monolithic applications, applications and analysis workflows which | |
67 are composed of multiple separate programs can be written in multiple | |
68 languages and execute concurrently on multiple hosts. However, POSIX | |
69 does not dictate computer-readable grammar or semantics for program input | |
70 and output, resulting in extremely heterogeneous command line grammar and | |
71 input/output semantics among program. This is a particular problem in | |
72 distributed computing (multi-node compute clusters) and virtualized | |
73 environments (such as Docker containers) where it is often necessary to | |
74 provision resources such as input files before executing the program. | |
75 | |
76 Often this gap is filled by hard coding program invocation and | |
77 implicitly assuming requirements will be met, or abstracting program | |
78 invocation with wrapper scripts or descriptor documents. Unfortunately, | |
79 where these approaches are application or platform specific it creates a | |
80 significant barrier to reproducibility and portability, as methods | |
81 developed for one platform must be manually ported to be used on new | |
82 platforms. Similarly it creates redundant work, as wrappers for popular | |
83 tools must be rewritten for each application or platform in use. | |
84 | |
85 The Common Workflow Language Command Line Tool Description is designed to | |
86 provide a common standard description of grammar and semantics for | |
87 invoking programs used in data-intensive fields such as Bioinformatics, | |
88 Chemistry, Physics, Astronomy, and Statistics. This specification | |
89 attempts to define a precise data and execution model for Command Line Tools that | |
90 can be implemented on a variety of computing platforms, ranging from a | |
91 single workstation to cluster, grid, cloud, and high performance | |
92 computing platforms. Details related to execution of these programs not | |
93 laid out in this specification are open to interpretation by the computing | |
94 platform implementing this specification. | |
95 | |
96 - {$include: concepts.md} | |
97 - {$include: invocation.md} | |
98 | |
99 | |
100 - type: record | |
101 name: EnvironmentDef | |
102 doc: | | |
103 Define an environment variable that will be set in the runtime environment | |
104 by the workflow platform when executing the command line tool. May be the | |
105 result of executing an expression, such as getting a parameter from input. | |
106 fields: | |
107 - name: envName | |
108 type: string | |
109 doc: The environment variable name | |
110 - name: envValue | |
111 type: [string, Expression] | |
112 doc: The environment variable value | |
113 | |
114 - type: record | |
115 name: CommandLineBinding | |
116 extends: InputBinding | |
117 docParent: "#CommandInputParameter" | |
118 doc: | | |
119 | |
120 When listed under `inputBinding` in the input schema, the term | |
121 "value" refers to the the corresponding value in the input object. For | |
122 binding objects listed in `CommandLineTool.arguments`, the term "value" | |
123 refers to the effective value after evaluating `valueFrom`. | |
124 | |
125 The binding behavior when building the command line depends on the data | |
126 type of the value. If there is a mismatch between the type described by | |
127 the input schema and the effective value, such as resulting from an | |
128 expression evaluation, an implementation must use the data type of the | |
129 effective value. | |
130 | |
131 - **string**: Add `prefix` and the string to the command line. | |
132 | |
133 - **number**: Add `prefix` and decimal representation to command line. | |
134 | |
135 - **boolean**: If true, add `prefix` to the command line. If false, add | |
136 nothing. | |
137 | |
138 - **File**: Add `prefix` and the value of | |
139 [`File.path`](#File) to the command line. | |
140 | |
141 - **Directory**: Add `prefix` and the value of | |
142 [`Directory.path`](#Directory) to the command line. | |
143 | |
144 - **array**: If `itemSeparator` is specified, add `prefix` and the join | |
145 the array into a single string with `itemSeparator` separating the | |
146 items. Otherwise first add `prefix`, then recursively process | |
147 individual elements. | |
148 If the array is empty, it does not add anything to command line. | |
149 | |
150 - **object**: Add `prefix` only, and recursively add object fields for | |
151 which `inputBinding` is specified. | |
152 | |
153 - **null**: Add nothing. | |
154 | |
155 fields: | |
156 - name: position | |
157 type: [ "null", int, Expression ] | |
158 doc: | | |
159 The sorting key. Default position is 0. If the inputBinding is | |
160 associated with an input parameter, then the value of `self` in the | |
161 expression will be the value of the input parameter. Input parameter | |
162 defaults (as specified by the `InputParameter.default` field) must be | |
163 applied before evaluating the expression. Expressions must return a | |
164 single value of type int or a null. | |
165 - name: prefix | |
166 type: string? | |
167 doc: "Command line prefix to add before the value." | |
168 - name: separate | |
169 type: boolean? | |
170 default: true | |
171 doc: | | |
172 If true (default), then the prefix and value must be added as separate | |
173 command line arguments; if false, prefix and value must be concatenated | |
174 into a single command line argument. | |
175 - name: itemSeparator | |
176 type: string? | |
177 doc: | | |
178 Join the array elements into a single string with the elements | |
179 separated by by `itemSeparator`. | |
180 - name: valueFrom | |
181 type: | |
182 - "null" | |
183 - string | |
184 - Expression | |
185 jsonldPredicate: "cwl:valueFrom" | |
186 doc: | | |
187 If `valueFrom` is a constant string value, use this as the value and | |
188 apply the binding rules above. | |
189 | |
190 If `valueFrom` is an expression, evaluate the expression to yield the | |
191 actual value to use to build the command line and apply the binding | |
192 rules above. If the inputBinding is associated with an input | |
193 parameter, the value of `self` in the expression will be the value of | |
194 the input parameter. Input parameter defaults (as specified by the | |
195 `InputParameter.default` field) must be applied before evaluating the | |
196 expression. | |
197 | |
198 If the value of the associated input parameter is `null`, `valueFrom` is | |
199 not evaluated and nothing is added to the command line. | |
200 | |
201 When a binding is part of the `CommandLineTool.arguments` field, | |
202 the `valueFrom` field is required. | |
203 | |
204 - name: shellQuote | |
205 type: boolean? | |
206 default: true | |
207 doc: | | |
208 If `ShellCommandRequirement` is in the requirements for the current command, | |
209 this controls whether the value is quoted on the command line (default is true). | |
210 Use `shellQuote: false` to inject metacharacters for operations such as pipes. | |
211 | |
212 If `shellQuote` is true or not provided, the implementation must not | |
213 permit interpretation of any shell metacharacters or directives. | |
214 | |
215 | |
216 - type: record | |
217 name: CommandOutputBinding | |
218 extends: LoadContents | |
219 doc: | | |
220 Describes how to generate an output parameter based on the files produced | |
221 by a CommandLineTool. | |
222 | |
223 The output parameter value is generated by applying these operations in the | |
224 following order: | |
225 | |
226 - glob | |
227 - loadContents | |
228 - outputEval | |
229 - secondaryFiles | |
230 fields: | |
231 - name: glob | |
232 type: | |
233 - "null" | |
234 - string | |
235 - Expression | |
236 - type: array | |
237 items: string | |
238 doc: | | |
239 Find files or directories relative to the output directory, using POSIX | |
240 glob(3) pathname matching. If an array is provided, find files or | |
241 directories that match any pattern in the array. If an expression is | |
242 provided, the expression must return a string or an array of strings, | |
243 which will then be evaluated as one or more glob patterns. Must only | |
244 match and return files/directories which actually exist. | |
245 | |
246 If the value of glob is a relative path pattern (does not | |
247 begin with a slash '/') then it is resolved relative to the | |
248 output directory. If the value of the glob is an absolute | |
249 path pattern (it does begin with a slash '/') then it must | |
250 refer to a path within the output directory. It is an error | |
251 if any glob resolves to a path outside the output directory. | |
252 Specifically this means globs that resolve to paths outside the output | |
253 directory are illegal. | |
254 | |
255 A glob may match a path within the output directory which is | |
256 actually a symlink to another file. In this case, the | |
257 expected behavior is for the resulting File/Directory object to take the | |
258 `basename` (and corresponding `nameroot` and `nameext`) of the | |
259 symlink. The `location` of the File/Directory is implementation | |
260 dependent, but logically the File/Directory should have the same content | |
261 as the symlink target. Platforms may stage output files/directories to | |
262 cloud storage that lack the concept of a symlink. In | |
263 this case file content and directories may be duplicated, or (to avoid | |
264 duplication) the File/Directory `location` may refer to the symlink | |
265 target. | |
266 | |
267 It is an error if a symlink in the output directory (or any | |
268 symlink in a chain of links) refers to any file or directory | |
269 that is not under an input or output directory. | |
270 | |
271 Implementations may shut down a container before globbing | |
272 output, so globs and expressions must not assume access to the | |
273 container filesystem except for declared input and output. | |
274 | |
275 - name: outputEval | |
276 type: Expression? | |
277 doc: | | |
278 Evaluate an expression to generate the output value. If | |
279 `glob` was specified, the value of `self` must be an array | |
280 containing file objects that were matched. If no files were | |
281 matched, `self` must be a zero length array; if a single file | |
282 was matched, the value of `self` is an array of a single | |
283 element. Additionally, if `loadContents` is `true`, the File | |
284 objects must include up to the first 64 KiB of file contents | |
285 in the `contents` field. The exit code of the process is | |
286 available in the expression as `runtime.exitCode`. | |
287 | |
288 - name: CommandLineBindable | |
289 type: record | |
290 fields: | |
291 inputBinding: | |
292 type: CommandLineBinding? | |
293 jsonldPredicate: "cwl:inputBinding" | |
294 doc: Describes how to turn this object into command line arguments. | |
295 | |
296 - name: CommandInputRecordField | |
297 type: record | |
298 extends: [InputRecordField, CommandLineBindable] | |
299 specialize: | |
300 - specializeFrom: InputRecordSchema | |
301 specializeTo: CommandInputRecordSchema | |
302 - specializeFrom: InputEnumSchema | |
303 specializeTo: CommandInputEnumSchema | |
304 - specializeFrom: InputArraySchema | |
305 specializeTo: CommandInputArraySchema | |
306 - specializeFrom: InputBinding | |
307 specializeTo: CommandLineBinding | |
308 | |
309 | |
310 - name: CommandInputRecordSchema | |
311 type: record | |
312 extends: [InputRecordSchema, CommandInputSchema, CommandLineBindable] | |
313 specialize: | |
314 - specializeFrom: InputRecordField | |
315 specializeTo: CommandInputRecordField | |
316 - specializeFrom: InputBinding | |
317 specializeTo: CommandLineBinding | |
318 | |
319 | |
320 - name: CommandInputEnumSchema | |
321 type: record | |
322 extends: [InputEnumSchema, CommandInputSchema, CommandLineBindable] | |
323 specialize: | |
324 - specializeFrom: InputBinding | |
325 specializeTo: CommandLineBinding | |
326 | |
327 | |
328 - name: CommandInputArraySchema | |
329 type: record | |
330 extends: [InputArraySchema, CommandInputSchema, CommandLineBindable] | |
331 specialize: | |
332 - specializeFrom: InputRecordSchema | |
333 specializeTo: CommandInputRecordSchema | |
334 - specializeFrom: InputEnumSchema | |
335 specializeTo: CommandInputEnumSchema | |
336 - specializeFrom: InputArraySchema | |
337 specializeTo: CommandInputArraySchema | |
338 - specializeFrom: InputBinding | |
339 specializeTo: CommandLineBinding | |
340 | |
341 | |
342 - name: CommandOutputRecordField | |
343 type: record | |
344 extends: OutputRecordField | |
345 specialize: | |
346 - specializeFrom: OutputRecordSchema | |
347 specializeTo: CommandOutputRecordSchema | |
348 - specializeFrom: OutputEnumSchema | |
349 specializeTo: CommandOutputEnumSchema | |
350 - specializeFrom: OutputArraySchema | |
351 specializeTo: CommandOutputArraySchema | |
352 fields: | |
353 - name: outputBinding | |
354 type: CommandOutputBinding? | |
355 jsonldPredicate: "cwl:outputBinding" | |
356 doc: | | |
357 Describes how to generate this output object based on the files | |
358 produced by a CommandLineTool | |
359 | |
360 | |
361 - name: CommandOutputRecordSchema | |
362 type: record | |
363 extends: OutputRecordSchema | |
364 specialize: | |
365 - specializeFrom: OutputRecordField | |
366 specializeTo: CommandOutputRecordField | |
367 | |
368 | |
369 - name: CommandOutputEnumSchema | |
370 type: record | |
371 extends: OutputEnumSchema | |
372 specialize: | |
373 - specializeFrom: OutputRecordSchema | |
374 specializeTo: CommandOutputRecordSchema | |
375 - specializeFrom: OutputEnumSchema | |
376 specializeTo: CommandOutputEnumSchema | |
377 - specializeFrom: OutputArraySchema | |
378 specializeTo: CommandOutputArraySchema | |
379 | |
380 | |
381 - name: CommandOutputArraySchema | |
382 type: record | |
383 extends: OutputArraySchema | |
384 specialize: | |
385 - specializeFrom: OutputRecordSchema | |
386 specializeTo: CommandOutputRecordSchema | |
387 - specializeFrom: OutputEnumSchema | |
388 specializeTo: CommandOutputEnumSchema | |
389 - specializeFrom: OutputArraySchema | |
390 specializeTo: CommandOutputArraySchema | |
391 | |
392 | |
393 - type: record | |
394 name: CommandInputParameter | |
395 extends: InputParameter | |
396 doc: An input parameter for a CommandLineTool. | |
397 fields: | |
398 - name: type | |
399 type: | |
400 - CWLType | |
401 - stdin | |
402 - CommandInputRecordSchema | |
403 - CommandInputEnumSchema | |
404 - CommandInputArraySchema | |
405 - string | |
406 - type: array | |
407 items: | |
408 - CWLType | |
409 - CommandInputRecordSchema | |
410 - CommandInputEnumSchema | |
411 - CommandInputArraySchema | |
412 - string | |
413 jsonldPredicate: | |
414 "_id": "sld:type" | |
415 "_type": "@vocab" | |
416 refScope: 2 | |
417 typeDSL: True | |
418 doc: | | |
419 Specify valid types of data that may be assigned to this parameter. | |
420 - name: inputBinding | |
421 type: CommandLineBinding? | |
422 doc: | | |
423 Describes how to turns the input parameters of a process into | |
424 command line arguments. | |
425 jsonldPredicate: "cwl:inputBinding" | |
426 | |
427 - type: record | |
428 name: CommandOutputParameter | |
429 extends: OutputParameter | |
430 doc: An output parameter for a CommandLineTool. | |
431 fields: | |
432 - name: type | |
433 type: | |
434 - CWLType | |
435 - stdout | |
436 - stderr | |
437 - CommandOutputRecordSchema | |
438 - CommandOutputEnumSchema | |
439 - CommandOutputArraySchema | |
440 - string | |
441 - type: array | |
442 items: | |
443 - CWLType | |
444 - CommandOutputRecordSchema | |
445 - CommandOutputEnumSchema | |
446 - CommandOutputArraySchema | |
447 - string | |
448 jsonldPredicate: | |
449 "_id": "sld:type" | |
450 "_type": "@vocab" | |
451 refScope: 2 | |
452 typeDSL: True | |
453 doc: | | |
454 Specify valid types of data that may be assigned to this parameter. | |
455 - name: outputBinding | |
456 type: CommandOutputBinding? | |
457 jsonldPredicate: "cwl:outputBinding" | |
458 doc: Describes how to generate this output object based on the files | |
459 produced by a CommandLineTool | |
460 | |
461 - name: stdin | |
462 type: enum | |
463 symbols: [ "cwl:stdin" ] | |
464 docParent: "#CommandOutputParameter" | |
465 doc: | | |
466 Only valid as a `type` for a `CommandLineTool` input with no | |
467 `inputBinding` set. `stdin` must not be specified at the `CommandLineTool` | |
468 level. | |
469 | |
470 The following | |
471 ``` | |
472 inputs: | |
473 an_input_name: | |
474 type: stdin | |
475 ``` | |
476 is equivalent to | |
477 ``` | |
478 inputs: | |
479 an_input_name: | |
480 type: File | |
481 streamable: true | |
482 | |
483 stdin: ${inputs.an_input_name.path} | |
484 ``` | |
485 | |
486 - name: stdout | |
487 type: enum | |
488 symbols: [ "cwl:stdout" ] | |
489 docParent: "#CommandOutputParameter" | |
490 doc: | | |
491 Only valid as a `type` for a `CommandLineTool` output with no | |
492 `outputBinding` set. | |
493 | |
494 The following | |
495 ``` | |
496 outputs: | |
497 an_output_name: | |
498 type: stdout | |
499 | |
500 stdout: a_stdout_file | |
501 ``` | |
502 is equivalent to | |
503 ``` | |
504 outputs: | |
505 an_output_name: | |
506 type: File | |
507 streamable: true | |
508 outputBinding: | |
509 glob: a_stdout_file | |
510 | |
511 stdout: a_stdout_file | |
512 ``` | |
513 | |
514 If there is no `stdout` name provided, a random filename will be created. | |
515 For example, the following | |
516 ``` | |
517 outputs: | |
518 an_output_name: | |
519 type: stdout | |
520 ``` | |
521 is equivalent to | |
522 ``` | |
523 outputs: | |
524 an_output_name: | |
525 type: File | |
526 streamable: true | |
527 outputBinding: | |
528 glob: random_stdout_filenameABCDEFG | |
529 | |
530 stdout: random_stdout_filenameABCDEFG | |
531 ``` | |
532 | |
533 | |
534 - name: stderr | |
535 type: enum | |
536 symbols: [ "cwl:stderr" ] | |
537 docParent: "#CommandOutputParameter" | |
538 doc: | | |
539 Only valid as a `type` for a `CommandLineTool` output with no | |
540 `outputBinding` set. | |
541 | |
542 The following | |
543 ``` | |
544 outputs: | |
545 an_output_name: | |
546 type: stderr | |
547 | |
548 stderr: a_stderr_file | |
549 ``` | |
550 is equivalent to | |
551 ``` | |
552 outputs: | |
553 an_output_name: | |
554 type: File | |
555 streamable: true | |
556 outputBinding: | |
557 glob: a_stderr_file | |
558 | |
559 stderr: a_stderr_file | |
560 ``` | |
561 | |
562 If there is no `stderr` name provided, a random filename will be created. | |
563 For example, the following | |
564 ``` | |
565 outputs: | |
566 an_output_name: | |
567 type: stderr | |
568 ``` | |
569 is equivalent to | |
570 ``` | |
571 outputs: | |
572 an_output_name: | |
573 type: File | |
574 streamable: true | |
575 outputBinding: | |
576 glob: random_stderr_filenameABCDEFG | |
577 | |
578 stderr: random_stderr_filenameABCDEFG | |
579 ``` | |
580 | |
581 | |
582 - type: record | |
583 name: CommandLineTool | |
584 extends: Process | |
585 documentRoot: true | |
586 specialize: | |
587 - specializeFrom: InputParameter | |
588 specializeTo: CommandInputParameter | |
589 - specializeFrom: OutputParameter | |
590 specializeTo: CommandOutputParameter | |
591 doc: | | |
592 This defines the schema of the CWL Command Line Tool Description document. | |
593 | |
594 fields: | |
595 - name: class | |
596 jsonldPredicate: | |
597 "_id": "@type" | |
598 "_type": "@vocab" | |
599 type: string | |
600 - name: baseCommand | |
601 doc: | | |
602 Specifies the program to execute. If an array, the first element of | |
603 the array is the command to execute, and subsequent elements are | |
604 mandatory command line arguments. The elements in `baseCommand` must | |
605 appear before any command line bindings from `inputBinding` or | |
606 `arguments`. | |
607 | |
608 If `baseCommand` is not provided or is an empty array, the first | |
609 element of the command line produced after processing `inputBinding` or | |
610 `arguments` must be used as the program to execute. | |
611 | |
612 If the program includes a path separator character it must | |
613 be an absolute path, otherwise it is an error. If the program does not | |
614 include a path separator, search the `$PATH` variable in the runtime | |
615 environment of the workflow runner find the absolute path of the | |
616 executable. | |
617 type: | |
618 - string? | |
619 - string[]? | |
620 jsonldPredicate: | |
621 "_id": "cwl:baseCommand" | |
622 "_container": "@list" | |
623 - name: arguments | |
624 doc: | | |
625 Command line bindings which are not directly associated with input | |
626 parameters. If the value is a string, it is used as a string literal | |
627 argument. If it is an Expression, the result of the evaluation is used | |
628 as an argument. | |
629 type: | |
630 - "null" | |
631 - type: array | |
632 items: [string, Expression, CommandLineBinding] | |
633 jsonldPredicate: | |
634 "_id": "cwl:arguments" | |
635 "_container": "@list" | |
636 - name: stdin | |
637 type: ["null", string, Expression] | |
638 jsonldPredicate: "https://w3id.org/cwl/cwl#stdin" | |
639 doc: | | |
640 A path to a file whose contents must be piped into the command's | |
641 standard input stream. | |
642 - name: stderr | |
643 type: ["null", string, Expression] | |
644 jsonldPredicate: "https://w3id.org/cwl/cwl#stderr" | |
645 doc: | | |
646 Capture the command's standard error stream to a file written to | |
647 the designated output directory. | |
648 | |
649 If `stderr` is a string, it specifies the file name to use. | |
650 | |
651 If `stderr` is an expression, the expression is evaluated and must | |
652 return a string with the file name to use to capture stderr. If the | |
653 return value is not a string, or the resulting path contains illegal | |
654 characters (such as the path separator `/`) it is an error. | |
655 - name: stdout | |
656 type: ["null", string, Expression] | |
657 jsonldPredicate: "https://w3id.org/cwl/cwl#stdout" | |
658 doc: | | |
659 Capture the command's standard output stream to a file written to | |
660 the designated output directory. | |
661 | |
662 If `stdout` is a string, it specifies the file name to use. | |
663 | |
664 If `stdout` is an expression, the expression is evaluated and must | |
665 return a string with the file name to use to capture stdout. If the | |
666 return value is not a string, or the resulting path contains illegal | |
667 characters (such as the path separator `/`) it is an error. | |
668 - name: successCodes | |
669 type: int[]? | |
670 doc: | | |
671 Exit codes that indicate the process completed successfully. | |
672 | |
673 - name: temporaryFailCodes | |
674 type: int[]? | |
675 doc: | | |
676 Exit codes that indicate the process failed due to a possibly | |
677 temporary condition, where executing the process with the same | |
678 runtime environment and inputs may produce different results. | |
679 | |
680 - name: permanentFailCodes | |
681 type: int[]? | |
682 doc: | |
683 Exit codes that indicate the process failed due to a permanent logic | |
684 error, where executing the process with the same runtime environment and | |
685 same inputs is expected to always fail. | |
686 | |
687 | |
688 - type: record | |
689 name: DockerRequirement | |
690 extends: ProcessRequirement | |
691 doc: | | |
692 Indicates that a workflow component should be run in a | |
693 [Docker](http://docker.com) or Docker-compatible (such as | |
694 [Singularity](https://www.sylabs.io/) and [udocker](https://github.com/indigo-dc/udocker)) container environment and | |
695 specifies how to fetch or build the image. | |
696 | |
697 If a CommandLineTool lists `DockerRequirement` under | |
698 `hints` (or `requirements`), it may (or must) be run in the specified Docker | |
699 container. | |
700 | |
701 The platform must first acquire or install the correct Docker image as | |
702 specified by `dockerPull`, `dockerImport`, `dockerLoad` or `dockerFile`. | |
703 | |
704 The platform must execute the tool in the container using `docker run` with | |
705 the appropriate Docker image and tool command line. | |
706 | |
707 The workflow platform may provide input files and the designated output | |
708 directory through the use of volume bind mounts. The platform should rewrite | |
709 file paths in the input object to correspond to the Docker bind mounted | |
710 locations. That is, the platform should rewrite values in the parameter context | |
711 such as `runtime.outdir`, `runtime.tmpdir` and others to be valid paths | |
712 within the container. The platform must ensure that `runtime.outdir` and | |
713 `runtime.tmpdir` are distinct directories. | |
714 | |
715 When running a tool contained in Docker, the workflow platform must not | |
716 assume anything about the contents of the Docker container, such as the | |
717 presence or absence of specific software, except to assume that the | |
718 generated command line represents a valid command within the runtime | |
719 environment of the container. | |
720 | |
721 A container image may specify an | |
722 [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint) | |
723 and/or | |
724 [CMD](https://docs.docker.com/engine/reference/builder/#cmd). | |
725 Command line arguments will be appended after all elements of | |
726 ENTRYPOINT, and will override all elements specified using CMD (in | |
727 other words, CMD is only used when the CommandLineTool definition | |
728 produces an empty command line). | |
729 | |
730 Use of implicit ENTRYPOINT or CMD are discouraged due to reproducibility | |
731 concerns of the implicit hidden execution point (For further discussion, see | |
732 [https://doi.org/10.12688/f1000research.15140.1](https://doi.org/10.12688/f1000research.15140.1)). Portable | |
733 CommandLineTool wrappers in which use of a container is optional must not rely on ENTRYPOINT or CMD. | |
734 CommandLineTools which do rely on ENTRYPOINT or CMD must list `DockerRequirement` in the | |
735 `requirements` section. | |
736 | |
737 ## Interaction with other requirements | |
738 | |
739 If [EnvVarRequirement](#EnvVarRequirement) is specified alongside a | |
740 DockerRequirement, the environment variables must be provided to Docker | |
741 using `--env` or `--env-file` and interact with the container's preexisting | |
742 environment as defined by Docker. | |
743 | |
744 fields: | |
745 - name: class | |
746 type: string | |
747 doc: "Always 'DockerRequirement'" | |
748 jsonldPredicate: | |
749 "_id": "@type" | |
750 "_type": "@vocab" | |
751 - name: dockerPull | |
752 type: string? | |
753 doc: | | |
754 Specify a Docker image to retrieve using `docker pull`. Can contain the | |
755 immutable digest to ensure an exact container is used: | |
756 `dockerPull: ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2` | |
757 - name: dockerLoad | |
758 type: string? | |
759 doc: "Specify a HTTP URL from which to download a Docker image using `docker load`." | |
760 - name: dockerFile | |
761 type: string? | |
762 doc: "Supply the contents of a Dockerfile which will be built using `docker build`." | |
763 - name: dockerImport | |
764 type: string? | |
765 doc: "Provide HTTP URL to download and gunzip a Docker images using `docker import." | |
766 - name: dockerImageId | |
767 type: string? | |
768 doc: | | |
769 The image id that will be used for `docker run`. May be a | |
770 human-readable image name or the image identifier hash. May be skipped | |
771 if `dockerPull` is specified, in which case the `dockerPull` image id | |
772 must be used. | |
773 - name: dockerOutputDirectory | |
774 type: string? | |
775 doc: | | |
776 Set the designated output directory to a specific location inside the | |
777 Docker container. | |
778 | |
779 | |
780 - type: record | |
781 name: SoftwareRequirement | |
782 extends: ProcessRequirement | |
783 doc: | | |
784 A list of software packages that should be configured in the environment of | |
785 the defined process. | |
786 fields: | |
787 - name: class | |
788 type: string | |
789 doc: "Always 'SoftwareRequirement'" | |
790 jsonldPredicate: | |
791 "_id": "@type" | |
792 "_type": "@vocab" | |
793 - name: packages | |
794 type: SoftwarePackage[] | |
795 doc: "The list of software to be configured." | |
796 jsonldPredicate: | |
797 mapSubject: package | |
798 mapPredicate: specs | |
799 | |
800 - name: SoftwarePackage | |
801 type: record | |
802 fields: | |
803 - name: package | |
804 type: string | |
805 doc: | | |
806 The name of the software to be made available. If the name is | |
807 common, inconsistent, or otherwise ambiguous it should be combined with | |
808 one or more identifiers in the `specs` field. | |
809 - name: version | |
810 type: string[]? | |
811 doc: | | |
812 The (optional) versions of the software that are known to be | |
813 compatible. | |
814 - name: specs | |
815 type: string[]? | |
816 jsonldPredicate: {_type: "@id", noLinkCheck: true} | |
817 doc: | | |
818 One or more [IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier)s | |
819 identifying resources for installing or enabling the software named in | |
820 the `package` field. Implementations may provide resolvers which map | |
821 these software identifer IRIs to some configuration action; or they can | |
822 use only the name from the `package` field on a best effort basis. | |
823 | |
824 For example, the IRI https://packages.debian.org/bowtie could | |
825 be resolved with `apt-get install bowtie`. The IRI | |
826 https://anaconda.org/bioconda/bowtie could be resolved with `conda | |
827 install -c bioconda bowtie`. | |
828 | |
829 IRIs can also be system independent and used to map to a specific | |
830 software installation or selection mechanism. | |
831 Using [RRID](https://www.identifiers.org/rrid/) as an example: | |
832 https://identifiers.org/rrid/RRID:SCR_005476 | |
833 could be fulfilled using the above mentioned Debian or bioconda | |
834 package, a local installation managed by [Environement Modules](http://modules.sourceforge.net/), | |
835 or any other mechanism the platform chooses. IRIs can also be from | |
836 identifer sources that are discipline specific yet still system | |
837 independent. As an example, the equivalent [ELIXIR Tools and Data | |
838 Service Registry](https://bio.tools) IRI to the previous RRID example is | |
839 https://bio.tools/tool/bowtie2/version/2.2.8. | |
840 If supported by a given registry, implementations are encouraged to | |
841 query these system independent sofware identifier IRIs directly for | |
842 links to packaging systems. | |
843 | |
844 A site specific IRI can be listed as well. For example, an academic | |
845 computing cluster using Environement Modules could list the IRI | |
846 `https://hpc.example.edu/modules/bowtie-tbb/1.22` to indicate that | |
847 `module load bowtie-tbb/1.1.2` should be executed to make available | |
848 `bowtie` version 1.1.2 compiled with the TBB library prior to running | |
849 the accompanying Workflow or CommandLineTool. Note that the example IRI | |
850 is specific to a particular institution and computing environment as | |
851 the Environment Modules system does not have a common namespace or | |
852 standardized naming convention. | |
853 | |
854 This last example is the least portable and should only be used if | |
855 mechanisms based off of the `package` field or more generic IRIs are | |
856 unavailable or unsuitable. While harmless to other sites, site specific | |
857 software IRIs should be left out of shared CWL descriptions to avoid | |
858 clutter. | |
859 | |
860 - name: Dirent | |
861 type: record | |
862 doc: | | |
863 Define a file or subdirectory that must be placed in the designated output | |
864 directory prior to executing the command line tool. May be the result of | |
865 executing an expression, such as building a configuration file from a | |
866 template. | |
867 fields: | |
868 - name: entryname | |
869 type: ["null", string, Expression] | |
870 jsonldPredicate: | |
871 _id: cwl:entryname | |
872 doc: | | |
873 The name of the file or subdirectory to create in the output directory. | |
874 If `entry` is a File or Directory, the `entryname` field overrides the value | |
875 of `basename` of the File or Directory object. Optional. | |
876 - name: entry | |
877 type: [string, Expression] | |
878 jsonldPredicate: | |
879 _id: cwl:entry | |
880 doc: | | |
881 If the value is a string literal or an expression which evaluates to a | |
882 string, a new file must be created with the string as the file contents. | |
883 | |
884 If the value is an expression that evaluates to a `File` or `Directory` | |
885 object, this indicates the referenced file or directory should be added | |
886 to the designated output directory prior to executing the tool. | |
887 | |
888 If the value is an expression that evaluates to an array of `File` or | |
889 `Directory` objects, this indicates the referenced files or directories | |
890 should be added to the designated output directory prior to executing | |
891 the tool. | |
892 | |
893 If the value is an expression that evaluates to a `Dirent` object, this | |
894 indicates that the File or Directory in `entry` should be added to the | |
895 designated output directory with the name in `entryname`. | |
896 | |
897 If the value is an expression that evaluates to `null`, nothing is added | |
898 to the designated output directory from this entry. | |
899 | |
900 If `writable` is false, the file may be made available using a bind | |
901 mount or file system link to avoid unnecessary copying of the input | |
902 file. | |
903 - name: writable | |
904 type: boolean? | |
905 default: false | |
906 doc: | | |
907 If true, the file or directory must be writable by the tool. Changes | |
908 to the file or directory must be isolated and not visible by any other | |
909 CommandLineTool process. This may be implemented by making a copy of | |
910 the original file or directory. Default false (files and directories | |
911 read-only by default). | |
912 | |
913 A directory marked as `writable: true` implies that all files and | |
914 subdirectories are recursively writable as well. | |
915 | |
916 | |
917 - name: InitialWorkDirRequirement | |
918 type: record | |
919 extends: ProcessRequirement | |
920 doc: | |
921 Define a list of files and subdirectories that must be created by the | |
922 workflow platform in the designated output directory prior to executing the | |
923 command line tool. | |
924 fields: | |
925 - name: class | |
926 type: string | |
927 doc: InitialWorkDirRequirement | |
928 jsonldPredicate: | |
929 "_id": "@type" | |
930 "_type": "@vocab" | |
931 - name: listing | |
932 type: | |
933 - type: array | |
934 items: | |
935 - "null" | |
936 - File | |
937 - type: array | |
938 items: | |
939 - File | |
940 - Directory | |
941 - Directory | |
942 - Dirent | |
943 - Expression | |
944 - Expression | |
945 jsonldPredicate: | |
946 _id: "cwl:listing" | |
947 doc: | | |
948 The list of files or subdirectories that must be placed in the | |
949 designated output directory prior to executing the command line tool. | |
950 | |
951 May be an expression. If so, the expression return value must validate as | |
952 `{type: array, items: ["null", File, Directory, Dirent, {type: array, items: [File, Directory]}]}`. | |
953 | |
954 Files or Directories which are listed in the input parameters and | |
955 appear in the `InitialWorkDirRequirement` listing must have their | |
956 `path` set to their staged location in the designated output directory. | |
957 If the same File or Directory appears more than once in the | |
958 `InitialWorkDirRequirement` listing, the implementation must choose | |
959 exactly one value for `path`; how this value is chosen is undefined. | |
960 | |
961 | |
962 - name: EnvVarRequirement | |
963 type: record | |
964 extends: ProcessRequirement | |
965 doc: | | |
966 Define a list of environment variables which will be set in the | |
967 execution environment of the tool. See `EnvironmentDef` for details. | |
968 fields: | |
969 - name: class | |
970 type: string | |
971 doc: "Always 'EnvVarRequirement'" | |
972 jsonldPredicate: | |
973 "_id": "@type" | |
974 "_type": "@vocab" | |
975 - name: envDef | |
976 type: EnvironmentDef[] | |
977 doc: The list of environment variables. | |
978 jsonldPredicate: | |
979 mapSubject: envName | |
980 mapPredicate: envValue | |
981 | |
982 | |
983 - type: record | |
984 name: ShellCommandRequirement | |
985 extends: ProcessRequirement | |
986 doc: | | |
987 Modify the behavior of CommandLineTool to generate a single string | |
988 containing a shell command line. Each item in the argument list must be | |
989 joined into a string separated by single spaces and quoted to prevent | |
990 intepretation by the shell, unless `CommandLineBinding` for that argument | |
991 contains `shellQuote: false`. If `shellQuote: false` is specified, the | |
992 argument is joined into the command string without quoting, which allows | |
993 the use of shell metacharacters such as `|` for pipes. | |
994 fields: | |
995 - name: class | |
996 type: string | |
997 doc: "Always 'ShellCommandRequirement'" | |
998 jsonldPredicate: | |
999 "_id": "@type" | |
1000 "_type": "@vocab" | |
1001 | |
1002 | |
1003 - type: record | |
1004 name: ResourceRequirement | |
1005 extends: ProcessRequirement | |
1006 doc: | | |
1007 Specify basic hardware resource requirements. | |
1008 | |
1009 "min" is the minimum amount of a resource that must be reserved to schedule | |
1010 a job. If "min" cannot be satisfied, the job should not be run. | |
1011 | |
1012 "max" is the maximum amount of a resource that the job shall be permitted | |
1013 to use. If a node has sufficient resources, multiple jobs may be scheduled | |
1014 on a single node provided each job's "max" resource requirements are | |
1015 met. If a job attempts to exceed its "max" resource allocation, an | |
1016 implementation may deny additional resources, which may result in job | |
1017 failure. | |
1018 | |
1019 If "min" is specified but "max" is not, then "max" == "min" | |
1020 If "max" is specified by "min" is not, then "min" == "max". | |
1021 | |
1022 It is an error if max < min. | |
1023 | |
1024 It is an error if the value of any of these fields is negative. | |
1025 | |
1026 If neither "min" nor "max" is specified for a resource, use the default values below. | |
1027 | |
1028 fields: | |
1029 - name: class | |
1030 type: string | |
1031 doc: "Always 'ResourceRequirement'" | |
1032 jsonldPredicate: | |
1033 "_id": "@type" | |
1034 "_type": "@vocab" | |
1035 - name: coresMin | |
1036 type: ["null", long, float, Expression] | |
1037 doc: | | |
1038 Minimum reserved number of CPU cores (default is 1). | |
1039 | |
1040 May be a fractional value to indicate to a scheduling | |
1041 algorithm that one core can be allocated to multiple | |
1042 jobs. For example, a value of 0.25 indicates that up to 4 | |
1043 jobs may run in parallel on 1 core. A value of 1.25 means | |
1044 that up to 3 jobs can run on a 4 core system (4/1.25 ≈ 3). | |
1045 | |
1046 Processes can only share a core allocation if the sum of each | |
1047 of their `ramMax`, `tmpdirMax`, and `outdirMax` requests also | |
1048 do not exceed the capacity of the node. | |
1049 | |
1050 Processes sharing a core must have the same level of isolation | |
1051 (typically a container or VM) that they would normally. | |
1052 | |
1053 The reported number of CPU cores reserved for the process, | |
1054 which is available to expressions on the CommandLineTool as | |
1055 `runtime.cores`, must be a non-zero integer, and may be | |
1056 calculated by rounding up the cores request to the next whole | |
1057 number. | |
1058 | |
1059 Scheduling systems may allocate fractional CPU resources by | |
1060 setting quotas or scheduling weights. Scheduling systems that | |
1061 do not support fractional CPUs may round up the request to the | |
1062 next whole number. | |
1063 | |
1064 - name: coresMax | |
1065 type: ["null", long, float, Expression] | |
1066 doc: | | |
1067 Maximum reserved number of CPU cores. | |
1068 | |
1069 See `coresMin` for discussion about fractional CPU requests. | |
1070 | |
1071 - name: ramMin | |
1072 type: ["null", long, Expression] | |
1073 doc: Minimum reserved RAM in mebibytes (2**20) (default is 256) | |
1074 | |
1075 - name: ramMax | |
1076 type: ["null", long, Expression] | |
1077 doc: Maximum reserved RAM in mebibytes (2**20) | |
1078 | |
1079 - name: tmpdirMin | |
1080 type: ["null", long, Expression] | |
1081 doc: Minimum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) (default is 1024) | |
1082 | |
1083 - name: tmpdirMax | |
1084 type: ["null", long, Expression] | |
1085 doc: Maximum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) | |
1086 | |
1087 - name: outdirMin | |
1088 type: ["null", long, Expression] | |
1089 doc: Minimum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) (default is 1024) | |
1090 | |
1091 - name: outdirMax | |
1092 type: ["null", long, Expression] | |
1093 doc: Maximum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) | |
1094 | |
1095 | |
1096 - type: record | |
1097 name: WorkReuse | |
1098 extends: ProcessRequirement | |
1099 doc: | | |
1100 For implementations that support reusing output from past work (on | |
1101 the assumption that same code and same input produce same | |
1102 results), control whether to enable or disable the reuse behavior | |
1103 for a particular tool or step (to accomodate situations where that | |
1104 assumption is incorrect). A reused step is not executed but | |
1105 instead returns the same output as the original execution. | |
1106 | |
1107 If `WorkReuse` is not specified, correct tools should assume it | |
1108 is enabled by default. | |
1109 fields: | |
1110 - name: class | |
1111 type: string | |
1112 doc: "Always 'WorkReuse'" | |
1113 jsonldPredicate: | |
1114 "_id": "@type" | |
1115 "_type": "@vocab" | |
1116 - name: enableReuse | |
1117 type: [boolean, Expression] | |
1118 default: true | |
1119 | |
1120 | |
1121 - type: record | |
1122 name: NetworkAccess | |
1123 extends: ProcessRequirement | |
1124 doc: | | |
1125 Indicate whether a process requires outgoing IPv4/IPv6 network | |
1126 access. Choice of IPv4 or IPv6 is implementation and site | |
1127 specific, correct tools must support both. | |
1128 | |
1129 If `networkAccess` is false or not specified, tools must not | |
1130 assume network access, except for localhost (the loopback device). | |
1131 | |
1132 If `networkAccess` is true, the tool must be able to make outgoing | |
1133 connections to network resources. Resources may be on a private | |
1134 subnet or the public Internet. However, implementations and sites | |
1135 may apply their own security policies to restrict what is | |
1136 accessible by the tool. | |
1137 | |
1138 Enabling network access does not imply a publically routable IP | |
1139 address or the ability to accept inbound connections. | |
1140 | |
1141 fields: | |
1142 - name: class | |
1143 type: string | |
1144 doc: "Always 'NetworkAccess'" | |
1145 jsonldPredicate: | |
1146 "_id": "@type" | |
1147 "_type": "@vocab" | |
1148 - name: networkAccess | |
1149 type: [boolean, Expression] | |
1150 | |
1151 - name: InplaceUpdateRequirement | |
1152 type: record | |
1153 extends: cwl:ProcessRequirement | |
1154 doc: | | |
1155 | |
1156 If `inplaceUpdate` is true, then an implementation supporting this | |
1157 feature may permit tools to directly update files with `writable: | |
1158 true` in InitialWorkDirRequirement. That is, as an optimization, | |
1159 files may be destructively modified in place as opposed to copied | |
1160 and updated. | |
1161 | |
1162 An implementation must ensure that only one workflow step may | |
1163 access a writable file at a time. It is an error if a file which | |
1164 is writable by one workflow step file is accessed (for reading or | |
1165 writing) by any other workflow step running independently. | |
1166 However, a file which has been updated in a previous completed | |
1167 step may be used as input to multiple steps, provided it is | |
1168 read-only in every step. | |
1169 | |
1170 Workflow steps which modify a file must produce the modified file | |
1171 as output. Downstream steps which futher process the file must | |
1172 use the output of previous steps, and not refer to a common input | |
1173 (this is necessary for both ordering and correctness). | |
1174 | |
1175 Workflow authors should provide this in the `hints` section. The | |
1176 intent of this feature is that workflows produce the same results | |
1177 whether or not InplaceUpdateRequirement is supported by the | |
1178 implementation, and this feature is primarily available as an | |
1179 optimization for particular environments. | |
1180 | |
1181 Users and implementers should be aware that workflows that | |
1182 destructively modify inputs may not be repeatable or reproducible. | |
1183 In particular, enabling this feature implies that WorkReuse should | |
1184 not be enabled. | |
1185 | |
1186 fields: | |
1187 class: | |
1188 type: string | |
1189 doc: "Always 'InplaceUpdateRequirement'" | |
1190 jsonldPredicate: | |
1191 "_id": "@type" | |
1192 "_type": "@vocab" | |
1193 inplaceUpdate: | |
1194 type: boolean | |
1195 | |
1196 - type: record | |
1197 name: ToolTimeLimit | |
1198 extends: ProcessRequirement | |
1199 doc: | | |
1200 Set an upper limit on the execution time of a CommandLineTool. | |
1201 A CommandLineTool whose execution duration exceeds the time | |
1202 limit may be preemptively terminated and considered failed. | |
1203 May also be used by batch systems to make scheduling decisions. | |
1204 The execution duration excludes external operations, such as | |
1205 staging of files, pulling a docker image etc, and only counts | |
1206 wall-time for the execution of the command line itself. | |
1207 fields: | |
1208 - name: class | |
1209 type: string | |
1210 doc: "Always 'ToolTimeLimit'" | |
1211 jsonldPredicate: | |
1212 "_id": "@type" | |
1213 "_type": "@vocab" | |
1214 - name: timelimit | |
1215 type: [long, Expression] | |
1216 doc: | | |
1217 The time limit, in seconds. A time limit of zero means no | |
1218 time limit. Negative time limits are an error. |