Mercurial > repos > shellac > sam_consensus_v3
comparison env/lib/python3.9/site-packages/cwltool/schemas/v1.2.0-dev2/CommandLineTool.yml @ 0:4f3585e2f14b draft default tip
"planemo upload commit 60cee0fc7c0cda8592644e1aad72851dec82c959"
author | shellac |
---|---|
date | Mon, 22 Mar 2021 18:12:50 +0000 |
parents | |
children |
comparison
equal
deleted
inserted
replaced
-1:000000000000 | 0:4f3585e2f14b |
---|---|
1 saladVersion: v1.1 | |
2 $base: "https://w3id.org/cwl/cwl#" | |
3 | |
4 $namespaces: | |
5 cwl: "https://w3id.org/cwl/cwl#" | |
6 | |
7 $graph: | |
8 | |
9 - name: CommandLineToolDoc | |
10 type: documentation | |
11 doc: | |
12 - | | |
13 # Common Workflow Language (CWL) Command Line Tool Description, v1.2.0-dev2 | |
14 | |
15 This version: | |
16 * https://w3id.org/cwl/v1.2.0-dev2/ | |
17 | |
18 Current version: | |
19 * https://w3id.org/cwl/ | |
20 - "\n\n" | |
21 - {$include: contrib.md} | |
22 - "\n\n" | |
23 - | | |
24 # Abstract | |
25 | |
26 A Command Line Tool is a non-interactive executable program that reads | |
27 some input, performs a computation, and terminates after producing some | |
28 output. Command line programs are a flexible unit of code sharing and | |
29 reuse, unfortunately the syntax and input/output semantics among command | |
30 line programs is extremely heterogeneous. A common layer for describing | |
31 the syntax and semantics of programs can reduce this incidental | |
32 complexity by providing a consistent way to connect programs together. | |
33 This specification defines the Common Workflow Language (CWL) Command | |
34 Line Tool Description, a vendor-neutral standard for describing the | |
35 syntax and input/output semantics of command line programs. | |
36 | |
37 - {$include: intro.md} | |
38 | |
39 - | | |
40 ## Introduction to the CWL Command Line Tool standard v1.2.0-dev2 | |
41 | |
42 This specification represents the latest development release from the | |
43 CWL group. Since the v1.1 release, v1.2.0-dev2 introduces the | |
44 following updates to the CWL Command Line Tool standard. | |
45 Documents should use `cwlVersion: v1.2.0-dev2` to make use of new | |
46 syntax and features introduced in v1.2.0-dev2. Existing v1.1 documents | |
47 should be trivially updatable by changing `cwlVersion`, however | |
48 CWL documents that relied on previously undefined or | |
49 underspecified behavior may have slightly different behavior in | |
50 v1.2.0-dev2. | |
51 | |
52 ## Changelog | |
53 | |
54 * `coresMin` and `coresMax` of | |
55 [ResourceRequirement](#ResourceRequirement) may now request fractional CPUs. | |
56 * (CommandLineTool)[#CommandLineTool] can now express `intent` | |
57 with an identifier for the type of computational operation. | |
58 | |
59 See also the [CWL Workflow Description, v1.2.0-dev2 changelog](Workflow.html#Changelog). | |
60 | |
61 ## Purpose | |
62 | |
63 Standalone programs are a flexible and interoperable form of code reuse. | |
64 Unlike monolithic applications, applications and analysis workflows which | |
65 are composed of multiple separate programs can be written in multiple | |
66 languages and execute concurrently on multiple hosts. However, POSIX | |
67 does not dictate computer-readable grammar or semantics for program input | |
68 and output, resulting in extremely heterogeneous command line grammar and | |
69 input/output semantics among program. This is a particular problem in | |
70 distributed computing (multi-node compute clusters) and virtualized | |
71 environments (such as Docker containers) where it is often necessary to | |
72 provision resources such as input files before executing the program. | |
73 | |
74 Often this gap is filled by hard coding program invocation and | |
75 implicitly assuming requirements will be met, or abstracting program | |
76 invocation with wrapper scripts or descriptor documents. Unfortunately, | |
77 where these approaches are application or platform specific it creates a | |
78 significant barrier to reproducibility and portability, as methods | |
79 developed for one platform must be manually ported to be used on new | |
80 platforms. Similarly it creates redundant work, as wrappers for popular | |
81 tools must be rewritten for each application or platform in use. | |
82 | |
83 The Common Workflow Language Command Line Tool Description is designed to | |
84 provide a common standard description of grammar and semantics for | |
85 invoking programs used in data-intensive fields such as Bioinformatics, | |
86 Chemistry, Physics, Astronomy, and Statistics. This specification | |
87 attempts to define a precise data and execution model for Command Line Tools that | |
88 can be implemented on a variety of computing platforms, ranging from a | |
89 single workstation to cluster, grid, cloud, and high performance | |
90 computing platforms. Details related to execution of these programs not | |
91 laid out in this specification are open to interpretation by the computing | |
92 platform implementing this specification. | |
93 | |
94 - {$include: concepts.md} | |
95 - {$include: invocation.md} | |
96 | |
97 | |
98 - type: record | |
99 name: EnvironmentDef | |
100 doc: | | |
101 Define an environment variable that will be set in the runtime environment | |
102 by the workflow platform when executing the command line tool. May be the | |
103 result of executing an expression, such as getting a parameter from input. | |
104 fields: | |
105 - name: envName | |
106 type: string | |
107 doc: The environment variable name | |
108 - name: envValue | |
109 type: [string, Expression] | |
110 doc: The environment variable value | |
111 | |
112 - type: record | |
113 name: CommandLineBinding | |
114 extends: InputBinding | |
115 docParent: "#CommandInputParameter" | |
116 doc: | | |
117 | |
118 When listed under `inputBinding` in the input schema, the term | |
119 "value" refers to the the corresponding value in the input object. For | |
120 binding objects listed in `CommandLineTool.arguments`, the term "value" | |
121 refers to the effective value after evaluating `valueFrom`. | |
122 | |
123 The binding behavior when building the command line depends on the data | |
124 type of the value. If there is a mismatch between the type described by | |
125 the input schema and the effective value, such as resulting from an | |
126 expression evaluation, an implementation must use the data type of the | |
127 effective value. | |
128 | |
129 - **string**: Add `prefix` and the string to the command line. | |
130 | |
131 - **number**: Add `prefix` and decimal representation to command line. | |
132 | |
133 - **boolean**: If true, add `prefix` to the command line. If false, add | |
134 nothing. | |
135 | |
136 - **File**: Add `prefix` and the value of | |
137 [`File.path`](#File) to the command line. | |
138 | |
139 - **Directory**: Add `prefix` and the value of | |
140 [`Directory.path`](#Directory) to the command line. | |
141 | |
142 - **array**: If `itemSeparator` is specified, add `prefix` and the join | |
143 the array into a single string with `itemSeparator` separating the | |
144 items. Otherwise first add `prefix`, then recursively process | |
145 individual elements. | |
146 If the array is empty, it does not add anything to command line. | |
147 | |
148 - **object**: Add `prefix` only, and recursively add object fields for | |
149 which `inputBinding` is specified. | |
150 | |
151 - **null**: Add nothing. | |
152 | |
153 fields: | |
154 - name: position | |
155 type: [ "null", int, Expression ] | |
156 doc: | | |
157 The sorting key. Default position is 0. If the inputBinding is | |
158 associated with an input parameter, then the value of `self` in the | |
159 expression will be the value of the input parameter. Input parameter | |
160 defaults (as specified by the `InputParameter.default` field) must be | |
161 applied before evaluating the expression. Expressions must return a | |
162 single value of type int or a null. | |
163 - name: prefix | |
164 type: string? | |
165 doc: "Command line prefix to add before the value." | |
166 - name: separate | |
167 type: boolean? | |
168 default: true | |
169 doc: | | |
170 If true (default), then the prefix and value must be added as separate | |
171 command line arguments; if false, prefix and value must be concatenated | |
172 into a single command line argument. | |
173 - name: itemSeparator | |
174 type: string? | |
175 doc: | | |
176 Join the array elements into a single string with the elements | |
177 separated by by `itemSeparator`. | |
178 - name: valueFrom | |
179 type: | |
180 - "null" | |
181 - string | |
182 - Expression | |
183 jsonldPredicate: "cwl:valueFrom" | |
184 doc: | | |
185 If `valueFrom` is a constant string value, use this as the value and | |
186 apply the binding rules above. | |
187 | |
188 If `valueFrom` is an expression, evaluate the expression to yield the | |
189 actual value to use to build the command line and apply the binding | |
190 rules above. If the inputBinding is associated with an input | |
191 parameter, the value of `self` in the expression will be the value of | |
192 the input parameter. Input parameter defaults (as specified by the | |
193 `InputParameter.default` field) must be applied before evaluating the | |
194 expression. | |
195 | |
196 If the value of the associated input parameter is `null`, `valueFrom` is | |
197 not evaluated and nothing is added to the command line. | |
198 | |
199 When a binding is part of the `CommandLineTool.arguments` field, | |
200 the `valueFrom` field is required. | |
201 | |
202 - name: shellQuote | |
203 type: boolean? | |
204 default: true | |
205 doc: | | |
206 If `ShellCommandRequirement` is in the requirements for the current command, | |
207 this controls whether the value is quoted on the command line (default is true). | |
208 Use `shellQuote: false` to inject metacharacters for operations such as pipes. | |
209 | |
210 If `shellQuote` is true or not provided, the implementation must not | |
211 permit interpretation of any shell metacharacters or directives. | |
212 | |
213 | |
214 - type: record | |
215 name: CommandOutputBinding | |
216 extends: LoadContents | |
217 doc: | | |
218 Describes how to generate an output parameter based on the files produced | |
219 by a CommandLineTool. | |
220 | |
221 The output parameter value is generated by applying these operations in the | |
222 following order: | |
223 | |
224 - glob | |
225 - loadContents | |
226 - outputEval | |
227 - secondaryFiles | |
228 fields: | |
229 - name: glob | |
230 type: | |
231 - "null" | |
232 - string | |
233 - Expression | |
234 - type: array | |
235 items: string | |
236 doc: | | |
237 Find files or directories relative to the output directory, using POSIX | |
238 glob(3) pathname matching. If an array is provided, find files or | |
239 directories that match any pattern in the array. If an expression is | |
240 provided, the expression must return a string or an array of strings, | |
241 which will then be evaluated as one or more glob patterns. Must only | |
242 match and return files/directories which actually exist. | |
243 | |
244 If the value of glob is a relative path pattern (does not | |
245 begin with a slash '/') then it is resolved relative to the | |
246 output directory. If the value of the glob is an absolute | |
247 path pattern (it does begin with a slash '/') then it must | |
248 refer to a path within the output directory. It is an error | |
249 if any glob resolves to a path outside the output directory. | |
250 Specifically this means globs that resolve to paths outside the output | |
251 directory are illegal. | |
252 | |
253 A glob may match a path within the output directory which is | |
254 actually a symlink to another file. In this case, the | |
255 expected behavior is for the resulting File/Directory object to take the | |
256 `basename` (and corresponding `nameroot` and `nameext`) of the | |
257 symlink. The `location` of the File/Directory is implementation | |
258 dependent, but logically the File/Directory should have the same content | |
259 as the symlink target. Platforms may stage output files/directories to | |
260 cloud storage that lack the concept of a symlink. In | |
261 this case file content and directories may be duplicated, or (to avoid | |
262 duplication) the File/Directory `location` may refer to the symlink | |
263 target. | |
264 | |
265 It is an error if a symlink in the output directory (or any | |
266 symlink in a chain of links) refers to any file or directory | |
267 that is not under an input or output directory. | |
268 | |
269 Implementations may shut down a container before globbing | |
270 output, so globs and expressions must not assume access to the | |
271 container filesystem except for declared input and output. | |
272 | |
273 - name: outputEval | |
274 type: Expression? | |
275 doc: | | |
276 Evaluate an expression to generate the output value. If | |
277 `glob` was specified, the value of `self` must be an array | |
278 containing file objects that were matched. If no files were | |
279 matched, `self` must be a zero length array; if a single file | |
280 was matched, the value of `self` is an array of a single | |
281 element. Additionally, if `loadContents` is `true`, the File | |
282 objects must include up to the first 64 KiB of file contents | |
283 in the `contents` field. The exit code of the process is | |
284 available in the expression as `runtime.exitCode`. | |
285 | |
286 - name: CommandLineBindable | |
287 type: record | |
288 fields: | |
289 inputBinding: | |
290 type: CommandLineBinding? | |
291 jsonldPredicate: "cwl:inputBinding" | |
292 doc: Describes how to turn this object into command line arguments. | |
293 | |
294 - name: CommandInputRecordField | |
295 type: record | |
296 extends: [InputRecordField, CommandLineBindable] | |
297 specialize: | |
298 - specializeFrom: InputRecordSchema | |
299 specializeTo: CommandInputRecordSchema | |
300 - specializeFrom: InputEnumSchema | |
301 specializeTo: CommandInputEnumSchema | |
302 - specializeFrom: InputArraySchema | |
303 specializeTo: CommandInputArraySchema | |
304 - specializeFrom: InputBinding | |
305 specializeTo: CommandLineBinding | |
306 | |
307 | |
308 - name: CommandInputRecordSchema | |
309 type: record | |
310 extends: [InputRecordSchema, CommandInputSchema, CommandLineBindable] | |
311 specialize: | |
312 - specializeFrom: InputRecordField | |
313 specializeTo: CommandInputRecordField | |
314 - specializeFrom: InputBinding | |
315 specializeTo: CommandLineBinding | |
316 | |
317 | |
318 - name: CommandInputEnumSchema | |
319 type: record | |
320 extends: [InputEnumSchema, CommandInputSchema, CommandLineBindable] | |
321 specialize: | |
322 - specializeFrom: InputBinding | |
323 specializeTo: CommandLineBinding | |
324 | |
325 | |
326 - name: CommandInputArraySchema | |
327 type: record | |
328 extends: [InputArraySchema, CommandInputSchema, CommandLineBindable] | |
329 specialize: | |
330 - specializeFrom: InputRecordSchema | |
331 specializeTo: CommandInputRecordSchema | |
332 - specializeFrom: InputEnumSchema | |
333 specializeTo: CommandInputEnumSchema | |
334 - specializeFrom: InputArraySchema | |
335 specializeTo: CommandInputArraySchema | |
336 - specializeFrom: InputBinding | |
337 specializeTo: CommandLineBinding | |
338 | |
339 | |
340 - name: CommandOutputRecordField | |
341 type: record | |
342 extends: OutputRecordField | |
343 specialize: | |
344 - specializeFrom: OutputRecordSchema | |
345 specializeTo: CommandOutputRecordSchema | |
346 - specializeFrom: OutputEnumSchema | |
347 specializeTo: CommandOutputEnumSchema | |
348 - specializeFrom: OutputArraySchema | |
349 specializeTo: CommandOutputArraySchema | |
350 fields: | |
351 - name: outputBinding | |
352 type: CommandOutputBinding? | |
353 jsonldPredicate: "cwl:outputBinding" | |
354 doc: | | |
355 Describes how to generate this output object based on the files | |
356 produced by a CommandLineTool | |
357 | |
358 | |
359 - name: CommandOutputRecordSchema | |
360 type: record | |
361 extends: OutputRecordSchema | |
362 specialize: | |
363 - specializeFrom: OutputRecordField | |
364 specializeTo: CommandOutputRecordField | |
365 | |
366 | |
367 - name: CommandOutputEnumSchema | |
368 type: record | |
369 extends: OutputEnumSchema | |
370 specialize: | |
371 - specializeFrom: OutputRecordSchema | |
372 specializeTo: CommandOutputRecordSchema | |
373 - specializeFrom: OutputEnumSchema | |
374 specializeTo: CommandOutputEnumSchema | |
375 - specializeFrom: OutputArraySchema | |
376 specializeTo: CommandOutputArraySchema | |
377 | |
378 | |
379 - name: CommandOutputArraySchema | |
380 type: record | |
381 extends: OutputArraySchema | |
382 specialize: | |
383 - specializeFrom: OutputRecordSchema | |
384 specializeTo: CommandOutputRecordSchema | |
385 - specializeFrom: OutputEnumSchema | |
386 specializeTo: CommandOutputEnumSchema | |
387 - specializeFrom: OutputArraySchema | |
388 specializeTo: CommandOutputArraySchema | |
389 | |
390 | |
391 - type: record | |
392 name: CommandInputParameter | |
393 extends: InputParameter | |
394 doc: An input parameter for a CommandLineTool. | |
395 fields: | |
396 - name: type | |
397 type: | |
398 - CWLType | |
399 - stdin | |
400 - CommandInputRecordSchema | |
401 - CommandInputEnumSchema | |
402 - CommandInputArraySchema | |
403 - string | |
404 - type: array | |
405 items: | |
406 - CWLType | |
407 - CommandInputRecordSchema | |
408 - CommandInputEnumSchema | |
409 - CommandInputArraySchema | |
410 - string | |
411 jsonldPredicate: | |
412 "_id": "sld:type" | |
413 "_type": "@vocab" | |
414 refScope: 2 | |
415 typeDSL: True | |
416 doc: | | |
417 Specify valid types of data that may be assigned to this parameter. | |
418 - name: inputBinding | |
419 type: CommandLineBinding? | |
420 doc: | | |
421 Describes how to turns the input parameters of a process into | |
422 command line arguments. | |
423 jsonldPredicate: "cwl:inputBinding" | |
424 | |
425 - type: record | |
426 name: CommandOutputParameter | |
427 extends: OutputParameter | |
428 doc: An output parameter for a CommandLineTool. | |
429 fields: | |
430 - name: type | |
431 type: | |
432 - CWLType | |
433 - stdout | |
434 - stderr | |
435 - CommandOutputRecordSchema | |
436 - CommandOutputEnumSchema | |
437 - CommandOutputArraySchema | |
438 - string | |
439 - type: array | |
440 items: | |
441 - CWLType | |
442 - CommandOutputRecordSchema | |
443 - CommandOutputEnumSchema | |
444 - CommandOutputArraySchema | |
445 - string | |
446 jsonldPredicate: | |
447 "_id": "sld:type" | |
448 "_type": "@vocab" | |
449 refScope: 2 | |
450 typeDSL: True | |
451 doc: | | |
452 Specify valid types of data that may be assigned to this parameter. | |
453 - name: outputBinding | |
454 type: CommandOutputBinding? | |
455 jsonldPredicate: "cwl:outputBinding" | |
456 doc: Describes how to generate this output object based on the files | |
457 produced by a CommandLineTool | |
458 | |
459 - name: stdin | |
460 type: enum | |
461 symbols: [ "cwl:stdin" ] | |
462 docParent: "#CommandOutputParameter" | |
463 doc: | | |
464 Only valid as a `type` for a `CommandLineTool` input with no | |
465 `inputBinding` set. `stdin` must not be specified at the `CommandLineTool` | |
466 level. | |
467 | |
468 The following | |
469 ``` | |
470 inputs: | |
471 an_input_name: | |
472 type: stdin | |
473 ``` | |
474 is equivalent to | |
475 ``` | |
476 inputs: | |
477 an_input_name: | |
478 type: File | |
479 streamable: true | |
480 | |
481 stdin: ${inputs.an_input_name.path} | |
482 ``` | |
483 | |
484 - name: stdout | |
485 type: enum | |
486 symbols: [ "cwl:stdout" ] | |
487 docParent: "#CommandOutputParameter" | |
488 doc: | | |
489 Only valid as a `type` for a `CommandLineTool` output with no | |
490 `outputBinding` set. | |
491 | |
492 The following | |
493 ``` | |
494 outputs: | |
495 an_output_name: | |
496 type: stdout | |
497 | |
498 stdout: a_stdout_file | |
499 ``` | |
500 is equivalent to | |
501 ``` | |
502 outputs: | |
503 an_output_name: | |
504 type: File | |
505 streamable: true | |
506 outputBinding: | |
507 glob: a_stdout_file | |
508 | |
509 stdout: a_stdout_file | |
510 ``` | |
511 | |
512 If there is no `stdout` name provided, a random filename will be created. | |
513 For example, the following | |
514 ``` | |
515 outputs: | |
516 an_output_name: | |
517 type: stdout | |
518 ``` | |
519 is equivalent to | |
520 ``` | |
521 outputs: | |
522 an_output_name: | |
523 type: File | |
524 streamable: true | |
525 outputBinding: | |
526 glob: random_stdout_filenameABCDEFG | |
527 | |
528 stdout: random_stdout_filenameABCDEFG | |
529 ``` | |
530 | |
531 | |
532 - name: stderr | |
533 type: enum | |
534 symbols: [ "cwl:stderr" ] | |
535 docParent: "#CommandOutputParameter" | |
536 doc: | | |
537 Only valid as a `type` for a `CommandLineTool` output with no | |
538 `outputBinding` set. | |
539 | |
540 The following | |
541 ``` | |
542 outputs: | |
543 an_output_name: | |
544 type: stderr | |
545 | |
546 stderr: a_stderr_file | |
547 ``` | |
548 is equivalent to | |
549 ``` | |
550 outputs: | |
551 an_output_name: | |
552 type: File | |
553 streamable: true | |
554 outputBinding: | |
555 glob: a_stderr_file | |
556 | |
557 stderr: a_stderr_file | |
558 ``` | |
559 | |
560 If there is no `stderr` name provided, a random filename will be created. | |
561 For example, the following | |
562 ``` | |
563 outputs: | |
564 an_output_name: | |
565 type: stderr | |
566 ``` | |
567 is equivalent to | |
568 ``` | |
569 outputs: | |
570 an_output_name: | |
571 type: File | |
572 streamable: true | |
573 outputBinding: | |
574 glob: random_stderr_filenameABCDEFG | |
575 | |
576 stderr: random_stderr_filenameABCDEFG | |
577 ``` | |
578 | |
579 | |
580 - type: record | |
581 name: CommandLineTool | |
582 extends: Process | |
583 documentRoot: true | |
584 specialize: | |
585 - specializeFrom: InputParameter | |
586 specializeTo: CommandInputParameter | |
587 - specializeFrom: OutputParameter | |
588 specializeTo: CommandOutputParameter | |
589 doc: | | |
590 This defines the schema of the CWL Command Line Tool Description document. | |
591 | |
592 fields: | |
593 - name: class | |
594 jsonldPredicate: | |
595 "_id": "@type" | |
596 "_type": "@vocab" | |
597 type: string | |
598 - name: baseCommand | |
599 doc: | | |
600 Specifies the program to execute. If an array, the first element of | |
601 the array is the command to execute, and subsequent elements are | |
602 mandatory command line arguments. The elements in `baseCommand` must | |
603 appear before any command line bindings from `inputBinding` or | |
604 `arguments`. | |
605 | |
606 If `baseCommand` is not provided or is an empty array, the first | |
607 element of the command line produced after processing `inputBinding` or | |
608 `arguments` must be used as the program to execute. | |
609 | |
610 If the program includes a path separator character it must | |
611 be an absolute path, otherwise it is an error. If the program does not | |
612 include a path separator, search the `$PATH` variable in the runtime | |
613 environment of the workflow runner find the absolute path of the | |
614 executable. | |
615 type: | |
616 - string? | |
617 - string[]? | |
618 jsonldPredicate: | |
619 "_id": "cwl:baseCommand" | |
620 "_container": "@list" | |
621 - name: arguments | |
622 doc: | | |
623 Command line bindings which are not directly associated with input | |
624 parameters. If the value is a string, it is used as a string literal | |
625 argument. If it is an Expression, the result of the evaluation is used | |
626 as an argument. | |
627 type: | |
628 - "null" | |
629 - type: array | |
630 items: [string, Expression, CommandLineBinding] | |
631 jsonldPredicate: | |
632 "_id": "cwl:arguments" | |
633 "_container": "@list" | |
634 - name: stdin | |
635 type: ["null", string, Expression] | |
636 jsonldPredicate: "https://w3id.org/cwl/cwl#stdin" | |
637 doc: | | |
638 A path to a file whose contents must be piped into the command's | |
639 standard input stream. | |
640 - name: stderr | |
641 type: ["null", string, Expression] | |
642 jsonldPredicate: "https://w3id.org/cwl/cwl#stderr" | |
643 doc: | | |
644 Capture the command's standard error stream to a file written to | |
645 the designated output directory. | |
646 | |
647 If `stderr` is a string, it specifies the file name to use. | |
648 | |
649 If `stderr` is an expression, the expression is evaluated and must | |
650 return a string with the file name to use to capture stderr. If the | |
651 return value is not a string, or the resulting path contains illegal | |
652 characters (such as the path separator `/`) it is an error. | |
653 - name: stdout | |
654 type: ["null", string, Expression] | |
655 jsonldPredicate: "https://w3id.org/cwl/cwl#stdout" | |
656 doc: | | |
657 Capture the command's standard output stream to a file written to | |
658 the designated output directory. | |
659 | |
660 If `stdout` is a string, it specifies the file name to use. | |
661 | |
662 If `stdout` is an expression, the expression is evaluated and must | |
663 return a string with the file name to use to capture stdout. If the | |
664 return value is not a string, or the resulting path contains illegal | |
665 characters (such as the path separator `/`) it is an error. | |
666 - name: successCodes | |
667 type: int[]? | |
668 doc: | | |
669 Exit codes that indicate the process completed successfully. | |
670 | |
671 - name: temporaryFailCodes | |
672 type: int[]? | |
673 doc: | | |
674 Exit codes that indicate the process failed due to a possibly | |
675 temporary condition, where executing the process with the same | |
676 runtime environment and inputs may produce different results. | |
677 | |
678 - name: permanentFailCodes | |
679 type: int[]? | |
680 doc: | |
681 Exit codes that indicate the process failed due to a permanent logic | |
682 error, where executing the process with the same runtime environment and | |
683 same inputs is expected to always fail. | |
684 | |
685 | |
686 - type: record | |
687 name: DockerRequirement | |
688 extends: ProcessRequirement | |
689 doc: | | |
690 Indicates that a workflow component should be run in a | |
691 [Docker](http://docker.com) or Docker-compatible (such as | |
692 [Singularity](https://www.sylabs.io/) and [udocker](https://github.com/indigo-dc/udocker)) container environment and | |
693 specifies how to fetch or build the image. | |
694 | |
695 If a CommandLineTool lists `DockerRequirement` under | |
696 `hints` (or `requirements`), it may (or must) be run in the specified Docker | |
697 container. | |
698 | |
699 The platform must first acquire or install the correct Docker image as | |
700 specified by `dockerPull`, `dockerImport`, `dockerLoad` or `dockerFile`. | |
701 | |
702 The platform must execute the tool in the container using `docker run` with | |
703 the appropriate Docker image and tool command line. | |
704 | |
705 The workflow platform may provide input files and the designated output | |
706 directory through the use of volume bind mounts. The platform should rewrite | |
707 file paths in the input object to correspond to the Docker bind mounted | |
708 locations. That is, the platform should rewrite values in the parameter context | |
709 such as `runtime.outdir`, `runtime.tmpdir` and others to be valid paths | |
710 within the container. The platform must ensure that `runtime.outdir` and | |
711 `runtime.tmpdir` are distinct directories. | |
712 | |
713 When running a tool contained in Docker, the workflow platform must not | |
714 assume anything about the contents of the Docker container, such as the | |
715 presence or absence of specific software, except to assume that the | |
716 generated command line represents a valid command within the runtime | |
717 environment of the container. | |
718 | |
719 A container image may specify an | |
720 [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint) | |
721 and/or | |
722 [CMD](https://docs.docker.com/engine/reference/builder/#cmd). | |
723 Command line arguments will be appended after all elements of | |
724 ENTRYPOINT, and will override all elements specified using CMD (in | |
725 other words, CMD is only used when the CommandLineTool definition | |
726 produces an empty command line). | |
727 | |
728 Use of implicit ENTRYPOINT or CMD are discouraged due to reproducibility | |
729 concerns of the implicit hidden execution point (For further discussion, see | |
730 [https://doi.org/10.12688/f1000research.15140.1](https://doi.org/10.12688/f1000research.15140.1)). Portable | |
731 CommandLineTool wrappers in which use of a container is optional must not rely on ENTRYPOINT or CMD. | |
732 CommandLineTools which do rely on ENTRYPOINT or CMD must list `DockerRequirement` in the | |
733 `requirements` section. | |
734 | |
735 ## Interaction with other requirements | |
736 | |
737 If [EnvVarRequirement](#EnvVarRequirement) is specified alongside a | |
738 DockerRequirement, the environment variables must be provided to Docker | |
739 using `--env` or `--env-file` and interact with the container's preexisting | |
740 environment as defined by Docker. | |
741 | |
742 fields: | |
743 - name: class | |
744 type: string | |
745 doc: "Always 'DockerRequirement'" | |
746 jsonldPredicate: | |
747 "_id": "@type" | |
748 "_type": "@vocab" | |
749 - name: dockerPull | |
750 type: string? | |
751 doc: | | |
752 Specify a Docker image to retrieve using `docker pull`. Can contain the | |
753 immutable digest to ensure an exact container is used: | |
754 `dockerPull: ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2` | |
755 - name: dockerLoad | |
756 type: string? | |
757 doc: "Specify a HTTP URL from which to download a Docker image using `docker load`." | |
758 - name: dockerFile | |
759 type: string? | |
760 doc: "Supply the contents of a Dockerfile which will be built using `docker build`." | |
761 - name: dockerImport | |
762 type: string? | |
763 doc: "Provide HTTP URL to download and gunzip a Docker images using `docker import." | |
764 - name: dockerImageId | |
765 type: string? | |
766 doc: | | |
767 The image id that will be used for `docker run`. May be a | |
768 human-readable image name or the image identifier hash. May be skipped | |
769 if `dockerPull` is specified, in which case the `dockerPull` image id | |
770 must be used. | |
771 - name: dockerOutputDirectory | |
772 type: string? | |
773 doc: | | |
774 Set the designated output directory to a specific location inside the | |
775 Docker container. | |
776 | |
777 | |
778 - type: record | |
779 name: SoftwareRequirement | |
780 extends: ProcessRequirement | |
781 doc: | | |
782 A list of software packages that should be configured in the environment of | |
783 the defined process. | |
784 fields: | |
785 - name: class | |
786 type: string | |
787 doc: "Always 'SoftwareRequirement'" | |
788 jsonldPredicate: | |
789 "_id": "@type" | |
790 "_type": "@vocab" | |
791 - name: packages | |
792 type: SoftwarePackage[] | |
793 doc: "The list of software to be configured." | |
794 jsonldPredicate: | |
795 mapSubject: package | |
796 mapPredicate: specs | |
797 | |
798 - name: SoftwarePackage | |
799 type: record | |
800 fields: | |
801 - name: package | |
802 type: string | |
803 doc: | | |
804 The name of the software to be made available. If the name is | |
805 common, inconsistent, or otherwise ambiguous it should be combined with | |
806 one or more identifiers in the `specs` field. | |
807 - name: version | |
808 type: string[]? | |
809 doc: | | |
810 The (optional) versions of the software that are known to be | |
811 compatible. | |
812 - name: specs | |
813 type: string[]? | |
814 jsonldPredicate: {_type: "@id", noLinkCheck: true} | |
815 doc: | | |
816 One or more [IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier)s | |
817 identifying resources for installing or enabling the software named in | |
818 the `package` field. Implementations may provide resolvers which map | |
819 these software identifer IRIs to some configuration action; or they can | |
820 use only the name from the `package` field on a best effort basis. | |
821 | |
822 For example, the IRI https://packages.debian.org/bowtie could | |
823 be resolved with `apt-get install bowtie`. The IRI | |
824 https://anaconda.org/bioconda/bowtie could be resolved with `conda | |
825 install -c bioconda bowtie`. | |
826 | |
827 IRIs can also be system independent and used to map to a specific | |
828 software installation or selection mechanism. | |
829 Using [RRID](https://www.identifiers.org/rrid/) as an example: | |
830 https://identifiers.org/rrid/RRID:SCR_005476 | |
831 could be fulfilled using the above mentioned Debian or bioconda | |
832 package, a local installation managed by [Environement Modules](http://modules.sourceforge.net/), | |
833 or any other mechanism the platform chooses. IRIs can also be from | |
834 identifer sources that are discipline specific yet still system | |
835 independent. As an example, the equivalent [ELIXIR Tools and Data | |
836 Service Registry](https://bio.tools) IRI to the previous RRID example is | |
837 https://bio.tools/tool/bowtie2/version/2.2.8. | |
838 If supported by a given registry, implementations are encouraged to | |
839 query these system independent sofware identifier IRIs directly for | |
840 links to packaging systems. | |
841 | |
842 A site specific IRI can be listed as well. For example, an academic | |
843 computing cluster using Environement Modules could list the IRI | |
844 `https://hpc.example.edu/modules/bowtie-tbb/1.22` to indicate that | |
845 `module load bowtie-tbb/1.1.2` should be executed to make available | |
846 `bowtie` version 1.1.2 compiled with the TBB library prior to running | |
847 the accompanying Workflow or CommandLineTool. Note that the example IRI | |
848 is specific to a particular institution and computing environment as | |
849 the Environment Modules system does not have a common namespace or | |
850 standardized naming convention. | |
851 | |
852 This last example is the least portable and should only be used if | |
853 mechanisms based off of the `package` field or more generic IRIs are | |
854 unavailable or unsuitable. While harmless to other sites, site specific | |
855 software IRIs should be left out of shared CWL descriptions to avoid | |
856 clutter. | |
857 | |
858 - name: Dirent | |
859 type: record | |
860 doc: | | |
861 Define a file or subdirectory that must be placed in the designated output | |
862 directory prior to executing the command line tool. May be the result of | |
863 executing an expression, such as building a configuration file from a | |
864 template. | |
865 fields: | |
866 - name: entryname | |
867 type: ["null", string, Expression] | |
868 jsonldPredicate: | |
869 _id: cwl:entryname | |
870 doc: | | |
871 The name of the file or subdirectory to create in the output directory. | |
872 If `entry` is a File or Directory, the `entryname` field overrides the value | |
873 of `basename` of the File or Directory object. Optional. | |
874 - name: entry | |
875 type: [string, Expression] | |
876 jsonldPredicate: | |
877 _id: cwl:entry | |
878 doc: | | |
879 If the value is a string literal or an expression which evaluates to a | |
880 string, a new file must be created with the string as the file contents. | |
881 | |
882 If the value is an expression that evaluates to a `File` or `Directory` | |
883 object, this indicates the referenced file or directory should be added | |
884 to the designated output directory prior to executing the tool. | |
885 | |
886 If the value is an expression that evaluates to an array of `File` or | |
887 `Directory` objects, this indicates the referenced files or directories | |
888 should be added to the designated output directory prior to executing | |
889 the tool, with ignoring the value in `entryname`. | |
890 | |
891 If the value is an expression that evaluates to a `Dirent` object, this | |
892 indicates that the File or Directory in `entry` should be added to the | |
893 designated output directory with the name in `entryname`. | |
894 | |
895 If the value is an expression that evaluates to `null`, nothing is added | |
896 to the designated output directory from this entry. | |
897 | |
898 If `writable` is false, the file may be made available using a bind | |
899 mount or file system link to avoid unnecessary copying of the input | |
900 file. | |
901 - name: writable | |
902 type: boolean? | |
903 default: false | |
904 doc: | | |
905 If true, the file or directory must be writable by the tool. Changes | |
906 to the file or directory must be isolated and not visible by any other | |
907 CommandLineTool process. This may be implemented by making a copy of | |
908 the original file or directory. Default false (files and directories | |
909 read-only by default). | |
910 | |
911 A directory marked as `writable: true` implies that all files and | |
912 subdirectories are recursively writable as well. | |
913 | |
914 | |
915 - name: InitialWorkDirRequirement | |
916 type: record | |
917 extends: ProcessRequirement | |
918 doc: | |
919 Define a list of files and subdirectories that must be created by the | |
920 workflow platform in the designated output directory prior to executing the | |
921 command line tool. | |
922 fields: | |
923 - name: class | |
924 type: string | |
925 doc: InitialWorkDirRequirement | |
926 jsonldPredicate: | |
927 "_id": "@type" | |
928 "_type": "@vocab" | |
929 - name: listing | |
930 type: | |
931 - type: array | |
932 items: | |
933 - "null" | |
934 - File | |
935 - type: array | |
936 items: | |
937 - File | |
938 - Directory | |
939 - Directory | |
940 - Dirent | |
941 - Expression | |
942 - Expression | |
943 jsonldPredicate: | |
944 _id: "cwl:listing" | |
945 doc: | | |
946 The list of files or subdirectories that must be placed in the | |
947 designated output directory prior to executing the command line tool. | |
948 | |
949 May be an expression. If so, the expression return value must validate as | |
950 `{type: array, items: ["null", File, Directory, Dirent, {type: array, items: [File, Directory]}]}`. | |
951 | |
952 Files or Directories which are listed in the input parameters and | |
953 appear in the `InitialWorkDirRequirement` listing must have their | |
954 `path` set to their staged location in the designated output directory. | |
955 If the same File or Directory appears more than once in the | |
956 `InitialWorkDirRequirement` listing, the implementation must choose | |
957 exactly one value for `path`; how this value is chosen is undefined. | |
958 | |
959 | |
960 - name: EnvVarRequirement | |
961 type: record | |
962 extends: ProcessRequirement | |
963 doc: | | |
964 Define a list of environment variables which will be set in the | |
965 execution environment of the tool. See `EnvironmentDef` for details. | |
966 fields: | |
967 - name: class | |
968 type: string | |
969 doc: "Always 'EnvVarRequirement'" | |
970 jsonldPredicate: | |
971 "_id": "@type" | |
972 "_type": "@vocab" | |
973 - name: envDef | |
974 type: EnvironmentDef[] | |
975 doc: The list of environment variables. | |
976 jsonldPredicate: | |
977 mapSubject: envName | |
978 mapPredicate: envValue | |
979 | |
980 | |
981 - type: record | |
982 name: ShellCommandRequirement | |
983 extends: ProcessRequirement | |
984 doc: | | |
985 Modify the behavior of CommandLineTool to generate a single string | |
986 containing a shell command line. Each item in the argument list must be | |
987 joined into a string separated by single spaces and quoted to prevent | |
988 intepretation by the shell, unless `CommandLineBinding` for that argument | |
989 contains `shellQuote: false`. If `shellQuote: false` is specified, the | |
990 argument is joined into the command string without quoting, which allows | |
991 the use of shell metacharacters such as `|` for pipes. | |
992 fields: | |
993 - name: class | |
994 type: string | |
995 doc: "Always 'ShellCommandRequirement'" | |
996 jsonldPredicate: | |
997 "_id": "@type" | |
998 "_type": "@vocab" | |
999 | |
1000 | |
1001 - type: record | |
1002 name: ResourceRequirement | |
1003 extends: ProcessRequirement | |
1004 doc: | | |
1005 Specify basic hardware resource requirements. | |
1006 | |
1007 "min" is the minimum amount of a resource that must be reserved to schedule | |
1008 a job. If "min" cannot be satisfied, the job should not be run. | |
1009 | |
1010 "max" is the maximum amount of a resource that the job shall be permitted | |
1011 to use. If a node has sufficient resources, multiple jobs may be scheduled | |
1012 on a single node provided each job's "max" resource requirements are | |
1013 met. If a job attempts to exceed its "max" resource allocation, an | |
1014 implementation may deny additional resources, which may result in job | |
1015 failure. | |
1016 | |
1017 If "min" is specified but "max" is not, then "max" == "min" | |
1018 If "max" is specified by "min" is not, then "min" == "max". | |
1019 | |
1020 It is an error if max < min. | |
1021 | |
1022 It is an error if the value of any of these fields is negative. | |
1023 | |
1024 If neither "min" nor "max" is specified for a resource, use the default values below. | |
1025 | |
1026 fields: | |
1027 - name: class | |
1028 type: string | |
1029 doc: "Always 'ResourceRequirement'" | |
1030 jsonldPredicate: | |
1031 "_id": "@type" | |
1032 "_type": "@vocab" | |
1033 - name: coresMin | |
1034 type: ["null", long, float, Expression] | |
1035 doc: | | |
1036 Minimum reserved number of CPU cores (default is 1). | |
1037 | |
1038 May be a fractional value to indicate to a scheduling | |
1039 algorithm that one core can be allocated to multiple | |
1040 jobs. For example, a value of 0.25 indicates that up to 4 | |
1041 jobs may run in parallel on 1 core. A value of 1.25 means | |
1042 that up to 3 jobs can run on a 4 core system (4/1.25 ≈ 3). | |
1043 | |
1044 Processes can only share a core allocation if the sum of each | |
1045 of their `ramMax`, `tmpdirMax`, and `outdirMax` requests also | |
1046 do not exceed the capacity of the node. | |
1047 | |
1048 Processes sharing a core must have the same level of isolation | |
1049 (typically a container or VM) that they would normally. | |
1050 | |
1051 The reported number of CPU cores reserved for the process, | |
1052 which is available to expressions on the CommandLineTool as | |
1053 `runtime.cores`, must be a non-zero integer, and may be | |
1054 calculated by rounding up the cores request to the next whole | |
1055 number. | |
1056 | |
1057 Scheduling systems may allocate fractional CPU resources by | |
1058 setting quotas or scheduling weights. Scheduling systems that | |
1059 do not support fractional CPUs may round up the request to the | |
1060 next whole number. | |
1061 | |
1062 - name: coresMax | |
1063 type: ["null", long, float, Expression] | |
1064 doc: | | |
1065 Maximum reserved number of CPU cores. | |
1066 | |
1067 See `coresMin` for discussion about fractional CPU requests. | |
1068 | |
1069 - name: ramMin | |
1070 type: ["null", long, Expression] | |
1071 doc: Minimum reserved RAM in mebibytes (2**20) (default is 256) | |
1072 | |
1073 - name: ramMax | |
1074 type: ["null", long, Expression] | |
1075 doc: Maximum reserved RAM in mebibytes (2**20) | |
1076 | |
1077 - name: tmpdirMin | |
1078 type: ["null", long, Expression] | |
1079 doc: Minimum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) (default is 1024) | |
1080 | |
1081 - name: tmpdirMax | |
1082 type: ["null", long, Expression] | |
1083 doc: Maximum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) | |
1084 | |
1085 - name: outdirMin | |
1086 type: ["null", long, Expression] | |
1087 doc: Minimum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) (default is 1024) | |
1088 | |
1089 - name: outdirMax | |
1090 type: ["null", long, Expression] | |
1091 doc: Maximum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) | |
1092 | |
1093 | |
1094 - type: record | |
1095 name: WorkReuse | |
1096 extends: ProcessRequirement | |
1097 doc: | | |
1098 For implementations that support reusing output from past work (on | |
1099 the assumption that same code and same input produce same | |
1100 results), control whether to enable or disable the reuse behavior | |
1101 for a particular tool or step (to accomodate situations where that | |
1102 assumption is incorrect). A reused step is not executed but | |
1103 instead returns the same output as the original execution. | |
1104 | |
1105 If `enableReuse` is not specified, correct tools should assume it | |
1106 is enabled by default. | |
1107 fields: | |
1108 - name: class | |
1109 type: string | |
1110 doc: "Always 'WorkReuse'" | |
1111 jsonldPredicate: | |
1112 "_id": "@type" | |
1113 "_type": "@vocab" | |
1114 - name: enableReuse | |
1115 type: [boolean, Expression] | |
1116 default: true | |
1117 | |
1118 | |
1119 - type: record | |
1120 name: NetworkAccess | |
1121 extends: ProcessRequirement | |
1122 doc: | | |
1123 Indicate whether a process requires outgoing IPv4/IPv6 network | |
1124 access. Choice of IPv4 or IPv6 is implementation and site | |
1125 specific, correct tools must support both. | |
1126 | |
1127 If `networkAccess` is false or not specified, tools must not | |
1128 assume network access, except for localhost (the loopback device). | |
1129 | |
1130 If `networkAccess` is true, the tool must be able to make outgoing | |
1131 connections to network resources. Resources may be on a private | |
1132 subnet or the public Internet. However, implementations and sites | |
1133 may apply their own security policies to restrict what is | |
1134 accessible by the tool. | |
1135 | |
1136 Enabling network access does not imply a publically routable IP | |
1137 address or the ability to accept inbound connections. | |
1138 | |
1139 fields: | |
1140 - name: class | |
1141 type: string | |
1142 doc: "Always 'NetworkAccess'" | |
1143 jsonldPredicate: | |
1144 "_id": "@type" | |
1145 "_type": "@vocab" | |
1146 - name: networkAccess | |
1147 type: [boolean, Expression] | |
1148 | |
1149 - name: InplaceUpdateRequirement | |
1150 type: record | |
1151 extends: cwl:ProcessRequirement | |
1152 doc: | | |
1153 | |
1154 If `inplaceUpdate` is true, then an implementation supporting this | |
1155 feature may permit tools to directly update files with `writable: | |
1156 true` in InitialWorkDirRequirement. That is, as an optimization, | |
1157 files may be destructively modified in place as opposed to copied | |
1158 and updated. | |
1159 | |
1160 An implementation must ensure that only one workflow step may | |
1161 access a writable file at a time. It is an error if a file which | |
1162 is writable by one workflow step file is accessed (for reading or | |
1163 writing) by any other workflow step running independently. | |
1164 However, a file which has been updated in a previous completed | |
1165 step may be used as input to multiple steps, provided it is | |
1166 read-only in every step. | |
1167 | |
1168 Workflow steps which modify a file must produce the modified file | |
1169 as output. Downstream steps which futher process the file must | |
1170 use the output of previous steps, and not refer to a common input | |
1171 (this is necessary for both ordering and correctness). | |
1172 | |
1173 Workflow authors should provide this in the `hints` section. The | |
1174 intent of this feature is that workflows produce the same results | |
1175 whether or not InplaceUpdateRequirement is supported by the | |
1176 implementation, and this feature is primarily available as an | |
1177 optimization for particular environments. | |
1178 | |
1179 Users and implementers should be aware that workflows that | |
1180 destructively modify inputs may not be repeatable or reproducible. | |
1181 In particular, enabling this feature implies that WorkReuse should | |
1182 not be enabled. | |
1183 | |
1184 fields: | |
1185 class: | |
1186 type: string | |
1187 doc: "Always 'InplaceUpdateRequirement'" | |
1188 jsonldPredicate: | |
1189 "_id": "@type" | |
1190 "_type": "@vocab" | |
1191 inplaceUpdate: | |
1192 type: boolean | |
1193 | |
1194 - type: record | |
1195 name: ToolTimeLimit | |
1196 extends: ProcessRequirement | |
1197 doc: | | |
1198 Set an upper limit on the execution time of a CommandLineTool. | |
1199 A CommandLineTool whose execution duration exceeds the time | |
1200 limit may be preemptively terminated and considered failed. | |
1201 May also be used by batch systems to make scheduling decisions. | |
1202 The execution duration excludes external operations, such as | |
1203 staging of files, pulling a docker image etc, and only counts | |
1204 wall-time for the execution of the command line itself. | |
1205 fields: | |
1206 - name: class | |
1207 type: string | |
1208 doc: "Always 'ToolTimeLimit'" | |
1209 jsonldPredicate: | |
1210 "_id": "@type" | |
1211 "_type": "@vocab" | |
1212 - name: timelimit | |
1213 type: [long, Expression] | |
1214 doc: | | |
1215 The time limit, in seconds. A time limit of zero means no | |
1216 time limit. Negative time limits are an error. |