Mercurial > repos > shellac > sam_consensus_v3

comparison env/lib/python3.9/site-packages/cwltool/schemas/v1.2.0-dev5/CommandLineTool.yml @ 0:4f3585e2f14b draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload commit 60cee0fc7c0cda8592644e1aad72851dec82c959"
author shellac
date Mon, 22 Mar 2021 18:12:50 +0000
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:4f3585e2f14b
1 saladVersion: v1.1
2 $base: "https://w3id.org/cwl/cwl#"
3
4 $namespaces:
5 cwl: "https://w3id.org/cwl/cwl#"
6
7 $graph:
8
9 - name: CommandLineToolDoc
10 type: documentation
11 doc:
12 - |
13 # Common Workflow Language (CWL) Command Line Tool Description, v1.2.0-dev5
14
15 This version:
16 * https://w3id.org/cwl/v1.2.0-dev5/
17
18 Current version:
19 * https://w3id.org/cwl/
20 - "\n\n"
21 - {$include: contrib.md}
22 - "\n\n"
23 - |
24 # Abstract
25
26 A Command Line Tool is a non-interactive executable program that reads
27 some input, performs a computation, and terminates after producing some
28 output. Command line programs are a flexible unit of code sharing and
29 reuse, unfortunately the syntax and input/output semantics among command
30 line programs is extremely heterogeneous. A common layer for describing
31 the syntax and semantics of programs can reduce this incidental
32 complexity by providing a consistent way to connect programs together.
33 This specification defines the Common Workflow Language (CWL) Command
34 Line Tool Description, a vendor-neutral standard for describing the
35 syntax and input/output semantics of command line programs.
36
37 - {$include: intro.md}
38
39 - |
40 ## Introduction to the CWL Command Line Tool standard v1.2.0-dev5
41
42 This specification represents the latest development release from the
43 CWL group. Since the v1.1 release, v1.2.0-dev5 introduces the
44 following updates to the CWL Command Line Tool standard.
45 Documents should use `cwlVersion: v1.2.0-dev5` to make use of new
46 syntax and features introduced in v1.2.0-dev5. Existing v1.1 documents
47 should be trivially updatable by changing `cwlVersion`, however
48 CWL documents that relied on previously undefined or
49 underspecified behavior may have slightly different behavior in
50 v1.2.0-dev5.
51
52 ## Changelog
53
54 * `coresMin` and `coresMax` of
55 [ResourceRequirement](#ResourceRequirement) may now request fractional CPUs.
56 * `ramMin`, `ramMax`, `tmpdirMin`, `tmpdirMax`, `outdirMin`, and `outdirMax` of
57 [ResourceRequirement](#ResourceRequirement) now accept floating point values.
58 * (CommandLineTool)[#CommandLineTool] can now express `intent`
59 with an identifier for the type of computational operation.
60 * Added conformance tests for order of operations evaluating `secondaryFiles` on input
61 and ensure that input and output secondaryFiles expressions can return a File object.
62 * Clarify there are no limits on the size of file literal `contents`.
63 * When using `loadContents` it now must fail when attempting to
64 load a file greater than 64 KiB instead of silently truncating
65 the data.
66 * Objects, arrays and numbers returned by parameter references
67 or expressions in `Dirent.entry` that are not a `File` or
68 `Directory` object (or array of such) are now specified to be
69 JSON serialized to produce file contents.
70 * Note that only enum and record types can be typedef-ed
71 * Added conformance tests for order of operations evaluating `secondaryFiles` on input
72 and ensure that input and output secondaryFiles expressions can return a File object.
73 * Escaping in [string interpolation](#String_Interpolation) has
74 been added to the specification along with conformance tests.
75 * It is now possible to have an absolute path in the `entryname`
76 field in
77 [InitialWorkDirRequirement](#InitialWorkDirRequirement) when
78 running in a mandatory container. Together with
79 `DockerRequirement.dockerOutputDirectory` this it possible to
80 control the locations of both input and output files when
81 running in containers.
82 * Specify default success/fail interpretation of exit codes when not given.
83
84 See also the [CWL Workflow Description, v1.2.0-dev5 changelog](Workflow.html#Changelog).
85 For other changes since CWL v1.0, see the
86 [CWL Command Line Tool Description, v1.1 changelog](https://www.commonwl.org/v1.1/CommandLineTool.html#Changelog)
87
88 ## Purpose
89
90 Standalone programs are a flexible and interoperable form of code reuse.
91 Unlike monolithic applications, applications and analysis workflows which
92 are composed of multiple separate programs can be written in multiple
93 languages and execute concurrently on multiple hosts. However, POSIX
94 does not dictate computer-readable grammar or semantics for program input
95 and output, resulting in extremely diverse command line grammar and
96 input/output semantics among programs. This is a particular problem in
97 distributed computing (multi-node compute clusters) and virtualized
98 environments (such as Docker containers) where it is often necessary to
99 provision resources such as input files before executing the program.
100
101 Often this gap is filled by hard coding program invocation and
102 implicitly assuming requirements will be met, or abstracting program
103 invocation with wrapper scripts or descriptor documents. Unfortunately,
104 where these approaches are application or platform specific it creates a
105 significant barrier to reproducibility and portability, as methods
106 developed for one platform must be manually ported to be used on new
107 platforms. Similarly it creates redundant work, as wrappers for popular
108 tools must be rewritten for each application or platform in use.
109
110 The Common Workflow Language Command Line Tool Description is designed to
111 provide a common standard description of grammar and semantics for
112 invoking programs used in data-intensive fields such as Bioinformatics,
113 Chemistry, Physics, Astronomy, and Statistics. This specification
114 attempts to define a precise data and execution model for Command Line Tools that
115 can be implemented on a variety of computing platforms, ranging from a
116 single workstation to cluster, grid, cloud, and high performance
117 computing platforms. Details related to execution of these programs not
118 laid out in this specification are open to interpretation by the computing
119 platform implementing this specification.
120
121 - {$include: concepts.md}
122 - {$include: invocation.md}
123
124
125 - type: record
126 name: EnvironmentDef
127 doc: |
128 Define an environment variable that will be set in the runtime environment
129 by the workflow platform when executing the command line tool. May be the
130 result of executing an expression, such as getting a parameter from input.
131 fields:
132 - name: envName
133 type: string
134 doc: The environment variable name
135 - name: envValue
136 type: [string, Expression]
137 doc: The environment variable value
138
139 - type: record
140 name: CommandLineBinding
141 extends: InputBinding
142 docParent: "#CommandInputParameter"
143 doc: |
144
145 When listed under `inputBinding` in the input schema, the term
146 "value" refers to the the corresponding value in the input object. For
147 binding objects listed in `CommandLineTool.arguments`, the term "value"
148 refers to the effective value after evaluating `valueFrom`.
149
150 The binding behavior when building the command line depends on the data
151 type of the value. If there is a mismatch between the type described by
152 the input schema and the effective value, such as resulting from an
153 expression evaluation, an implementation must use the data type of the
154 effective value.
155
156 - **string**: Add `prefix` and the string to the command line.
157
158 - **number**: Add `prefix` and decimal representation to command line.
159
160 - **boolean**: If true, add `prefix` to the command line. If false, add
161 nothing.
162
163 - **File**: Add `prefix` and the value of
164 [`File.path`](#File) to the command line.
165
166 - **Directory**: Add `prefix` and the value of
167 [`Directory.path`](#Directory) to the command line.
168
169 - **array**: If `itemSeparator` is specified, add `prefix` and the join
170 the array into a single string with `itemSeparator` separating the
171 items. Otherwise first add `prefix`, then recursively process
172 individual elements.
173 If the array is empty, it does not add anything to command line.
174
175 - **object**: Add `prefix` only, and recursively add object fields for
176 which `inputBinding` is specified.
177
178 - **null**: Add nothing.
179
180 fields:
181 - name: position
182 type: [ "null", int, Expression ]
183 doc: |
184 The sorting key. Default position is 0. If the inputBinding is
185 associated with an input parameter, then the value of `self` in the
186 expression will be the value of the input parameter. Input parameter
187 defaults (as specified by the `InputParameter.default` field) must be
188 applied before evaluating the expression. Expressions must return a
189 single value of type int or a null.
190 - name: prefix
191 type: string?
192 doc: "Command line prefix to add before the value."
193 - name: separate
194 type: boolean?
195 default: true
196 doc: |
197 If true (default), then the prefix and value must be added as separate
198 command line arguments; if false, prefix and value must be concatenated
199 into a single command line argument.
200 - name: itemSeparator
201 type: string?
202 doc: |
203 Join the array elements into a single string with the elements
204 separated by by `itemSeparator`.
205 - name: valueFrom
206 type:
207 - "null"
208 - string
209 - Expression
210 jsonldPredicate: "cwl:valueFrom"
211 doc: |
212 If `valueFrom` is a constant string value, use this as the value and
213 apply the binding rules above.
214
215 If `valueFrom` is an expression, evaluate the expression to yield the
216 actual value to use to build the command line and apply the binding
217 rules above. If the inputBinding is associated with an input
218 parameter, the value of `self` in the expression will be the value of
219 the input parameter. Input parameter defaults (as specified by the
220 `InputParameter.default` field) must be applied before evaluating the
221 expression.
222
223 If the value of the associated input parameter is `null`, `valueFrom` is
224 not evaluated and nothing is added to the command line.
225
226 When a binding is part of the `CommandLineTool.arguments` field,
227 the `valueFrom` field is required.
228
229 - name: shellQuote
230 type: boolean?
231 default: true
232 doc: |
233 If `ShellCommandRequirement` is in the requirements for the current command,
234 this controls whether the value is quoted on the command line (default is true).
235 Use `shellQuote: false` to inject metacharacters for operations such as pipes.
236
237 If `shellQuote` is true or not provided, the implementation must not
238 permit interpretation of any shell metacharacters or directives.
239
240
241 - type: record
242 name: CommandOutputBinding
243 extends: LoadContents
244 doc: |
245 Describes how to generate an output parameter based on the files produced
246 by a CommandLineTool.
247
248 The output parameter value is generated by applying these operations in the
249 following order:
250
251 - glob
252 - loadContents
253 - outputEval
254 - secondaryFiles
255 fields:
256 - name: glob
257 type:
258 - "null"
259 - string
260 - Expression
261 - type: array
262 items: string
263 doc: |
264 Find files or directories relative to the output directory, using POSIX
265 glob(3) pathname matching. If an array is provided, find files or
266 directories that match any pattern in the array. If an expression is
267 provided, the expression must return a string or an array of strings,
268 which will then be evaluated as one or more glob patterns. Must only
269 match and return files/directories which actually exist.
270
271 If the value of glob is a relative path pattern (does not
272 begin with a slash '/') then it is resolved relative to the
273 output directory. If the value of the glob is an absolute
274 path pattern (it does begin with a slash '/') then it must
275 refer to a path within the output directory. It is an error
276 if any glob resolves to a path outside the output directory.
277 Specifically this means globs that resolve to paths outside the output
278 directory are illegal.
279
280 A glob may match a path within the output directory which is
281 actually a symlink to another file. In this case, the
282 expected behavior is for the resulting File/Directory object to take the
283 `basename` (and corresponding `nameroot` and `nameext`) of the
284 symlink. The `location` of the File/Directory is implementation
285 dependent, but logically the File/Directory should have the same content
286 as the symlink target. Platforms may stage output files/directories to
287 cloud storage that lack the concept of a symlink. In
288 this case file content and directories may be duplicated, or (to avoid
289 duplication) the File/Directory `location` may refer to the symlink
290 target.
291
292 It is an error if a symlink in the output directory (or any
293 symlink in a chain of links) refers to any file or directory
294 that is not under an input or output directory.
295
296 Implementations may shut down a container before globbing
297 output, so globs and expressions must not assume access to the
298 container filesystem except for declared input and output.
299
300 - name: outputEval
301 type: Expression?
302 doc: |
303 Evaluate an expression to generate the output value. If
304 `glob` was specified, the value of `self` must be an array
305 containing file objects that were matched. If no files were
306 matched, `self` must be a zero length array; if a single file
307 was matched, the value of `self` is an array of a single
308 element. The exit code of the process is
309 available in the expression as `runtime.exitCode`.
310
311 Additionally if `loadContents` is true, the file must be a
312 UTF-8 text file 64 KiB or smaller, and the implementation must
313 read the entire contents of the file (or file array) and place
314 it in the `contents` field of the File object for use in
315 `outputEval`. If the size of the file is greater than 64 KiB,
316 the implementation must raise a fatal error.
317
318 If a tool needs to return a large amount of structured data to
319 the workflow, loading the output object from `cwl.output.json`
320 bypasses `outputEval` and is not subject to the 64 KiB
321 `loadContents` limit.
322
323 - name: CommandLineBindable
324 type: record
325 fields:
326 inputBinding:
327 type: CommandLineBinding?
328 jsonldPredicate: "cwl:inputBinding"
329 doc: Describes how to turn this object into command line arguments.
330
331 - name: CommandInputRecordField
332 type: record
333 extends: [InputRecordField, CommandLineBindable]
334 specialize:
335 - specializeFrom: InputRecordSchema
336 specializeTo: CommandInputRecordSchema
337 - specializeFrom: InputEnumSchema
338 specializeTo: CommandInputEnumSchema
339 - specializeFrom: InputArraySchema
340 specializeTo: CommandInputArraySchema
341 - specializeFrom: InputBinding
342 specializeTo: CommandLineBinding
343
344
345 - name: CommandInputRecordSchema
346 type: record
347 extends: [InputRecordSchema, CommandInputSchema, CommandLineBindable]
348 specialize:
349 - specializeFrom: InputRecordField
350 specializeTo: CommandInputRecordField
351 - specializeFrom: InputBinding
352 specializeTo: CommandLineBinding
353
354
355 - name: CommandInputEnumSchema
356 type: record
357 extends: [InputEnumSchema, CommandInputSchema, CommandLineBindable]
358 specialize:
359 - specializeFrom: InputBinding
360 specializeTo: CommandLineBinding
361
362
363 - name: CommandInputArraySchema
364 type: record
365 extends: [InputArraySchema, CommandInputSchema, CommandLineBindable]
366 specialize:
367 - specializeFrom: InputRecordSchema
368 specializeTo: CommandInputRecordSchema
369 - specializeFrom: InputEnumSchema
370 specializeTo: CommandInputEnumSchema
371 - specializeFrom: InputArraySchema
372 specializeTo: CommandInputArraySchema
373 - specializeFrom: InputBinding
374 specializeTo: CommandLineBinding
375
376
377 - name: CommandOutputRecordField
378 type: record
379 extends: OutputRecordField
380 specialize:
381 - specializeFrom: OutputRecordSchema
382 specializeTo: CommandOutputRecordSchema
383 - specializeFrom: OutputEnumSchema
384 specializeTo: CommandOutputEnumSchema
385 - specializeFrom: OutputArraySchema
386 specializeTo: CommandOutputArraySchema
387 fields:
388 - name: outputBinding
389 type: CommandOutputBinding?
390 jsonldPredicate: "cwl:outputBinding"
391 doc: |
392 Describes how to generate this output object based on the files
393 produced by a CommandLineTool
394
395
396 - name: CommandOutputRecordSchema
397 type: record
398 extends: OutputRecordSchema
399 specialize:
400 - specializeFrom: OutputRecordField
401 specializeTo: CommandOutputRecordField
402
403
404 - name: CommandOutputEnumSchema
405 type: record
406 extends: OutputEnumSchema
407 specialize:
408 - specializeFrom: OutputRecordSchema
409 specializeTo: CommandOutputRecordSchema
410 - specializeFrom: OutputEnumSchema
411 specializeTo: CommandOutputEnumSchema
412 - specializeFrom: OutputArraySchema
413 specializeTo: CommandOutputArraySchema
414
415
416 - name: CommandOutputArraySchema
417 type: record
418 extends: OutputArraySchema
419 specialize:
420 - specializeFrom: OutputRecordSchema
421 specializeTo: CommandOutputRecordSchema
422 - specializeFrom: OutputEnumSchema
423 specializeTo: CommandOutputEnumSchema
424 - specializeFrom: OutputArraySchema
425 specializeTo: CommandOutputArraySchema
426
427
428 - type: record
429 name: CommandInputParameter
430 extends: InputParameter
431 doc: An input parameter for a CommandLineTool.
432 fields:
433 - name: type
434 type:
435 - CWLType
436 - stdin
437 - CommandInputRecordSchema
438 - CommandInputEnumSchema
439 - CommandInputArraySchema
440 - string
441 - type: array
442 items:
443 - CWLType
444 - CommandInputRecordSchema
445 - CommandInputEnumSchema
446 - CommandInputArraySchema
447 - string
448 jsonldPredicate:
449 "_id": "sld:type"
450 "_type": "@vocab"
451 refScope: 2
452 typeDSL: True
453 doc: |
454 Specify valid types of data that may be assigned to this parameter.
455 - name: inputBinding
456 type: CommandLineBinding?
457 doc: |
458 Describes how to turns the input parameters of a process into
459 command line arguments.
460 jsonldPredicate: "cwl:inputBinding"
461
462 - type: record
463 name: CommandOutputParameter
464 extends: OutputParameter
465 doc: An output parameter for a CommandLineTool.
466 fields:
467 - name: type
468 type:
469 - CWLType
470 - stdout
471 - stderr
472 - CommandOutputRecordSchema
473 - CommandOutputEnumSchema
474 - CommandOutputArraySchema
475 - string
476 - type: array
477 items:
478 - CWLType
479 - CommandOutputRecordSchema
480 - CommandOutputEnumSchema
481 - CommandOutputArraySchema
482 - string
483 jsonldPredicate:
484 "_id": "sld:type"
485 "_type": "@vocab"
486 refScope: 2
487 typeDSL: True
488 doc: |
489 Specify valid types of data that may be assigned to this parameter.
490 - name: outputBinding
491 type: CommandOutputBinding?
492 jsonldPredicate: "cwl:outputBinding"
493 doc: Describes how to generate this output object based on the files
494 produced by a CommandLineTool
495
496 - name: stdin
497 type: enum
498 symbols: [ "cwl:stdin" ]
499 docParent: "#CommandOutputParameter"
500 doc: |
501 Only valid as a `type` for a `CommandLineTool` input with no
502 `inputBinding` set. `stdin` must not be specified at the `CommandLineTool`
503 level.
504
505 The following
506 ```
507 inputs:
508 an_input_name:
509 type: stdin
510 ```
511 is equivalent to
512 ```
513 inputs:
514 an_input_name:
515 type: File
516 streamable: true
517
518 stdin: ${inputs.an_input_name.path}
519 ```
520
521 - name: stdout
522 type: enum
523 symbols: [ "cwl:stdout" ]
524 docParent: "#CommandOutputParameter"
525 doc: |
526 Only valid as a `type` for a `CommandLineTool` output with no
527 `outputBinding` set.
528
529 The following
530 ```
531 outputs:
532 an_output_name:
533 type: stdout
534
535 stdout: a_stdout_file
536 ```
537 is equivalent to
538 ```
539 outputs:
540 an_output_name:
541 type: File
542 streamable: true
543 outputBinding:
544 glob: a_stdout_file
545
546 stdout: a_stdout_file
547 ```
548
549 If there is no `stdout` name provided, a random filename will be created.
550 For example, the following
551 ```
552 outputs:
553 an_output_name:
554 type: stdout
555 ```
556 is equivalent to
557 ```
558 outputs:
559 an_output_name:
560 type: File
561 streamable: true
562 outputBinding:
563 glob: random_stdout_filenameABCDEFG
564
565 stdout: random_stdout_filenameABCDEFG
566 ```
567
568
569 - name: stderr
570 type: enum
571 symbols: [ "cwl:stderr" ]
572 docParent: "#CommandOutputParameter"
573 doc: |
574 Only valid as a `type` for a `CommandLineTool` output with no
575 `outputBinding` set.
576
577 The following
578 ```
579 outputs:
580 an_output_name:
581 type: stderr
582
583 stderr: a_stderr_file
584 ```
585 is equivalent to
586 ```
587 outputs:
588 an_output_name:
589 type: File
590 streamable: true
591 outputBinding:
592 glob: a_stderr_file
593
594 stderr: a_stderr_file
595 ```
596
597 If there is no `stderr` name provided, a random filename will be created.
598 For example, the following
599 ```
600 outputs:
601 an_output_name:
602 type: stderr
603 ```
604 is equivalent to
605 ```
606 outputs:
607 an_output_name:
608 type: File
609 streamable: true
610 outputBinding:
611 glob: random_stderr_filenameABCDEFG
612
613 stderr: random_stderr_filenameABCDEFG
614 ```
615
616
617 - type: record
618 name: CommandLineTool
619 extends: Process
620 documentRoot: true
621 specialize:
622 - specializeFrom: InputParameter
623 specializeTo: CommandInputParameter
624 - specializeFrom: OutputParameter
625 specializeTo: CommandOutputParameter
626 doc: |
627 This defines the schema of the CWL Command Line Tool Description document.
628
629 fields:
630 - name: class
631 jsonldPredicate:
632 "_id": "@type"
633 "_type": "@vocab"
634 type: string
635 - name: baseCommand
636 doc: |
637 Specifies the program to execute. If an array, the first element of
638 the array is the command to execute, and subsequent elements are
639 mandatory command line arguments. The elements in `baseCommand` must
640 appear before any command line bindings from `inputBinding` or
641 `arguments`.
642
643 If `baseCommand` is not provided or is an empty array, the first
644 element of the command line produced after processing `inputBinding` or
645 `arguments` must be used as the program to execute.
646
647 If the program includes a path separator character it must
648 be an absolute path, otherwise it is an error. If the program does not
649 include a path separator, search the `$PATH` variable in the runtime
650 environment of the workflow runner find the absolute path of the
651 executable.
652 type:
653 - string?
654 - string[]?
655 jsonldPredicate:
656 "_id": "cwl:baseCommand"
657 "_container": "@list"
658 - name: arguments
659 doc: |
660 Command line bindings which are not directly associated with input
661 parameters. If the value is a string, it is used as a string literal
662 argument. If it is an Expression, the result of the evaluation is used
663 as an argument.
664 type:
665 - "null"
666 - type: array
667 items: [string, Expression, CommandLineBinding]
668 jsonldPredicate:
669 "_id": "cwl:arguments"
670 "_container": "@list"
671 - name: stdin
672 type: ["null", string, Expression]
673 jsonldPredicate: "https://w3id.org/cwl/cwl#stdin"
674 doc: |
675 A path to a file whose contents must be piped into the command's
676 standard input stream.
677 - name: stderr
678 type: ["null", string, Expression]
679 jsonldPredicate: "https://w3id.org/cwl/cwl#stderr"
680 doc: |
681 Capture the command's standard error stream to a file written to
682 the designated output directory.
683
684 If `stderr` is a string, it specifies the file name to use.
685
686 If `stderr` is an expression, the expression is evaluated and must
687 return a string with the file name to use to capture stderr. If the
688 return value is not a string, or the resulting path contains illegal
689 characters (such as the path separator `/`) it is an error.
690 - name: stdout
691 type: ["null", string, Expression]
692 jsonldPredicate: "https://w3id.org/cwl/cwl#stdout"
693 doc: |
694 Capture the command's standard output stream to a file written to
695 the designated output directory.
696
697 If `stdout` is a string, it specifies the file name to use.
698
699 If `stdout` is an expression, the expression is evaluated and must
700 return a string with the file name to use to capture stdout. If the
701 return value is not a string, or the resulting path contains illegal
702 characters (such as the path separator `/`) it is an error.
703 - name: successCodes
704 type: int[]?
705 doc: |
706 Exit codes that indicate the process completed successfully.
707
708 If not specified, only exit code 0 is considered success.
709
710 - name: temporaryFailCodes
711 type: int[]?
712 doc: |
713 Exit codes that indicate the process failed due to a possibly
714 temporary condition, where executing the process with the same
715 runtime environment and inputs may produce different results.
716
717 If not specified, no exit codes are considered temporary failure.
718
719 - name: permanentFailCodes
720 type: int[]?
721 doc:
722 Exit codes that indicate the process failed due to a permanent logic
723 error, where executing the process with the same runtime environment and
724 same inputs is expected to always fail.
725
726 If not specified, all exit codes except 0 are considered permanent failure.
727
728
729 - type: record
730 name: DockerRequirement
731 extends: ProcessRequirement
732 doc: |
733 Indicates that a workflow component should be run in a
734 [Docker](http://docker.com) or Docker-compatible (such as
735 [Singularity](https://www.sylabs.io/) and [udocker](https://github.com/indigo-dc/udocker)) container environment and
736 specifies how to fetch or build the image.
737
738 If a CommandLineTool lists `DockerRequirement` under
739 `hints` (or `requirements`), it may (or must) be run in the specified Docker
740 container.
741
742 The platform must first acquire or install the correct Docker image as
743 specified by `dockerPull`, `dockerImport`, `dockerLoad` or `dockerFile`.
744
745 The platform must execute the tool in the container using `docker run` with
746 the appropriate Docker image and tool command line.
747
748 The workflow platform may provide input files and the designated output
749 directory through the use of volume bind mounts. The platform should rewrite
750 file paths in the input object to correspond to the Docker bind mounted
751 locations. That is, the platform should rewrite values in the parameter context
752 such as `runtime.outdir`, `runtime.tmpdir` and others to be valid paths
753 within the container. The platform must ensure that `runtime.outdir` and
754 `runtime.tmpdir` are distinct directories.
755
756 When running a tool contained in Docker, the workflow platform must not
757 assume anything about the contents of the Docker container, such as the
758 presence or absence of specific software, except to assume that the
759 generated command line represents a valid command within the runtime
760 environment of the container.
761
762 A container image may specify an
763 [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)
764 and/or
765 [CMD](https://docs.docker.com/engine/reference/builder/#cmd).
766 Command line arguments will be appended after all elements of
767 ENTRYPOINT, and will override all elements specified using CMD (in
768 other words, CMD is only used when the CommandLineTool definition
769 produces an empty command line).
770
771 Use of implicit ENTRYPOINT or CMD are discouraged due to reproducibility
772 concerns of the implicit hidden execution point (For further discussion, see
773 [https://doi.org/10.12688/f1000research.15140.1](https://doi.org/10.12688/f1000research.15140.1)). Portable
774 CommandLineTool wrappers in which use of a container is optional must not rely on ENTRYPOINT or CMD.
775 CommandLineTools which do rely on ENTRYPOINT or CMD must list `DockerRequirement` in the
776 `requirements` section.
777
778 ## Interaction with other requirements
779
780 If [EnvVarRequirement](#EnvVarRequirement) is specified alongside a
781 DockerRequirement, the environment variables must be provided to Docker
782 using `--env` or `--env-file` and interact with the container's preexisting
783 environment as defined by Docker.
784
785 fields:
786 - name: class
787 type: string
788 doc: "Always 'DockerRequirement'"
789 jsonldPredicate:
790 "_id": "@type"
791 "_type": "@vocab"
792 - name: dockerPull
793 type: string?
794 doc: |
795 Specify a Docker image to retrieve using `docker pull`. Can contain the
796 immutable digest to ensure an exact container is used:
797 `dockerPull: ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2`
798 - name: dockerLoad
799 type: string?
800 doc: "Specify a HTTP URL from which to download a Docker image using `docker load`."
801 - name: dockerFile
802 type: string?
803 doc: "Supply the contents of a Dockerfile which will be built using `docker build`."
804 - name: dockerImport
805 type: string?
806 doc: "Provide HTTP URL to download and gunzip a Docker images using `docker import."
807 - name: dockerImageId
808 type: string?
809 doc: |
810 The image id that will be used for `docker run`. May be a
811 human-readable image name or the image identifier hash. May be skipped
812 if `dockerPull` is specified, in which case the `dockerPull` image id
813 must be used.
814 - name: dockerOutputDirectory
815 type: string?
816 doc: |
817 Set the designated output directory to a specific location inside the
818 Docker container.
819
820
821 - type: record
822 name: SoftwareRequirement
823 extends: ProcessRequirement
824 doc: |
825 A list of software packages that should be configured in the environment of
826 the defined process.
827 fields:
828 - name: class
829 type: string
830 doc: "Always 'SoftwareRequirement'"
831 jsonldPredicate:
832 "_id": "@type"
833 "_type": "@vocab"
834 - name: packages
835 type: SoftwarePackage[]
836 doc: "The list of software to be configured."
837 jsonldPredicate:
838 mapSubject: package
839 mapPredicate: specs
840
841 - name: SoftwarePackage
842 type: record
843 fields:
844 - name: package
845 type: string
846 doc: |
847 The name of the software to be made available. If the name is
848 common, inconsistent, or otherwise ambiguous it should be combined with
849 one or more identifiers in the `specs` field.
850 - name: version
851 type: string[]?
852 doc: |
853 The (optional) versions of the software that are known to be
854 compatible.
855 - name: specs
856 type: string[]?
857 jsonldPredicate: {_type: "@id", noLinkCheck: true}
858 doc: |
859 One or more [IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier)s
860 identifying resources for installing or enabling the software named in
861 the `package` field. Implementations may provide resolvers which map
862 these software identifer IRIs to some configuration action; or they can
863 use only the name from the `package` field on a best effort basis.
864
865 For example, the IRI https://packages.debian.org/bowtie could
866 be resolved with `apt-get install bowtie`. The IRI
867 https://anaconda.org/bioconda/bowtie could be resolved with `conda
868 install -c bioconda bowtie`.
869
870 IRIs can also be system independent and used to map to a specific
871 software installation or selection mechanism.
872 Using [RRID](https://www.identifiers.org/rrid/) as an example:
873 https://identifiers.org/rrid/RRID:SCR_005476
874 could be fulfilled using the above mentioned Debian or bioconda
875 package, a local installation managed by [Environement Modules](http://modules.sourceforge.net/),
876 or any other mechanism the platform chooses. IRIs can also be from
877 identifer sources that are discipline specific yet still system
878 independent. As an example, the equivalent [ELIXIR Tools and Data
879 Service Registry](https://bio.tools) IRI to the previous RRID example is
880 https://bio.tools/tool/bowtie2/version/2.2.8.
881 If supported by a given registry, implementations are encouraged to
882 query these system independent sofware identifier IRIs directly for
883 links to packaging systems.
884
885 A site specific IRI can be listed as well. For example, an academic
886 computing cluster using Environement Modules could list the IRI
887 `https://hpc.example.edu/modules/bowtie-tbb/1.22` to indicate that
888 `module load bowtie-tbb/1.1.2` should be executed to make available
889 `bowtie` version 1.1.2 compiled with the TBB library prior to running
890 the accompanying Workflow or CommandLineTool. Note that the example IRI
891 is specific to a particular institution and computing environment as
892 the Environment Modules system does not have a common namespace or
893 standardized naming convention.
894
895 This last example is the least portable and should only be used if
896 mechanisms based off of the `package` field or more generic IRIs are
897 unavailable or unsuitable. While harmless to other sites, site specific
898 software IRIs should be left out of shared CWL descriptions to avoid
899 clutter.
900
901 - name: Dirent
902 type: record
903 doc: |
904 Define a file or subdirectory that must be staged to a particular
905 place prior to executing the command line tool. May be the result
906 of executing an expression, such as building a configuration file
907 from a template.
908
909 Usually files are staged within the [designated output directory](#Runtime_environment).
910 However, under certain circumstances, files may be staged at
911 arbitrary locations, see discussion for `entryname`.
912
913 fields:
914 - name: entryname
915 type: ["null", string, Expression]
916 jsonldPredicate:
917 _id: cwl:entryname
918 doc: |
919 The "target" name of the file or subdirectory. If `entry` is
920 a File or Directory, the `entryname` field overrides the value
921 of `basename` of the File or Directory object.
922
923 * Required when `entry` evaluates to file contents only
924 * Optional when `entry` evaluates to a File or Directory object with a `basename`
925 * Invalid when `entry` evaluates to an array of File or Directory objects.
926
927 If `entryname` is a relative path, it specifies a name within
928 the designated output directory. A relative path starting
929 with `../` or that resolves to location above the designated output directory is an error.
930
931 If `entryname` is an absolute path (starts with a slash `/`)
932 it is an error unless the following conditions are met:
933
934 * `DockerRequirement` is present in `requirements`
935 * The program is will run inside a software container
936 where, from the perspective of the program, the root
937 filesystem is not shared with any other user or
938 running program.
939
940 In this case, and the above conditions are met, then
941 `entryname` may specify the absolute path within the container
942 where the file or directory must be placed.
943
944 - name: entry
945 type: [string, Expression]
946 jsonldPredicate:
947 _id: cwl:entry
948 doc: |
949 If the value is a string literal or an expression which evaluates to a
950 string, a new text file must be created with the string as the file contents.
951
952 If the value is an expression that evaluates to a `File` or
953 `Directory` object, or an array of `File` or `Directory`
954 objects, this indicates the referenced file or directory
955 should be added to the designated output directory prior to
956 executing the tool.
957
958 If the value is an expression that evaluates to `null`,
959 nothing is added to the designated output directory, the entry
960 has no effect.
961
962 If the value is an expression that evaluates to some other
963 array, number, or object not consisting of `File` or
964 `Directory` objects, a new file must be created with the value
965 serialized to JSON text as the file contents. The JSON
966 serialization behavior should match the behavior of string
967 interpolation of [Parameter
968 references](#Parameter_references).
969
970 - name: writable
971 type: boolean?
972 default: false
973 doc: |
974 If true, the File or Directory (or array of Files or
975 Directories) declared in `entry` must be writable by the tool.
976
977 Changes to the file or directory must be isolated and not
978 visible by any other CommandLineTool process. This may be
979 implemented by making a copy of the original file or
980 directory.
981
982 Default false (files and directories read-only by default).
983
984 A directory marked as `writable: true` implies that all files and
985 subdirectories are recursively writable as well.
986
987 If `writable` is false, the file may be made available using a
988 bind mount or file system link to avoid unnecessary copying of
989 the input file. Command line tools may receive an error on
990 attempting to rename or delete files or directories that are
991 not explicitly marked as writable.
992
993
994 - name: InitialWorkDirRequirement
995 type: record
996 extends: ProcessRequirement
997 doc:
998 Define a list of files and subdirectories that must be staged by
999 the workflow platform prior to executing the command line tool.
1000
1001 Normally files are staged within the designated output directory.
1002 However, when running inside containers, files may be staged at
1003 arbitrary locations, see discussion for `Dirent.entryname`.
1004 Together with `DockerRequirement.dockerOutputDirectory` this it
1005 possible to control the locations of both input and output files
1006 when running in containers.
1007
1008 fields:
1009 - name: class
1010 type: string
1011 doc: InitialWorkDirRequirement
1012 jsonldPredicate:
1013 "_id": "@type"
1014 "_type": "@vocab"
1015 - name: listing
1016 type:
1017 - Expression
1018 - type: array
1019 items:
1020 - "null"
1021 - Dirent
1022 - Expression
1023 - File
1024 - Directory
1025 - type: array
1026 items:
1027 - File
1028 - Directory
1029 jsonldPredicate:
1030 _id: "cwl:listing"
1031 doc: |
1032 The list of files or subdirectories that must be staged prior
1033 to executing the command line tool.
1034
1035 Return type of each expression must validate as `["null",
1036 File, Directory, Dirent, {type: array, items: [File,
1037 Directory]}]`.
1038
1039 Each `File` or `Directory` that is returned by an Expression
1040 must be added to the designated output directory prior to
1041 executing the tool.
1042
1043 Each `Dirent` record that is listed or returned by an
1044 expression specifies a file to be created or staged in the
1045 designated output directory prior to executing the tool.
1046
1047 Expressions may return null, in which case they have no effect.
1048
1049 Files or Directories which are listed in the input parameters
1050 and appear in the `InitialWorkDirRequirement` listing must
1051 have their `path` set to their staged location. If the same
1052 File or Directory appears more than once in the
1053 `InitialWorkDirRequirement` listing, the implementation must
1054 choose exactly one value for `path`; how this value is chosen
1055 is undefined.
1056
1057
1058 - name: EnvVarRequirement
1059 type: record
1060 extends: ProcessRequirement
1061 doc: |
1062 Define a list of environment variables which will be set in the
1063 execution environment of the tool. See `EnvironmentDef` for details.
1064 fields:
1065 - name: class
1066 type: string
1067 doc: "Always 'EnvVarRequirement'"
1068 jsonldPredicate:
1069 "_id": "@type"
1070 "_type": "@vocab"
1071 - name: envDef
1072 type: EnvironmentDef[]
1073 doc: The list of environment variables.
1074 jsonldPredicate:
1075 mapSubject: envName
1076 mapPredicate: envValue
1077
1078
1079 - type: record
1080 name: ShellCommandRequirement
1081 extends: ProcessRequirement
1082 doc: |
1083 Modify the behavior of CommandLineTool to generate a single string
1084 containing a shell command line. Each item in the argument list must be
1085 joined into a string separated by single spaces and quoted to prevent
1086 intepretation by the shell, unless `CommandLineBinding` for that argument
1087 contains `shellQuote: false`. If `shellQuote: false` is specified, the
1088 argument is joined into the command string without quoting, which allows
1089 the use of shell metacharacters such as `|` for pipes.
1090 fields:
1091 - name: class
1092 type: string
1093 doc: "Always 'ShellCommandRequirement'"
1094 jsonldPredicate:
1095 "_id": "@type"
1096 "_type": "@vocab"
1097
1098
1099 - type: record
1100 name: ResourceRequirement
1101 extends: ProcessRequirement
1102 doc: |
1103 Specify basic hardware resource requirements.
1104
1105 "min" is the minimum amount of a resource that must be reserved to
1106 schedule a job. If "min" cannot be satisfied, the job should not
1107 be run.
1108
1109 "max" is the maximum amount of a resource that the job shall be
1110 allocated. If a node has sufficient resources, multiple jobs may
1111 be scheduled on a single node provided each job's "max" resource
1112 requirements are met. If a job attempts to exceed its resource
1113 allocation, an implementation may deny additional resources, which
1114 may result in job failure.
1115
1116 If both "min" and "max" are specified, an implementation may
1117 choose to allocate any amount between "min" and "max", with the
1118 actual allocation provided in the `runtime` object.
1119
1120 If "min" is specified but "max" is not, then "max" == "min"
1121 If "max" is specified by "min" is not, then "min" == "max".
1122
1123 It is an error if max < min.
1124
1125 It is an error if the value of any of these fields is negative.
1126
1127 If neither "min" nor "max" is specified for a resource, use the default values below.
1128
1129 fields:
1130 - name: class
1131 type: string
1132 doc: "Always 'ResourceRequirement'"
1133 jsonldPredicate:
1134 "_id": "@type"
1135 "_type": "@vocab"
1136 - name: coresMin
1137 type: ["null", long, float, Expression]
1138 doc: |
1139 Minimum reserved number of CPU cores (default is 1).
1140
1141 May be a fractional value to indicate to a scheduling
1142 algorithm that one core can be allocated to multiple
1143 jobs. For example, a value of 0.25 indicates that up to 4
1144 jobs may run in parallel on 1 core. A value of 1.25 means
1145 that up to 3 jobs can run on a 4 core system (4/1.25 ≈ 3).
1146
1147 Processes can only share a core allocation if the sum of each
1148 of their `ramMax`, `tmpdirMax`, and `outdirMax` requests also
1149 do not exceed the capacity of the node.
1150
1151 Processes sharing a core must have the same level of isolation
1152 (typically a container or VM) that they would normally.
1153
1154 The reported number of CPU cores reserved for the process,
1155 which is available to expressions on the CommandLineTool as
1156 `runtime.cores`, must be a non-zero integer, and may be
1157 calculated by rounding up the cores request to the next whole
1158 number.
1159
1160 Scheduling systems may allocate fractional CPU resources by
1161 setting quotas or scheduling weights. Scheduling systems that
1162 do not support fractional CPUs may round up the request to the
1163 next whole number.
1164
1165 - name: coresMax
1166 type: ["null", long, float, Expression]
1167 doc: |
1168 Maximum reserved number of CPU cores.
1169
1170 See `coresMin` for discussion about fractional CPU requests.
1171
1172 - name: ramMin
1173 type: ["null", long, float, Expression]
1174 doc: |
1175 Minimum reserved RAM in mebibytes (2**20) (default is 256)
1176
1177 May be a fractional value. If so, the actual RAM request must
1178 be rounded up to the next whole number. The reported amount of
1179 RAM reserved for the process, which is available to
1180 expressions on the CommandLineTool as `runtime.ram`, must be a
1181 non-zero integer.
1182
1183 - name: ramMax
1184 type: ["null", long, float, Expression]
1185 doc: |
1186 Maximum reserved RAM in mebibytes (2**20)
1187
1188 See `ramMin` for discussion about fractional RAM requests.
1189
1190 - name: tmpdirMin
1191 type: ["null", long, float, Expression]
1192 doc: |
1193 Minimum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) (default is 1024)
1194
1195 May be a fractional value. If so, the actual storage request
1196 must be rounded up to the next whole number. The reported
1197 amount of storage reserved for the process, which is available
1198 to expressions on the CommandLineTool as `runtime.tmpdirSize`,
1199 must be a non-zero integer.
1200
1201 - name: tmpdirMax
1202 type: ["null", long, float, Expression]
1203 doc: |
1204 Maximum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20)
1205
1206 See `tmpdirMin` for discussion about fractional storage requests.
1207
1208 - name: outdirMin
1209 type: ["null", long, float, Expression]
1210 doc: |
1211 Minimum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) (default is 1024)
1212
1213 May be a fractional value. If so, the actual storage request
1214 must be rounded up to the next whole number. The reported
1215 amount of storage reserved for the process, which is available
1216 to expressions on the CommandLineTool as `runtime.outdirSize`,
1217 must be a non-zero integer.
1218
1219 - name: outdirMax
1220 type: ["null", long, float, Expression]
1221 doc: |
1222 Maximum reserved filesystem based storage for the designated output directory, in mebibytes (2**20)
1223
1224 See `outdirMin` for discussion about fractional storage requests.
1225
1226
1227 - type: record
1228 name: WorkReuse
1229 extends: ProcessRequirement
1230 doc: |
1231 For implementations that support reusing output from past work (on
1232 the assumption that same code and same input produce same
1233 results), control whether to enable or disable the reuse behavior
1234 for a particular tool or step (to accomodate situations where that
1235 assumption is incorrect). A reused step is not executed but
1236 instead returns the same output as the original execution.
1237
1238 If `WorkReuse` is not specified, correct tools should assume it
1239 is enabled by default.
1240 fields:
1241 - name: class
1242 type: string
1243 doc: "Always 'WorkReuse'"
1244 jsonldPredicate:
1245 "_id": "@type"
1246 "_type": "@vocab"
1247 - name: enableReuse
1248 type: [boolean, Expression]
1249 default: true
1250
1251
1252 - type: record
1253 name: NetworkAccess
1254 extends: ProcessRequirement
1255 doc: |
1256 Indicate whether a process requires outgoing IPv4/IPv6 network
1257 access. Choice of IPv4 or IPv6 is implementation and site
1258 specific, correct tools must support both.
1259
1260 If `networkAccess` is false or not specified, tools must not
1261 assume network access, except for localhost (the loopback device).
1262
1263 If `networkAccess` is true, the tool must be able to make outgoing
1264 connections to network resources. Resources may be on a private
1265 subnet or the public Internet. However, implementations and sites
1266 may apply their own security policies to restrict what is
1267 accessible by the tool.
1268
1269 Enabling network access does not imply a publically routable IP
1270 address or the ability to accept inbound connections.
1271
1272 fields:
1273 - name: class
1274 type: string
1275 doc: "Always 'NetworkAccess'"
1276 jsonldPredicate:
1277 "_id": "@type"
1278 "_type": "@vocab"
1279 - name: networkAccess
1280 type: [boolean, Expression]
1281
1282 - name: InplaceUpdateRequirement
1283 type: record
1284 extends: cwl:ProcessRequirement
1285 doc: |
1286
1287 If `inplaceUpdate` is true, then an implementation supporting this
1288 feature may permit tools to directly update files with `writable:
1289 true` in InitialWorkDirRequirement. That is, as an optimization,
1290 files may be destructively modified in place as opposed to copied
1291 and updated.
1292
1293 An implementation must ensure that only one workflow step may
1294 access a writable file at a time. It is an error if a file which
1295 is writable by one workflow step file is accessed (for reading or
1296 writing) by any other workflow step running independently.
1297 However, a file which has been updated in a previous completed
1298 step may be used as input to multiple steps, provided it is
1299 read-only in every step.
1300
1301 Workflow steps which modify a file must produce the modified file
1302 as output. Downstream steps which futher process the file must
1303 use the output of previous steps, and not refer to a common input
1304 (this is necessary for both ordering and correctness).
1305
1306 Workflow authors should provide this in the `hints` section. The
1307 intent of this feature is that workflows produce the same results
1308 whether or not InplaceUpdateRequirement is supported by the
1309 implementation, and this feature is primarily available as an
1310 optimization for particular environments.
1311
1312 Users and implementers should be aware that workflows that
1313 destructively modify inputs may not be repeatable or reproducible.
1314 In particular, enabling this feature implies that WorkReuse should
1315 not be enabled.
1316
1317 fields:
1318 class:
1319 type: string
1320 doc: "Always 'InplaceUpdateRequirement'"
1321 jsonldPredicate:
1322 "_id": "@type"
1323 "_type": "@vocab"
1324 inplaceUpdate:
1325 type: boolean
1326
1327 - type: record
1328 name: ToolTimeLimit
1329 extends: ProcessRequirement
1330 doc: |
1331 Set an upper limit on the execution time of a CommandLineTool.
1332 A CommandLineTool whose execution duration exceeds the time
1333 limit may be preemptively terminated and considered failed.
1334 May also be used by batch systems to make scheduling decisions.
1335 The execution duration excludes external operations, such as
1336 staging of files, pulling a docker image etc, and only counts
1337 wall-time for the execution of the command line itself.
1338 fields:
1339 - name: class
1340 type: string
1341 doc: "Always 'ToolTimeLimit'"
1342 jsonldPredicate:
1343 "_id": "@type"
1344 "_type": "@vocab"
1345 - name: timelimit
1346 type: [long, Expression]
1347 doc: |
1348 The time limit, in seconds. A time limit of zero means no
1349 time limit. Negative time limits are an error.