comparison env/lib/python3.9/site-packages/cwltool/schemas/v1.2.0-dev3/CommandLineTool.yml @ 0:4f3585e2f14b draft default tip

"planemo upload commit 60cee0fc7c0cda8592644e1aad72851dec82c959"
author shellac
date Mon, 22 Mar 2021 18:12:50 +0000
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:4f3585e2f14b
1 saladVersion: v1.1
2 $base: "https://w3id.org/cwl/cwl#"
3
4 $namespaces:
5 cwl: "https://w3id.org/cwl/cwl#"
6
7 $graph:
8
9 - name: CommandLineToolDoc
10 type: documentation
11 doc:
12 - |
13 # Common Workflow Language (CWL) Command Line Tool Description, v1.2.0-dev3
14
15 This version:
16 * https://w3id.org/cwl/v1.2.0-dev3/
17
18 Current version:
19 * https://w3id.org/cwl/
20 - "\n\n"
21 - {$include: contrib.md}
22 - "\n\n"
23 - |
24 # Abstract
25
26 A Command Line Tool is a non-interactive executable program that reads
27 some input, performs a computation, and terminates after producing some
28 output. Command line programs are a flexible unit of code sharing and
29 reuse, unfortunately the syntax and input/output semantics among command
30 line programs is extremely heterogeneous. A common layer for describing
31 the syntax and semantics of programs can reduce this incidental
32 complexity by providing a consistent way to connect programs together.
33 This specification defines the Common Workflow Language (CWL) Command
34 Line Tool Description, a vendor-neutral standard for describing the
35 syntax and input/output semantics of command line programs.
36
37 - {$include: intro.md}
38
39 - |
40 ## Introduction to the CWL Command Line Tool standard v1.2.0-dev3
41
42 This specification represents the latest development release from the
43 CWL group. Since the v1.1 release, v1.2.0-dev3 introduces the
44 following updates to the CWL Command Line Tool standard.
45 Documents should use `cwlVersion: v1.2.0-dev3` to make use of new
46 syntax and features introduced in v1.2.0-dev3. Existing v1.1 documents
47 should be trivially updatable by changing `cwlVersion`, however
48 CWL documents that relied on previously undefined or
49 underspecified behavior may have slightly different behavior in
50 v1.2.0-dev3.
51
52 ## Changelog
53
54 * `coresMin` and `coresMax` of
55 [ResourceRequirement](#ResourceRequirement) may now request fractional CPUs.
56 * (CommandLineTool)[#CommandLineTool] can now express `intent`
57 with an identifier for the type of computational operation.
58 * Added conformance tests for order of operations evaluating `secondaryFiles` on input
59 and ensure that input and output secondaryFiles expressions can return a File object.
60
61 See also the [CWL Workflow Description, v1.2.0-dev3 changelog](Workflow.html#Changelog).
62
63 ## Purpose
64
65 Standalone programs are a flexible and interoperable form of code reuse.
66 Unlike monolithic applications, applications and analysis workflows which
67 are composed of multiple separate programs can be written in multiple
68 languages and execute concurrently on multiple hosts. However, POSIX
69 does not dictate computer-readable grammar or semantics for program input
70 and output, resulting in extremely heterogeneous command line grammar and
71 input/output semantics among program. This is a particular problem in
72 distributed computing (multi-node compute clusters) and virtualized
73 environments (such as Docker containers) where it is often necessary to
74 provision resources such as input files before executing the program.
75
76 Often this gap is filled by hard coding program invocation and
77 implicitly assuming requirements will be met, or abstracting program
78 invocation with wrapper scripts or descriptor documents. Unfortunately,
79 where these approaches are application or platform specific it creates a
80 significant barrier to reproducibility and portability, as methods
81 developed for one platform must be manually ported to be used on new
82 platforms. Similarly it creates redundant work, as wrappers for popular
83 tools must be rewritten for each application or platform in use.
84
85 The Common Workflow Language Command Line Tool Description is designed to
86 provide a common standard description of grammar and semantics for
87 invoking programs used in data-intensive fields such as Bioinformatics,
88 Chemistry, Physics, Astronomy, and Statistics. This specification
89 attempts to define a precise data and execution model for Command Line Tools that
90 can be implemented on a variety of computing platforms, ranging from a
91 single workstation to cluster, grid, cloud, and high performance
92 computing platforms. Details related to execution of these programs not
93 laid out in this specification are open to interpretation by the computing
94 platform implementing this specification.
95
96 - {$include: concepts.md}
97 - {$include: invocation.md}
98
99
100 - type: record
101 name: EnvironmentDef
102 doc: |
103 Define an environment variable that will be set in the runtime environment
104 by the workflow platform when executing the command line tool. May be the
105 result of executing an expression, such as getting a parameter from input.
106 fields:
107 - name: envName
108 type: string
109 doc: The environment variable name
110 - name: envValue
111 type: [string, Expression]
112 doc: The environment variable value
113
114 - type: record
115 name: CommandLineBinding
116 extends: InputBinding
117 docParent: "#CommandInputParameter"
118 doc: |
119
120 When listed under `inputBinding` in the input schema, the term
121 "value" refers to the the corresponding value in the input object. For
122 binding objects listed in `CommandLineTool.arguments`, the term "value"
123 refers to the effective value after evaluating `valueFrom`.
124
125 The binding behavior when building the command line depends on the data
126 type of the value. If there is a mismatch between the type described by
127 the input schema and the effective value, such as resulting from an
128 expression evaluation, an implementation must use the data type of the
129 effective value.
130
131 - **string**: Add `prefix` and the string to the command line.
132
133 - **number**: Add `prefix` and decimal representation to command line.
134
135 - **boolean**: If true, add `prefix` to the command line. If false, add
136 nothing.
137
138 - **File**: Add `prefix` and the value of
139 [`File.path`](#File) to the command line.
140
141 - **Directory**: Add `prefix` and the value of
142 [`Directory.path`](#Directory) to the command line.
143
144 - **array**: If `itemSeparator` is specified, add `prefix` and the join
145 the array into a single string with `itemSeparator` separating the
146 items. Otherwise first add `prefix`, then recursively process
147 individual elements.
148 If the array is empty, it does not add anything to command line.
149
150 - **object**: Add `prefix` only, and recursively add object fields for
151 which `inputBinding` is specified.
152
153 - **null**: Add nothing.
154
155 fields:
156 - name: position
157 type: [ "null", int, Expression ]
158 doc: |
159 The sorting key. Default position is 0. If the inputBinding is
160 associated with an input parameter, then the value of `self` in the
161 expression will be the value of the input parameter. Input parameter
162 defaults (as specified by the `InputParameter.default` field) must be
163 applied before evaluating the expression. Expressions must return a
164 single value of type int or a null.
165 - name: prefix
166 type: string?
167 doc: "Command line prefix to add before the value."
168 - name: separate
169 type: boolean?
170 default: true
171 doc: |
172 If true (default), then the prefix and value must be added as separate
173 command line arguments; if false, prefix and value must be concatenated
174 into a single command line argument.
175 - name: itemSeparator
176 type: string?
177 doc: |
178 Join the array elements into a single string with the elements
179 separated by by `itemSeparator`.
180 - name: valueFrom
181 type:
182 - "null"
183 - string
184 - Expression
185 jsonldPredicate: "cwl:valueFrom"
186 doc: |
187 If `valueFrom` is a constant string value, use this as the value and
188 apply the binding rules above.
189
190 If `valueFrom` is an expression, evaluate the expression to yield the
191 actual value to use to build the command line and apply the binding
192 rules above. If the inputBinding is associated with an input
193 parameter, the value of `self` in the expression will be the value of
194 the input parameter. Input parameter defaults (as specified by the
195 `InputParameter.default` field) must be applied before evaluating the
196 expression.
197
198 If the value of the associated input parameter is `null`, `valueFrom` is
199 not evaluated and nothing is added to the command line.
200
201 When a binding is part of the `CommandLineTool.arguments` field,
202 the `valueFrom` field is required.
203
204 - name: shellQuote
205 type: boolean?
206 default: true
207 doc: |
208 If `ShellCommandRequirement` is in the requirements for the current command,
209 this controls whether the value is quoted on the command line (default is true).
210 Use `shellQuote: false` to inject metacharacters for operations such as pipes.
211
212 If `shellQuote` is true or not provided, the implementation must not
213 permit interpretation of any shell metacharacters or directives.
214
215
216 - type: record
217 name: CommandOutputBinding
218 extends: LoadContents
219 doc: |
220 Describes how to generate an output parameter based on the files produced
221 by a CommandLineTool.
222
223 The output parameter value is generated by applying these operations in the
224 following order:
225
226 - glob
227 - loadContents
228 - outputEval
229 - secondaryFiles
230 fields:
231 - name: glob
232 type:
233 - "null"
234 - string
235 - Expression
236 - type: array
237 items: string
238 doc: |
239 Find files or directories relative to the output directory, using POSIX
240 glob(3) pathname matching. If an array is provided, find files or
241 directories that match any pattern in the array. If an expression is
242 provided, the expression must return a string or an array of strings,
243 which will then be evaluated as one or more glob patterns. Must only
244 match and return files/directories which actually exist.
245
246 If the value of glob is a relative path pattern (does not
247 begin with a slash '/') then it is resolved relative to the
248 output directory. If the value of the glob is an absolute
249 path pattern (it does begin with a slash '/') then it must
250 refer to a path within the output directory. It is an error
251 if any glob resolves to a path outside the output directory.
252 Specifically this means globs that resolve to paths outside the output
253 directory are illegal.
254
255 A glob may match a path within the output directory which is
256 actually a symlink to another file. In this case, the
257 expected behavior is for the resulting File/Directory object to take the
258 `basename` (and corresponding `nameroot` and `nameext`) of the
259 symlink. The `location` of the File/Directory is implementation
260 dependent, but logically the File/Directory should have the same content
261 as the symlink target. Platforms may stage output files/directories to
262 cloud storage that lack the concept of a symlink. In
263 this case file content and directories may be duplicated, or (to avoid
264 duplication) the File/Directory `location` may refer to the symlink
265 target.
266
267 It is an error if a symlink in the output directory (or any
268 symlink in a chain of links) refers to any file or directory
269 that is not under an input or output directory.
270
271 Implementations may shut down a container before globbing
272 output, so globs and expressions must not assume access to the
273 container filesystem except for declared input and output.
274
275 - name: outputEval
276 type: Expression?
277 doc: |
278 Evaluate an expression to generate the output value. If
279 `glob` was specified, the value of `self` must be an array
280 containing file objects that were matched. If no files were
281 matched, `self` must be a zero length array; if a single file
282 was matched, the value of `self` is an array of a single
283 element. Additionally, if `loadContents` is `true`, the File
284 objects must include up to the first 64 KiB of file contents
285 in the `contents` field. The exit code of the process is
286 available in the expression as `runtime.exitCode`.
287
288 - name: CommandLineBindable
289 type: record
290 fields:
291 inputBinding:
292 type: CommandLineBinding?
293 jsonldPredicate: "cwl:inputBinding"
294 doc: Describes how to turn this object into command line arguments.
295
296 - name: CommandInputRecordField
297 type: record
298 extends: [InputRecordField, CommandLineBindable]
299 specialize:
300 - specializeFrom: InputRecordSchema
301 specializeTo: CommandInputRecordSchema
302 - specializeFrom: InputEnumSchema
303 specializeTo: CommandInputEnumSchema
304 - specializeFrom: InputArraySchema
305 specializeTo: CommandInputArraySchema
306 - specializeFrom: InputBinding
307 specializeTo: CommandLineBinding
308
309
310 - name: CommandInputRecordSchema
311 type: record
312 extends: [InputRecordSchema, CommandInputSchema, CommandLineBindable]
313 specialize:
314 - specializeFrom: InputRecordField
315 specializeTo: CommandInputRecordField
316 - specializeFrom: InputBinding
317 specializeTo: CommandLineBinding
318
319
320 - name: CommandInputEnumSchema
321 type: record
322 extends: [InputEnumSchema, CommandInputSchema, CommandLineBindable]
323 specialize:
324 - specializeFrom: InputBinding
325 specializeTo: CommandLineBinding
326
327
328 - name: CommandInputArraySchema
329 type: record
330 extends: [InputArraySchema, CommandInputSchema, CommandLineBindable]
331 specialize:
332 - specializeFrom: InputRecordSchema
333 specializeTo: CommandInputRecordSchema
334 - specializeFrom: InputEnumSchema
335 specializeTo: CommandInputEnumSchema
336 - specializeFrom: InputArraySchema
337 specializeTo: CommandInputArraySchema
338 - specializeFrom: InputBinding
339 specializeTo: CommandLineBinding
340
341
342 - name: CommandOutputRecordField
343 type: record
344 extends: OutputRecordField
345 specialize:
346 - specializeFrom: OutputRecordSchema
347 specializeTo: CommandOutputRecordSchema
348 - specializeFrom: OutputEnumSchema
349 specializeTo: CommandOutputEnumSchema
350 - specializeFrom: OutputArraySchema
351 specializeTo: CommandOutputArraySchema
352 fields:
353 - name: outputBinding
354 type: CommandOutputBinding?
355 jsonldPredicate: "cwl:outputBinding"
356 doc: |
357 Describes how to generate this output object based on the files
358 produced by a CommandLineTool
359
360
361 - name: CommandOutputRecordSchema
362 type: record
363 extends: OutputRecordSchema
364 specialize:
365 - specializeFrom: OutputRecordField
366 specializeTo: CommandOutputRecordField
367
368
369 - name: CommandOutputEnumSchema
370 type: record
371 extends: OutputEnumSchema
372 specialize:
373 - specializeFrom: OutputRecordSchema
374 specializeTo: CommandOutputRecordSchema
375 - specializeFrom: OutputEnumSchema
376 specializeTo: CommandOutputEnumSchema
377 - specializeFrom: OutputArraySchema
378 specializeTo: CommandOutputArraySchema
379
380
381 - name: CommandOutputArraySchema
382 type: record
383 extends: OutputArraySchema
384 specialize:
385 - specializeFrom: OutputRecordSchema
386 specializeTo: CommandOutputRecordSchema
387 - specializeFrom: OutputEnumSchema
388 specializeTo: CommandOutputEnumSchema
389 - specializeFrom: OutputArraySchema
390 specializeTo: CommandOutputArraySchema
391
392
393 - type: record
394 name: CommandInputParameter
395 extends: InputParameter
396 doc: An input parameter for a CommandLineTool.
397 fields:
398 - name: type
399 type:
400 - CWLType
401 - stdin
402 - CommandInputRecordSchema
403 - CommandInputEnumSchema
404 - CommandInputArraySchema
405 - string
406 - type: array
407 items:
408 - CWLType
409 - CommandInputRecordSchema
410 - CommandInputEnumSchema
411 - CommandInputArraySchema
412 - string
413 jsonldPredicate:
414 "_id": "sld:type"
415 "_type": "@vocab"
416 refScope: 2
417 typeDSL: True
418 doc: |
419 Specify valid types of data that may be assigned to this parameter.
420 - name: inputBinding
421 type: CommandLineBinding?
422 doc: |
423 Describes how to turns the input parameters of a process into
424 command line arguments.
425 jsonldPredicate: "cwl:inputBinding"
426
427 - type: record
428 name: CommandOutputParameter
429 extends: OutputParameter
430 doc: An output parameter for a CommandLineTool.
431 fields:
432 - name: type
433 type:
434 - CWLType
435 - stdout
436 - stderr
437 - CommandOutputRecordSchema
438 - CommandOutputEnumSchema
439 - CommandOutputArraySchema
440 - string
441 - type: array
442 items:
443 - CWLType
444 - CommandOutputRecordSchema
445 - CommandOutputEnumSchema
446 - CommandOutputArraySchema
447 - string
448 jsonldPredicate:
449 "_id": "sld:type"
450 "_type": "@vocab"
451 refScope: 2
452 typeDSL: True
453 doc: |
454 Specify valid types of data that may be assigned to this parameter.
455 - name: outputBinding
456 type: CommandOutputBinding?
457 jsonldPredicate: "cwl:outputBinding"
458 doc: Describes how to generate this output object based on the files
459 produced by a CommandLineTool
460
461 - name: stdin
462 type: enum
463 symbols: [ "cwl:stdin" ]
464 docParent: "#CommandOutputParameter"
465 doc: |
466 Only valid as a `type` for a `CommandLineTool` input with no
467 `inputBinding` set. `stdin` must not be specified at the `CommandLineTool`
468 level.
469
470 The following
471 ```
472 inputs:
473 an_input_name:
474 type: stdin
475 ```
476 is equivalent to
477 ```
478 inputs:
479 an_input_name:
480 type: File
481 streamable: true
482
483 stdin: ${inputs.an_input_name.path}
484 ```
485
486 - name: stdout
487 type: enum
488 symbols: [ "cwl:stdout" ]
489 docParent: "#CommandOutputParameter"
490 doc: |
491 Only valid as a `type` for a `CommandLineTool` output with no
492 `outputBinding` set.
493
494 The following
495 ```
496 outputs:
497 an_output_name:
498 type: stdout
499
500 stdout: a_stdout_file
501 ```
502 is equivalent to
503 ```
504 outputs:
505 an_output_name:
506 type: File
507 streamable: true
508 outputBinding:
509 glob: a_stdout_file
510
511 stdout: a_stdout_file
512 ```
513
514 If there is no `stdout` name provided, a random filename will be created.
515 For example, the following
516 ```
517 outputs:
518 an_output_name:
519 type: stdout
520 ```
521 is equivalent to
522 ```
523 outputs:
524 an_output_name:
525 type: File
526 streamable: true
527 outputBinding:
528 glob: random_stdout_filenameABCDEFG
529
530 stdout: random_stdout_filenameABCDEFG
531 ```
532
533
534 - name: stderr
535 type: enum
536 symbols: [ "cwl:stderr" ]
537 docParent: "#CommandOutputParameter"
538 doc: |
539 Only valid as a `type` for a `CommandLineTool` output with no
540 `outputBinding` set.
541
542 The following
543 ```
544 outputs:
545 an_output_name:
546 type: stderr
547
548 stderr: a_stderr_file
549 ```
550 is equivalent to
551 ```
552 outputs:
553 an_output_name:
554 type: File
555 streamable: true
556 outputBinding:
557 glob: a_stderr_file
558
559 stderr: a_stderr_file
560 ```
561
562 If there is no `stderr` name provided, a random filename will be created.
563 For example, the following
564 ```
565 outputs:
566 an_output_name:
567 type: stderr
568 ```
569 is equivalent to
570 ```
571 outputs:
572 an_output_name:
573 type: File
574 streamable: true
575 outputBinding:
576 glob: random_stderr_filenameABCDEFG
577
578 stderr: random_stderr_filenameABCDEFG
579 ```
580
581
582 - type: record
583 name: CommandLineTool
584 extends: Process
585 documentRoot: true
586 specialize:
587 - specializeFrom: InputParameter
588 specializeTo: CommandInputParameter
589 - specializeFrom: OutputParameter
590 specializeTo: CommandOutputParameter
591 doc: |
592 This defines the schema of the CWL Command Line Tool Description document.
593
594 fields:
595 - name: class
596 jsonldPredicate:
597 "_id": "@type"
598 "_type": "@vocab"
599 type: string
600 - name: baseCommand
601 doc: |
602 Specifies the program to execute. If an array, the first element of
603 the array is the command to execute, and subsequent elements are
604 mandatory command line arguments. The elements in `baseCommand` must
605 appear before any command line bindings from `inputBinding` or
606 `arguments`.
607
608 If `baseCommand` is not provided or is an empty array, the first
609 element of the command line produced after processing `inputBinding` or
610 `arguments` must be used as the program to execute.
611
612 If the program includes a path separator character it must
613 be an absolute path, otherwise it is an error. If the program does not
614 include a path separator, search the `$PATH` variable in the runtime
615 environment of the workflow runner find the absolute path of the
616 executable.
617 type:
618 - string?
619 - string[]?
620 jsonldPredicate:
621 "_id": "cwl:baseCommand"
622 "_container": "@list"
623 - name: arguments
624 doc: |
625 Command line bindings which are not directly associated with input
626 parameters. If the value is a string, it is used as a string literal
627 argument. If it is an Expression, the result of the evaluation is used
628 as an argument.
629 type:
630 - "null"
631 - type: array
632 items: [string, Expression, CommandLineBinding]
633 jsonldPredicate:
634 "_id": "cwl:arguments"
635 "_container": "@list"
636 - name: stdin
637 type: ["null", string, Expression]
638 jsonldPredicate: "https://w3id.org/cwl/cwl#stdin"
639 doc: |
640 A path to a file whose contents must be piped into the command's
641 standard input stream.
642 - name: stderr
643 type: ["null", string, Expression]
644 jsonldPredicate: "https://w3id.org/cwl/cwl#stderr"
645 doc: |
646 Capture the command's standard error stream to a file written to
647 the designated output directory.
648
649 If `stderr` is a string, it specifies the file name to use.
650
651 If `stderr` is an expression, the expression is evaluated and must
652 return a string with the file name to use to capture stderr. If the
653 return value is not a string, or the resulting path contains illegal
654 characters (such as the path separator `/`) it is an error.
655 - name: stdout
656 type: ["null", string, Expression]
657 jsonldPredicate: "https://w3id.org/cwl/cwl#stdout"
658 doc: |
659 Capture the command's standard output stream to a file written to
660 the designated output directory.
661
662 If `stdout` is a string, it specifies the file name to use.
663
664 If `stdout` is an expression, the expression is evaluated and must
665 return a string with the file name to use to capture stdout. If the
666 return value is not a string, or the resulting path contains illegal
667 characters (such as the path separator `/`) it is an error.
668 - name: successCodes
669 type: int[]?
670 doc: |
671 Exit codes that indicate the process completed successfully.
672
673 - name: temporaryFailCodes
674 type: int[]?
675 doc: |
676 Exit codes that indicate the process failed due to a possibly
677 temporary condition, where executing the process with the same
678 runtime environment and inputs may produce different results.
679
680 - name: permanentFailCodes
681 type: int[]?
682 doc:
683 Exit codes that indicate the process failed due to a permanent logic
684 error, where executing the process with the same runtime environment and
685 same inputs is expected to always fail.
686
687
688 - type: record
689 name: DockerRequirement
690 extends: ProcessRequirement
691 doc: |
692 Indicates that a workflow component should be run in a
693 [Docker](http://docker.com) or Docker-compatible (such as
694 [Singularity](https://www.sylabs.io/) and [udocker](https://github.com/indigo-dc/udocker)) container environment and
695 specifies how to fetch or build the image.
696
697 If a CommandLineTool lists `DockerRequirement` under
698 `hints` (or `requirements`), it may (or must) be run in the specified Docker
699 container.
700
701 The platform must first acquire or install the correct Docker image as
702 specified by `dockerPull`, `dockerImport`, `dockerLoad` or `dockerFile`.
703
704 The platform must execute the tool in the container using `docker run` with
705 the appropriate Docker image and tool command line.
706
707 The workflow platform may provide input files and the designated output
708 directory through the use of volume bind mounts. The platform should rewrite
709 file paths in the input object to correspond to the Docker bind mounted
710 locations. That is, the platform should rewrite values in the parameter context
711 such as `runtime.outdir`, `runtime.tmpdir` and others to be valid paths
712 within the container. The platform must ensure that `runtime.outdir` and
713 `runtime.tmpdir` are distinct directories.
714
715 When running a tool contained in Docker, the workflow platform must not
716 assume anything about the contents of the Docker container, such as the
717 presence or absence of specific software, except to assume that the
718 generated command line represents a valid command within the runtime
719 environment of the container.
720
721 A container image may specify an
722 [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)
723 and/or
724 [CMD](https://docs.docker.com/engine/reference/builder/#cmd).
725 Command line arguments will be appended after all elements of
726 ENTRYPOINT, and will override all elements specified using CMD (in
727 other words, CMD is only used when the CommandLineTool definition
728 produces an empty command line).
729
730 Use of implicit ENTRYPOINT or CMD are discouraged due to reproducibility
731 concerns of the implicit hidden execution point (For further discussion, see
732 [https://doi.org/10.12688/f1000research.15140.1](https://doi.org/10.12688/f1000research.15140.1)). Portable
733 CommandLineTool wrappers in which use of a container is optional must not rely on ENTRYPOINT or CMD.
734 CommandLineTools which do rely on ENTRYPOINT or CMD must list `DockerRequirement` in the
735 `requirements` section.
736
737 ## Interaction with other requirements
738
739 If [EnvVarRequirement](#EnvVarRequirement) is specified alongside a
740 DockerRequirement, the environment variables must be provided to Docker
741 using `--env` or `--env-file` and interact with the container's preexisting
742 environment as defined by Docker.
743
744 fields:
745 - name: class
746 type: string
747 doc: "Always 'DockerRequirement'"
748 jsonldPredicate:
749 "_id": "@type"
750 "_type": "@vocab"
751 - name: dockerPull
752 type: string?
753 doc: |
754 Specify a Docker image to retrieve using `docker pull`. Can contain the
755 immutable digest to ensure an exact container is used:
756 `dockerPull: ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2`
757 - name: dockerLoad
758 type: string?
759 doc: "Specify a HTTP URL from which to download a Docker image using `docker load`."
760 - name: dockerFile
761 type: string?
762 doc: "Supply the contents of a Dockerfile which will be built using `docker build`."
763 - name: dockerImport
764 type: string?
765 doc: "Provide HTTP URL to download and gunzip a Docker images using `docker import."
766 - name: dockerImageId
767 type: string?
768 doc: |
769 The image id that will be used for `docker run`. May be a
770 human-readable image name or the image identifier hash. May be skipped
771 if `dockerPull` is specified, in which case the `dockerPull` image id
772 must be used.
773 - name: dockerOutputDirectory
774 type: string?
775 doc: |
776 Set the designated output directory to a specific location inside the
777 Docker container.
778
779
780 - type: record
781 name: SoftwareRequirement
782 extends: ProcessRequirement
783 doc: |
784 A list of software packages that should be configured in the environment of
785 the defined process.
786 fields:
787 - name: class
788 type: string
789 doc: "Always 'SoftwareRequirement'"
790 jsonldPredicate:
791 "_id": "@type"
792 "_type": "@vocab"
793 - name: packages
794 type: SoftwarePackage[]
795 doc: "The list of software to be configured."
796 jsonldPredicate:
797 mapSubject: package
798 mapPredicate: specs
799
800 - name: SoftwarePackage
801 type: record
802 fields:
803 - name: package
804 type: string
805 doc: |
806 The name of the software to be made available. If the name is
807 common, inconsistent, or otherwise ambiguous it should be combined with
808 one or more identifiers in the `specs` field.
809 - name: version
810 type: string[]?
811 doc: |
812 The (optional) versions of the software that are known to be
813 compatible.
814 - name: specs
815 type: string[]?
816 jsonldPredicate: {_type: "@id", noLinkCheck: true}
817 doc: |
818 One or more [IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier)s
819 identifying resources for installing or enabling the software named in
820 the `package` field. Implementations may provide resolvers which map
821 these software identifer IRIs to some configuration action; or they can
822 use only the name from the `package` field on a best effort basis.
823
824 For example, the IRI https://packages.debian.org/bowtie could
825 be resolved with `apt-get install bowtie`. The IRI
826 https://anaconda.org/bioconda/bowtie could be resolved with `conda
827 install -c bioconda bowtie`.
828
829 IRIs can also be system independent and used to map to a specific
830 software installation or selection mechanism.
831 Using [RRID](https://www.identifiers.org/rrid/) as an example:
832 https://identifiers.org/rrid/RRID:SCR_005476
833 could be fulfilled using the above mentioned Debian or bioconda
834 package, a local installation managed by [Environement Modules](http://modules.sourceforge.net/),
835 or any other mechanism the platform chooses. IRIs can also be from
836 identifer sources that are discipline specific yet still system
837 independent. As an example, the equivalent [ELIXIR Tools and Data
838 Service Registry](https://bio.tools) IRI to the previous RRID example is
839 https://bio.tools/tool/bowtie2/version/2.2.8.
840 If supported by a given registry, implementations are encouraged to
841 query these system independent sofware identifier IRIs directly for
842 links to packaging systems.
843
844 A site specific IRI can be listed as well. For example, an academic
845 computing cluster using Environement Modules could list the IRI
846 `https://hpc.example.edu/modules/bowtie-tbb/1.22` to indicate that
847 `module load bowtie-tbb/1.1.2` should be executed to make available
848 `bowtie` version 1.1.2 compiled with the TBB library prior to running
849 the accompanying Workflow or CommandLineTool. Note that the example IRI
850 is specific to a particular institution and computing environment as
851 the Environment Modules system does not have a common namespace or
852 standardized naming convention.
853
854 This last example is the least portable and should only be used if
855 mechanisms based off of the `package` field or more generic IRIs are
856 unavailable or unsuitable. While harmless to other sites, site specific
857 software IRIs should be left out of shared CWL descriptions to avoid
858 clutter.
859
860 - name: Dirent
861 type: record
862 doc: |
863 Define a file or subdirectory that must be placed in the designated output
864 directory prior to executing the command line tool. May be the result of
865 executing an expression, such as building a configuration file from a
866 template.
867 fields:
868 - name: entryname
869 type: ["null", string, Expression]
870 jsonldPredicate:
871 _id: cwl:entryname
872 doc: |
873 The name of the file or subdirectory to create in the output directory.
874 If `entry` is a File or Directory, the `entryname` field overrides the value
875 of `basename` of the File or Directory object. Optional.
876 - name: entry
877 type: [string, Expression]
878 jsonldPredicate:
879 _id: cwl:entry
880 doc: |
881 If the value is a string literal or an expression which evaluates to a
882 string, a new file must be created with the string as the file contents.
883
884 If the value is an expression that evaluates to a `File` or `Directory`
885 object, this indicates the referenced file or directory should be added
886 to the designated output directory prior to executing the tool.
887
888 If the value is an expression that evaluates to an array of `File` or
889 `Directory` objects, this indicates the referenced files or directories
890 should be added to the designated output directory prior to executing
891 the tool.
892
893 If the value is an expression that evaluates to a `Dirent` object, this
894 indicates that the File or Directory in `entry` should be added to the
895 designated output directory with the name in `entryname`.
896
897 If the value is an expression that evaluates to `null`, nothing is added
898 to the designated output directory from this entry.
899
900 If `writable` is false, the file may be made available using a bind
901 mount or file system link to avoid unnecessary copying of the input
902 file.
903 - name: writable
904 type: boolean?
905 default: false
906 doc: |
907 If true, the file or directory must be writable by the tool. Changes
908 to the file or directory must be isolated and not visible by any other
909 CommandLineTool process. This may be implemented by making a copy of
910 the original file or directory. Default false (files and directories
911 read-only by default).
912
913 A directory marked as `writable: true` implies that all files and
914 subdirectories are recursively writable as well.
915
916
917 - name: InitialWorkDirRequirement
918 type: record
919 extends: ProcessRequirement
920 doc:
921 Define a list of files and subdirectories that must be created by the
922 workflow platform in the designated output directory prior to executing the
923 command line tool.
924 fields:
925 - name: class
926 type: string
927 doc: InitialWorkDirRequirement
928 jsonldPredicate:
929 "_id": "@type"
930 "_type": "@vocab"
931 - name: listing
932 type:
933 - type: array
934 items:
935 - "null"
936 - File
937 - type: array
938 items:
939 - File
940 - Directory
941 - Directory
942 - Dirent
943 - Expression
944 - Expression
945 jsonldPredicate:
946 _id: "cwl:listing"
947 doc: |
948 The list of files or subdirectories that must be placed in the
949 designated output directory prior to executing the command line tool.
950
951 May be an expression. If so, the expression return value must validate as
952 `{type: array, items: ["null", File, Directory, Dirent, {type: array, items: [File, Directory]}]}`.
953
954 Files or Directories which are listed in the input parameters and
955 appear in the `InitialWorkDirRequirement` listing must have their
956 `path` set to their staged location in the designated output directory.
957 If the same File or Directory appears more than once in the
958 `InitialWorkDirRequirement` listing, the implementation must choose
959 exactly one value for `path`; how this value is chosen is undefined.
960
961
962 - name: EnvVarRequirement
963 type: record
964 extends: ProcessRequirement
965 doc: |
966 Define a list of environment variables which will be set in the
967 execution environment of the tool. See `EnvironmentDef` for details.
968 fields:
969 - name: class
970 type: string
971 doc: "Always 'EnvVarRequirement'"
972 jsonldPredicate:
973 "_id": "@type"
974 "_type": "@vocab"
975 - name: envDef
976 type: EnvironmentDef[]
977 doc: The list of environment variables.
978 jsonldPredicate:
979 mapSubject: envName
980 mapPredicate: envValue
981
982
983 - type: record
984 name: ShellCommandRequirement
985 extends: ProcessRequirement
986 doc: |
987 Modify the behavior of CommandLineTool to generate a single string
988 containing a shell command line. Each item in the argument list must be
989 joined into a string separated by single spaces and quoted to prevent
990 intepretation by the shell, unless `CommandLineBinding` for that argument
991 contains `shellQuote: false`. If `shellQuote: false` is specified, the
992 argument is joined into the command string without quoting, which allows
993 the use of shell metacharacters such as `|` for pipes.
994 fields:
995 - name: class
996 type: string
997 doc: "Always 'ShellCommandRequirement'"
998 jsonldPredicate:
999 "_id": "@type"
1000 "_type": "@vocab"
1001
1002
1003 - type: record
1004 name: ResourceRequirement
1005 extends: ProcessRequirement
1006 doc: |
1007 Specify basic hardware resource requirements.
1008
1009 "min" is the minimum amount of a resource that must be reserved to schedule
1010 a job. If "min" cannot be satisfied, the job should not be run.
1011
1012 "max" is the maximum amount of a resource that the job shall be permitted
1013 to use. If a node has sufficient resources, multiple jobs may be scheduled
1014 on a single node provided each job's "max" resource requirements are
1015 met. If a job attempts to exceed its "max" resource allocation, an
1016 implementation may deny additional resources, which may result in job
1017 failure.
1018
1019 If "min" is specified but "max" is not, then "max" == "min"
1020 If "max" is specified by "min" is not, then "min" == "max".
1021
1022 It is an error if max < min.
1023
1024 It is an error if the value of any of these fields is negative.
1025
1026 If neither "min" nor "max" is specified for a resource, use the default values below.
1027
1028 fields:
1029 - name: class
1030 type: string
1031 doc: "Always 'ResourceRequirement'"
1032 jsonldPredicate:
1033 "_id": "@type"
1034 "_type": "@vocab"
1035 - name: coresMin
1036 type: ["null", long, float, Expression]
1037 doc: |
1038 Minimum reserved number of CPU cores (default is 1).
1039
1040 May be a fractional value to indicate to a scheduling
1041 algorithm that one core can be allocated to multiple
1042 jobs. For example, a value of 0.25 indicates that up to 4
1043 jobs may run in parallel on 1 core. A value of 1.25 means
1044 that up to 3 jobs can run on a 4 core system (4/1.25 ≈ 3).
1045
1046 Processes can only share a core allocation if the sum of each
1047 of their `ramMax`, `tmpdirMax`, and `outdirMax` requests also
1048 do not exceed the capacity of the node.
1049
1050 Processes sharing a core must have the same level of isolation
1051 (typically a container or VM) that they would normally.
1052
1053 The reported number of CPU cores reserved for the process,
1054 which is available to expressions on the CommandLineTool as
1055 `runtime.cores`, must be a non-zero integer, and may be
1056 calculated by rounding up the cores request to the next whole
1057 number.
1058
1059 Scheduling systems may allocate fractional CPU resources by
1060 setting quotas or scheduling weights. Scheduling systems that
1061 do not support fractional CPUs may round up the request to the
1062 next whole number.
1063
1064 - name: coresMax
1065 type: ["null", long, float, Expression]
1066 doc: |
1067 Maximum reserved number of CPU cores.
1068
1069 See `coresMin` for discussion about fractional CPU requests.
1070
1071 - name: ramMin
1072 type: ["null", long, Expression]
1073 doc: Minimum reserved RAM in mebibytes (2**20) (default is 256)
1074
1075 - name: ramMax
1076 type: ["null", long, Expression]
1077 doc: Maximum reserved RAM in mebibytes (2**20)
1078
1079 - name: tmpdirMin
1080 type: ["null", long, Expression]
1081 doc: Minimum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20) (default is 1024)
1082
1083 - name: tmpdirMax
1084 type: ["null", long, Expression]
1085 doc: Maximum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20)
1086
1087 - name: outdirMin
1088 type: ["null", long, Expression]
1089 doc: Minimum reserved filesystem based storage for the designated output directory, in mebibytes (2**20) (default is 1024)
1090
1091 - name: outdirMax
1092 type: ["null", long, Expression]
1093 doc: Maximum reserved filesystem based storage for the designated output directory, in mebibytes (2**20)
1094
1095
1096 - type: record
1097 name: WorkReuse
1098 extends: ProcessRequirement
1099 doc: |
1100 For implementations that support reusing output from past work (on
1101 the assumption that same code and same input produce same
1102 results), control whether to enable or disable the reuse behavior
1103 for a particular tool or step (to accomodate situations where that
1104 assumption is incorrect). A reused step is not executed but
1105 instead returns the same output as the original execution.
1106
1107 If `WorkReuse` is not specified, correct tools should assume it
1108 is enabled by default.
1109 fields:
1110 - name: class
1111 type: string
1112 doc: "Always 'WorkReuse'"
1113 jsonldPredicate:
1114 "_id": "@type"
1115 "_type": "@vocab"
1116 - name: enableReuse
1117 type: [boolean, Expression]
1118 default: true
1119
1120
1121 - type: record
1122 name: NetworkAccess
1123 extends: ProcessRequirement
1124 doc: |
1125 Indicate whether a process requires outgoing IPv4/IPv6 network
1126 access. Choice of IPv4 or IPv6 is implementation and site
1127 specific, correct tools must support both.
1128
1129 If `networkAccess` is false or not specified, tools must not
1130 assume network access, except for localhost (the loopback device).
1131
1132 If `networkAccess` is true, the tool must be able to make outgoing
1133 connections to network resources. Resources may be on a private
1134 subnet or the public Internet. However, implementations and sites
1135 may apply their own security policies to restrict what is
1136 accessible by the tool.
1137
1138 Enabling network access does not imply a publically routable IP
1139 address or the ability to accept inbound connections.
1140
1141 fields:
1142 - name: class
1143 type: string
1144 doc: "Always 'NetworkAccess'"
1145 jsonldPredicate:
1146 "_id": "@type"
1147 "_type": "@vocab"
1148 - name: networkAccess
1149 type: [boolean, Expression]
1150
1151 - name: InplaceUpdateRequirement
1152 type: record
1153 extends: cwl:ProcessRequirement
1154 doc: |
1155
1156 If `inplaceUpdate` is true, then an implementation supporting this
1157 feature may permit tools to directly update files with `writable:
1158 true` in InitialWorkDirRequirement. That is, as an optimization,
1159 files may be destructively modified in place as opposed to copied
1160 and updated.
1161
1162 An implementation must ensure that only one workflow step may
1163 access a writable file at a time. It is an error if a file which
1164 is writable by one workflow step file is accessed (for reading or
1165 writing) by any other workflow step running independently.
1166 However, a file which has been updated in a previous completed
1167 step may be used as input to multiple steps, provided it is
1168 read-only in every step.
1169
1170 Workflow steps which modify a file must produce the modified file
1171 as output. Downstream steps which futher process the file must
1172 use the output of previous steps, and not refer to a common input
1173 (this is necessary for both ordering and correctness).
1174
1175 Workflow authors should provide this in the `hints` section. The
1176 intent of this feature is that workflows produce the same results
1177 whether or not InplaceUpdateRequirement is supported by the
1178 implementation, and this feature is primarily available as an
1179 optimization for particular environments.
1180
1181 Users and implementers should be aware that workflows that
1182 destructively modify inputs may not be repeatable or reproducible.
1183 In particular, enabling this feature implies that WorkReuse should
1184 not be enabled.
1185
1186 fields:
1187 class:
1188 type: string
1189 doc: "Always 'InplaceUpdateRequirement'"
1190 jsonldPredicate:
1191 "_id": "@type"
1192 "_type": "@vocab"
1193 inplaceUpdate:
1194 type: boolean
1195
1196 - type: record
1197 name: ToolTimeLimit
1198 extends: ProcessRequirement
1199 doc: |
1200 Set an upper limit on the execution time of a CommandLineTool.
1201 A CommandLineTool whose execution duration exceeds the time
1202 limit may be preemptively terminated and considered failed.
1203 May also be used by batch systems to make scheduling decisions.
1204 The execution duration excludes external operations, such as
1205 staging of files, pulling a docker image etc, and only counts
1206 wall-time for the execution of the command line itself.
1207 fields:
1208 - name: class
1209 type: string
1210 doc: "Always 'ToolTimeLimit'"
1211 jsonldPredicate:
1212 "_id": "@type"
1213 "_type": "@vocab"
1214 - name: timelimit
1215 type: [long, Expression]
1216 doc: |
1217 The time limit, in seconds. A time limit of zero means no
1218 time limit. Negative time limits are an error.