Mercurial > repos > shellac > guppy_basecaller

comparison env/lib/python3.7/site-packages/cwltool/schemas/v1.1.0-dev1/CommandLineTool.yml @ 0:26e78fe6e8c4 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload commit c699937486c35866861690329de38ec1a5d9f783"
author shellac
date Sat, 02 May 2020 07:14:21 -0400
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:26e78fe6e8c4
1 saladVersion: v1.1
2 $base: "https://w3id.org/cwl/cwl#"
3
4 $namespaces:
5 cwl: "https://w3id.org/cwl/cwl#"
6
7 $graph:
8
9 - name: CommandLineToolDoc
10 type: documentation
11 doc:
12 - |
13 # Common Workflow Language (CWL) Command Line Tool Description, v1.1.0-dev1
14
15 This version:
16 * https://w3id.org/cwl/v1.1.0-dev1/
17
18 Current version:
19 * https://w3id.org/cwl/
20 - "\n\n"
21 - {$include: contrib.md}
22 - "\n\n"
23 - |
24 # Abstract
25
26 A Command Line Tool is a non-interactive executable program that reads
27 some input, performs a computation, and terminates after producing some
28 output. Command line programs are a flexible unit of code sharing and
29 reuse, unfortunately the syntax and input/output semantics among command
30 line programs is extremely heterogeneous. A common layer for describing
31 the syntax and semantics of programs can reduce this incidental
32 complexity by providing a consistent way to connect programs together.
33 This specification defines the Common Workflow Language (CWL) Command
34 Line Tool Description, a vendor-neutral standard for describing the
35 syntax and input/output semantics of command line programs.
36
37 - {$include: intro.md}
38
39 - |
40 ## Introduction to the CWL Command Line Tool standard v1.1.0-dev1
41
42 This specification represents the latest development release from the
43 CWL group. Since the v1.0 release, v1.1 introduces the
44 following updates to the CWL Command Line Tool standard.
45 Documents should use `cwlVersion: v1.1.0-dev1` to make use of new
46 syntax and features introduced in v1.1.0-dev1. Existing v1.0 documents
47 should be trivially updatable by changing `cwlVersion`, however
48 CWL documents that relied on previously undefined or
49 underspecified behavior may have slightly different behavior in
50 v1.1.0-dev1.
51
52 ## Changelog
53
54 * Clarify behavior around `ENTRYPOINT` and `CMD` in containers.
55 * Clarify documentation around `valueFrom` and `null` inputs.
56 * Default values for some fields are now expressed in the schema.
57 * When defining record types with `CommandInputRecordSchema`, fields of
58 type `File` may now include `format`, `loadContents`,
59 `secondaryFiles` and `streamable`.
60 * `CommandInputRecordSchema`, `CommandOutputRecordSchema`,
61 `CommandInputEnumSchema and `CommandInputArraySchema` now have an optional
62 `doc` field.
63 * `inputBinding` has been added as an optional field for
64 `CommandInputRecordSchema` (was previously in CWL `draft-3` but
65 disappeared in `v1.0`).
66 * Any `doc` field may be an array of strings in addition to the
67 previously allowed single string.
68 * Addition of `stdin` type shortcut for
69 [`CommandInputParameter`](#CommandInputParameter).
70 * Clarify that the designated output directory should be empty
71 except for files or directories specified using
72 [InitialWorkDirRequirement](#InitialWorkDirRequirement).
73 * Clarify semantics of `shellQuote`.
74 * Expressions are now allowed to evaluate to `null` or `Dirent` in
75 [InitialWorkDirRequirement.listing](#InitialWorkDirRequirement).
76 * Items in [InitialWorkDirRequirement.listing](#InitialWorkDirRequirement)
77 are now allowed to be `null` and `array<File | Directory>`.
78 * Clarify behavior of secondaryFiles on output.
79 * [Addition](#Requirements_and_hints) of `cwl:requirements` field to
80 input object documents.
81 * Clarify behavior of `glob` for absolute paths and symlinks.
82 * Clarify behavior of `glob` to include directories.
83 * `secondaryFiles` can now be explicitly marked as `required` or not.
84 * Clarify `CommandLineTool.arguments` documentation.
85 * Clarify that `runtime.outdir` and `runtime.tmpdir` must be distinct
86 directories.
87 * Clarify that unspecified details related to execution are left open to
88 the platform.
89 * Added `InputParameter.loadContents` field. Use of `loadContents` in
90 `InputBinding` is deprecated; it is preserved for v1.0 backwards
91 compatability and will be removed in CWL v2.0.
92 * [Added](#ToolTimeLimit) `ToolTimeLimit` feature, allows setting
93 an upper limit on the execution time of a CommandLineTool.
94 * [Added](#WorkReuse) `WorkReuse` feature, allowing to enable or disable the reuse
95 behavior for a particular tool or step for implementations that
96 support reusing output from past work.
97 * [Added](#NetworkAccess) `NetworkAccess` feature, allowing to indicate whether a
98 process requires outgoing network access.
99 * [Added](#InplaceUpdateRequirement) `InplaceUpdateRequirement` feature, allowing tools to directly
100 update files with `writable: true` in `InitialWorkDirRequirement`.
101 * [Added](#LoadListingRequirement) `LoadListingRequirement`
102 and [loadListing](#LoadContents) to control whether and how
103 `Directory` listings should be loaded for use in expressions.
104 * The position field of the [CommandLineBinding](#CommandLineBinding) can
105 now be calculated from a CWL Expression.
106 * The exit code of a CommandLineTool invocation is now
107 available to expressions in `outputEval` as `runtime.exitCode`
108 * [Better explain](#map) the `map<…>` notation that has existed since v1.0.
109 * Fixed schema error where the `type` field inside the `inputs` and
110 `outputs` field was incorrectly listed as optional.
111 * For multi-Process CWL documents, if no particular process is named then
112 the process with the `id` of `#main` is chosen.
113
114 See also the [CWL Workflow Description, v1.1.0-dev1 changelog](Workflow.html#Changelog).
115
116 ## Purpose
117
118 Standalone programs are a flexible and interoperable form of code reuse.
119 Unlike monolithic applications, applications and analysis workflows which
120 are composed of multiple separate programs can be written in multiple
121 languages and execute concurrently on multiple hosts. However, POSIX
122 does not dictate computer-readable grammar or semantics for program input
123 and output, resulting in extremely heterogeneous command line grammar and
124 input/output semantics among program. This is a particular problem in
125 distributed computing (multi-node compute clusters) and virtualized
126 environments (such as Docker containers) where it is often necessary to
127 provision resources such as input files before executing the program.
128
129 Often this gap is filled by hard coding program invocation and
130 implicitly assuming requirements will be met, or abstracting program
131 invocation with wrapper scripts or descriptor documents. Unfortunately,
132 where these approaches are application or platform specific it creates a
133 significant barrier to reproducibility and portability, as methods
134 developed for one platform must be manually ported to be used on new
135 platforms. Similarly it creates redundant work, as wrappers for popular
136 tools must be rewritten for each application or platform in use.
137
138 The Common Workflow Language Command Line Tool Description is designed to
139 provide a common standard description of grammar and semantics for
140 invoking programs used in data-intensive fields such as Bioinformatics,
141 Chemistry, Physics, Astronomy, and Statistics. This specification
142 attempts to define a precise data and execution model for Command Line Tools that
143 can be implemented on a variety of computing platforms, ranging from a
144 single workstation to cluster, grid, cloud, and high performance
145 computing platforms. Details related to execution of these programs not
146 laid out in this specification are open to interpretation by the computing
147 platform implementing this specification.
148
149 - {$include: concepts.md}
150 - {$include: invocation.md}
151
152
153 - type: record
154 name: EnvironmentDef
155 doc: |
156 Define an environment variable that will be set in the runtime environment
157 by the workflow platform when executing the command line tool. May be the
158 result of executing an expression, such as getting a parameter from input.
159 fields:
160 - name: envName
161 type: string
162 doc: The environment variable name
163 - name: envValue
164 type: [string, Expression]
165 doc: The environment variable value
166
167 - type: record
168 name: CommandLineBinding
169 extends: InputBinding
170 docParent: "#CommandInputParameter"
171 doc: |
172
173 When listed under `inputBinding` in the input schema, the term
174 "value" refers to the the corresponding value in the input object. For
175 binding objects listed in `CommandLineTool.arguments`, the term "value"
176 refers to the effective value after evaluating `valueFrom`.
177
178 The binding behavior when building the command line depends on the data
179 type of the value. If there is a mismatch between the type described by
180 the input schema and the effective value, such as resulting from an
181 expression evaluation, an implementation must use the data type of the
182 effective value.
183
184 - **string**: Add `prefix` and the string to the command line.
185
186 - **number**: Add `prefix` and decimal representation to command line.
187
188 - **boolean**: If true, add `prefix` to the command line. If false, add
189 nothing.
190
191 - **File**: Add `prefix` and the value of
192 [`File.path`](#File) to the command line.
193
194 - **Directory**: Add `prefix` and the value of
195 [`Directory.path`](#Directory) to the command line.
196
197 - **array**: If `itemSeparator` is specified, add `prefix` and the join
198 the array into a single string with `itemSeparator` separating the
199 items. Otherwise first add `prefix`, then recursively process
200 individual elements.
201 If the array is empty, it does not add anything to command line.
202
203 - **object**: Add `prefix` only, and recursively add object fields for
204 which `inputBinding` is specified.
205
206 - **null**: Add nothing.
207
208 fields:
209 - name: position
210 type: [ int, Expression, string, "null" ]
211 doc: |
212 The sorting key. Default position is 0. If the inputBinding is
213 associated with an input parameter, then the value of `self` in the
214 expression will be the value of the input parameter. Input parameter
215 defaults (as specified by the `InputParameter.default` field) must be
216 applied before evaluating the expression. Expressions must return a
217 single value of type int or a null.
218 - name: prefix
219 type: string?
220 doc: "Command line prefix to add before the value."
221 - name: separate
222 type: boolean?
223 default: true
224 doc: |
225 If true (default), then the prefix and value must be added as separate
226 command line arguments; if false, prefix and value must be concatenated
227 into a single command line argument.
228 - name: itemSeparator
229 type: string?
230 doc: |
231 Join the array elements into a single string with the elements
232 separated by by `itemSeparator`.
233 - name: valueFrom
234 type:
235 - "null"
236 - string
237 - Expression
238 jsonldPredicate: "cwl:valueFrom"
239 doc: |
240 If `valueFrom` is a constant string value, use this as the value and
241 apply the binding rules above.
242
243 If `valueFrom` is an expression, evaluate the expression to yield the
244 actual value to use to build the command line and apply the binding
245 rules above. If the inputBinding is associated with an input
246 parameter, the value of `self` in the expression will be the value of
247 the input parameter. Input parameter defaults (as specified by the
248 `InputParameter.default` field) must be applied before evaluating the
249 expression.
250
251 If the value of the associated input parameter is `null`, `valueFrom` is
252 not evaluated and nothing is added to the command line.
253
254 When a binding is part of the `CommandLineTool.arguments` field,
255 the `valueFrom` field is required.
256
257 - name: shellQuote
258 type: boolean?
259 default: true
260 doc: |
261 If `ShellCommandRequirement` is in the requirements for the current command,
262 this controls whether the value is quoted on the command line (default is true).
263 Use `shellQuote: false` to inject metacharacters for operations such as pipes.
264
265 If `shellQuote` is true or not provided, the implementation must not
266 permit interpretation of any shell metacharacters or directives.
267
268
269 - type: record
270 name: CommandOutputBinding
271 extends: LoadContents
272 doc: |
273 Describes how to generate an output parameter based on the files produced
274 by a CommandLineTool.
275
276 The output parameter value is generated by applying these operations in the
277 following order:
278
279 - glob
280 - loadContents
281 - outputEval
282 - secondaryFiles
283 fields:
284 - name: glob
285 type:
286 - "null"
287 - string
288 - Expression
289 - type: array
290 items: string
291 doc: |
292 Find files or directories relative to the output directory, using POSIX
293 glob(3) pathname matching. If an array is provided, find files or
294 directories that match any pattern in the array. If an expression is
295 provided, the expression must return a string or an array of strings,
296 which will then be evaluated as one or more glob patterns. Must only
297 match and return files/directories which actually exist.
298
299 If the value of glob is a relative path pattern (does not
300 begin with a slash '/') then it is resolved relative to the
301 output directory. If the value of the glob is an absolute
302 path pattern (it does begin with a slash '/') then it must
303 refer to a path within the output directory. It is an error
304 if any glob resolves to a path outside the output directory.
305 Specifically this means globs with relative paths containing
306 '..' or absolute paths that refer outside the output directory
307 are illegal.
308
309 A glob may match a path within the output directory which is
310 actually a symlink to another file. In this case, the
311 expected behavior is for the resulting File/Directory object to take the
312 `basename` (and corresponding `nameroot` and `nameext`) of the
313 symlink. The `location` of the File/Directory is implementation
314 dependent, but logically the File/Directory should have the same content
315 as the symlink target. Platforms may stage output files/directories to
316 cloud storage that lack the concept of a symlink. In
317 this case file content and directories may be duplicated, or (to avoid
318 duplication) the File/Directory `location` may refer to the symlink
319 target.
320
321 It is an error if a symlink in the output directory (or any
322 symlink in a chain of links) refers to any file or directory
323 that is not under an input or output directory.
324
325 Implementations may shut down a container before globbing
326 output, so globs and expressions must not assume access to the
327 container filesystem except for declared input and output.
328
329 - name: outputEval
330 type:
331 - "null"
332 - string
333 - Expression
334 doc: |
335 Evaluate an expression to generate the output value. If
336 `glob` was specified, the value of `self` must be an array
337 containing file objects that were matched. If no files were
338 matched, `self` must be a zero length array; if a single file
339 was matched, the value of `self` is an array of a single
340 element. Additionally, if `loadContents` is `true`, the File
341 objects must include up to the first 64 KiB of file contents
342 in the `contents` field. The exit code of the process is
343 available in the expression as `runtime.exitCode`.
344
345 - name: CommandLineBindable
346 type: record
347 fields:
348 inputBinding:
349 type: CommandLineBinding?
350 jsonldPredicate: "cwl:inputBinding"
351 doc: Describes how to turn this object into command line arguments.
352
353 - name: CommandInputRecordField
354 type: record
355 extends: [InputRecordField, CommandLineBindable]
356 specialize:
357 - specializeFrom: InputRecordSchema
358 specializeTo: CommandInputRecordSchema
359 - specializeFrom: InputEnumSchema
360 specializeTo: CommandInputEnumSchema
361 - specializeFrom: InputArraySchema
362 specializeTo: CommandInputArraySchema
363 - specializeFrom: InputBinding
364 specializeTo: CommandLineBinding
365
366
367 - name: CommandInputRecordSchema
368 type: record
369 extends: [InputRecordSchema, CommandInputSchema, CommandLineBindable]
370 specialize:
371 - specializeFrom: InputRecordField
372 specializeTo: CommandInputRecordField
373 - specializeFrom: InputBinding
374 specializeTo: CommandLineBinding
375
376
377 - name: CommandInputEnumSchema
378 type: record
379 extends: [InputEnumSchema, CommandInputSchema, CommandLineBindable]
380 specialize:
381 - specializeFrom: InputBinding
382 specializeTo: CommandLineBinding
383
384
385 - name: CommandInputArraySchema
386 type: record
387 extends: [InputArraySchema, CommandInputSchema, CommandLineBindable]
388 specialize:
389 - specializeFrom: InputRecordSchema
390 specializeTo: CommandInputRecordSchema
391 - specializeFrom: InputEnumSchema
392 specializeTo: CommandInputEnumSchema
393 - specializeFrom: InputArraySchema
394 specializeTo: CommandInputArraySchema
395 - specializeFrom: InputBinding
396 specializeTo: CommandLineBinding
397
398
399 - name: CommandOutputRecordField
400 type: record
401 extends: OutputRecordField
402 specialize:
403 - specializeFrom: OutputRecordSchema
404 specializeTo: CommandOutputRecordSchema
405 - specializeFrom: OutputEnumSchema
406 specializeTo: CommandOutputEnumSchema
407 - specializeFrom: OutputArraySchema
408 specializeTo: CommandOutputArraySchema
409 fields:
410 - name: outputBinding
411 type: CommandOutputBinding?
412 jsonldPredicate: "cwl:outputBinding"
413 doc: |
414 Describes how to generate this output object based on the files
415 produced by a CommandLineTool
416
417
418 - name: CommandOutputRecordSchema
419 type: record
420 extends: OutputRecordSchema
421 specialize:
422 - specializeFrom: OutputRecordField
423 specializeTo: CommandOutputRecordField
424
425
426 - name: CommandOutputEnumSchema
427 type: record
428 extends: OutputEnumSchema
429 specialize:
430 - specializeFrom: OutputRecordSchema
431 specializeTo: CommandOutputRecordSchema
432 - specializeFrom: OutputEnumSchema
433 specializeTo: CommandOutputEnumSchema
434 - specializeFrom: OutputArraySchema
435 specializeTo: CommandOutputArraySchema
436
437
438 - name: CommandOutputArraySchema
439 type: record
440 extends: OutputArraySchema
441 specialize:
442 - specializeFrom: OutputRecordSchema
443 specializeTo: CommandOutputRecordSchema
444 - specializeFrom: OutputEnumSchema
445 specializeTo: CommandOutputEnumSchema
446 - specializeFrom: OutputArraySchema
447 specializeTo: CommandOutputArraySchema
448
449
450 - type: record
451 name: CommandInputParameter
452 extends: InputParameter
453 doc: An input parameter for a CommandLineTool.
454 fields:
455 - name: type
456 type:
457 - CWLType
458 - stdin
459 - CommandInputRecordSchema
460 - CommandInputEnumSchema
461 - CommandInputArraySchema
462 - string
463 - type: array
464 items:
465 - CWLType
466 - CommandInputRecordSchema
467 - CommandInputEnumSchema
468 - CommandInputArraySchema
469 - string
470 jsonldPredicate:
471 "_id": "sld:type"
472 "_type": "@vocab"
473 refScope: 2
474 typeDSL: True
475 doc: |
476 Specify valid types of data that may be assigned to this parameter.
477 - name: inputBinding
478 type: CommandLineBinding?
479 doc: |
480 Describes how to turns the input parameters of a process into
481 command line arguments.
482 jsonldPredicate: "cwl:inputBinding"
483
484 - type: record
485 name: CommandOutputParameter
486 extends: OutputParameter
487 doc: An output parameter for a CommandLineTool.
488 fields:
489 - name: type
490 type:
491 - CWLType
492 - stdout
493 - stderr
494 - CommandOutputRecordSchema
495 - CommandOutputEnumSchema
496 - CommandOutputArraySchema
497 - string
498 - type: array
499 items:
500 - CWLType
501 - CommandOutputRecordSchema
502 - CommandOutputEnumSchema
503 - CommandOutputArraySchema
504 - string
505 jsonldPredicate:
506 "_id": "sld:type"
507 "_type": "@vocab"
508 refScope: 2
509 typeDSL: True
510 doc: |
511 Specify valid types of data that may be assigned to this parameter.
512 - name: outputBinding
513 type: CommandOutputBinding?
514 jsonldPredicate: "cwl:outputBinding"
515 doc: Describes how to generate this output object based on the files
516 produced by a CommandLineTool
517
518 - name: stdin
519 type: enum
520 symbols: [ "cwl:stdin" ]
521 docParent: "#CommandOutputParameter"
522 doc: |
523 Only valid as a `type` for a `CommandLineTool` input with no
524 `inputBinding` set. `stdin` must not be specified at the `CommandLineTool`
525 level.
526
527 The following
528 ```
529 inputs:
530 an_input_name:
531 type: stdin
532 ```
533 is equivalent to
534 ```
535 inputs:
536 an_input_name:
537 type: File
538 streamable: true
539
540 stdin: ${inputs.an_input_name.path}
541 ```
542
543 - name: stdout
544 type: enum
545 symbols: [ "cwl:stdout" ]
546 docParent: "#CommandOutputParameter"
547 doc: |
548 Only valid as a `type` for a `CommandLineTool` output with no
549 `outputBinding` set.
550
551 The following
552 ```
553 outputs:
554 an_output_name:
555 type: stdout
556
557 stdout: a_stdout_file
558 ```
559 is equivalent to
560 ```
561 outputs:
562 an_output_name:
563 type: File
564 streamable: true
565 outputBinding:
566 glob: a_stdout_file
567
568 stdout: a_stdout_file
569 ```
570
571 If there is no `stdout` name provided, a random filename will be created.
572 For example, the following
573 ```
574 outputs:
575 an_output_name:
576 type: stdout
577 ```
578 is equivalent to
579 ```
580 outputs:
581 an_output_name:
582 type: File
583 streamable: true
584 outputBinding:
585 glob: random_stdout_filenameABCDEFG
586
587 stdout: random_stdout_filenameABCDEFG
588 ```
589
590
591 - name: stderr
592 type: enum
593 symbols: [ "cwl:stderr" ]
594 docParent: "#CommandOutputParameter"
595 doc: |
596 Only valid as a `type` for a `CommandLineTool` output with no
597 `outputBinding` set.
598
599 The following
600 ```
601 outputs:
602 an_output_name:
603 type: stderr
604
605 stderr: a_stderr_file
606 ```
607 is equivalent to
608 ```
609 outputs:
610 an_output_name:
611 type: File
612 streamable: true
613 outputBinding:
614 glob: a_stderr_file
615
616 stderr: a_stderr_file
617 ```
618
619 If there is no `stderr` name provided, a random filename will be created.
620 For example, the following
621 ```
622 outputs:
623 an_output_name:
624 type: stderr
625 ```
626 is equivalent to
627 ```
628 outputs:
629 an_output_name:
630 type: File
631 streamable: true
632 outputBinding:
633 glob: random_stderr_filenameABCDEFG
634
635 stderr: random_stderr_filenameABCDEFG
636 ```
637
638
639 - type: record
640 name: CommandLineTool
641 extends: Process
642 documentRoot: true
643 specialize:
644 - specializeFrom: InputParameter
645 specializeTo: CommandInputParameter
646 - specializeFrom: OutputParameter
647 specializeTo: CommandOutputParameter
648 doc: |
649 This defines the schema of the CWL Command Line Tool Description document.
650
651 fields:
652 - name: class
653 jsonldPredicate:
654 "_id": "@type"
655 "_type": "@vocab"
656 type: string
657 - name: baseCommand
658 doc: |
659 Specifies the program to execute. If an array, the first element of
660 the array is the command to execute, and subsequent elements are
661 mandatory command line arguments. The elements in `baseCommand` must
662 appear before any command line bindings from `inputBinding` or
663 `arguments`.
664
665 If `baseCommand` is not provided or is an empty array, the first
666 element of the command line produced after processing `inputBinding` or
667 `arguments` must be used as the program to execute.
668
669 If the program includes a path separator character it must
670 be an absolute path, otherwise it is an error. If the program does not
671 include a path separator, search the `$PATH` variable in the runtime
672 environment of the workflow runner find the absolute path of the
673 executable.
674 type:
675 - string?
676 - string[]?
677 jsonldPredicate:
678 "_id": "cwl:baseCommand"
679 "_container": "@list"
680 - name: arguments
681 doc: |
682 Command line bindings which are not directly associated with input
683 parameters. If the value is a string, it is used as a string literal
684 argument. If it is an Expression, the result of the evaluation is used
685 as an argument.
686 type:
687 - "null"
688 - type: array
689 items: [string, Expression, CommandLineBinding]
690 jsonldPredicate:
691 "_id": "cwl:arguments"
692 "_container": "@list"
693 - name: stdin
694 type: ["null", string, Expression]
695 jsonldPredicate: "https://w3id.org/cwl/cwl#stdin"
696 doc: |
697 A path to a file whose contents must be piped into the command's
698 standard input stream.
699 - name: stderr
700 type: ["null", string, Expression]
701 jsonldPredicate: "https://w3id.org/cwl/cwl#stderr"
702 doc: |
703 Capture the command's standard error stream to a file written to
704 the designated output directory.
705
706 If `stderr` is a string, it specifies the file name to use.
707
708 If `stderr` is an expression, the expression is evaluated and must
709 return a string with the file name to use to capture stderr. If the
710 return value is not a string, or the resulting path contains illegal
711 characters (such as the path separator `/`) it is an error.
712 - name: stdout
713 type: ["null", string, Expression]
714 jsonldPredicate: "https://w3id.org/cwl/cwl#stdout"
715 doc: |
716 Capture the command's standard output stream to a file written to
717 the designated output directory.
718
719 If `stdout` is a string, it specifies the file name to use.
720
721 If `stdout` is an expression, the expression is evaluated and must
722 return a string with the file name to use to capture stdout. If the
723 return value is not a string, or the resulting path contains illegal
724 characters (such as the path separator `/`) it is an error.
725 - name: successCodes
726 type: int[]?
727 doc: |
728 Exit codes that indicate the process completed successfully.
729
730 - name: temporaryFailCodes
731 type: int[]?
732 doc: |
733 Exit codes that indicate the process failed due to a possibly
734 temporary condition, where executing the process with the same
735 runtime environment and inputs may produce different results.
736
737 - name: permanentFailCodes
738 type: int[]?
739 doc:
740 Exit codes that indicate the process failed due to a permanent logic
741 error, where executing the process with the same runtime environment and
742 same inputs is expected to always fail.
743
744
745 - type: record
746 name: DockerRequirement
747 extends: ProcessRequirement
748 doc: |
749 Indicates that a workflow component should be run in a
750 [Docker](http://docker.com) or Docker-compatible (such as
751 [Singularity](https://www.sylabs.io/) and [udocker](https://github.com/indigo-dc/udocker)) container environment and
752 specifies how to fetch or build the image.
753
754 If a CommandLineTool lists `DockerRequirement` under
755 `hints` (or `requirements`), it may (or must) be run in the specified Docker
756 container.
757
758 The platform must first acquire or install the correct Docker image as
759 specified by `dockerPull`, `dockerImport`, `dockerLoad` or `dockerFile`.
760
761 The platform must execute the tool in the container using `docker run` with
762 the appropriate Docker image and tool command line.
763
764 The workflow platform may provide input files and the designated output
765 directory through the use of volume bind mounts. The platform should rewrite
766 file paths in the input object to correspond to the Docker bind mounted
767 locations. That is, the platform should rewrite values in the parameter context
768 such as `runtime.outdir`, `runtime.tmpdir` and others to be valid paths
769 within the container. The platform must ensure that `runtime.outdir` and
770 `runtime.tmpdir` are distinct directories.
771
772 When running a tool contained in Docker, the workflow platform must not
773 assume anything about the contents of the Docker container, such as the
774 presence or absence of specific software, except to assume that the
775 generated command line represents a valid command within the runtime
776 environment of the container.
777
778 A container image may specify an
779 [ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)
780 and/or
781 [CMD](https://docs.docker.com/engine/reference/builder/#cmd).
782 Command line arguments will be appended after all elements of
783 ENTRYPOINT, and will override all elements specified using CMD (in
784 other words, CMD is only used when the CommandLineTool definition
785 produces an empty command line).
786
787 Use of implicit ENTRYPOINT or CMD are discouraged due to reproducibility
788 concerns of the implicit hidden execution point (For further discussion, see
789 [https://doi.org/10.12688/f1000research.15140.1](https://doi.org/10.12688/f1000research.15140.1)). Portable
790 CommandLineTool wrappers in which use of a container is optional must not rely on ENTRYPOINT or CMD.
791 CommandLineTools which do rely on ENTRYPOINT or CMD must list `DockerRequirement` in the
792 `requirements` section.
793
794 ## Interaction with other requirements
795
796 If [EnvVarRequirement](#EnvVarRequirement) is specified alongside a
797 DockerRequirement, the environment variables must be provided to Docker
798 using `--env` or `--env-file` and interact with the container's preexisting
799 environment as defined by Docker.
800
801 fields:
802 - name: class
803 type: string
804 doc: "Always 'DockerRequirement'"
805 jsonldPredicate:
806 "_id": "@type"
807 "_type": "@vocab"
808 - name: dockerPull
809 type: string?
810 doc: "Specify a Docker image to retrieve using `docker pull`."
811 - name: dockerLoad
812 type: string?
813 doc: "Specify a HTTP URL from which to download a Docker image using `docker load`."
814 - name: dockerFile
815 type: string?
816 doc: "Supply the contents of a Dockerfile which will be built using `docker build`."
817 - name: dockerImport
818 type: string?
819 doc: "Provide HTTP URL to download and gunzip a Docker images using `docker import."
820 - name: dockerImageId
821 type: string?
822 doc: |
823 The image id that will be used for `docker run`. May be a
824 human-readable image name or the image identifier hash. May be skipped
825 if `dockerPull` is specified, in which case the `dockerPull` image id
826 must be used.
827 - name: dockerOutputDirectory
828 type: string?
829 doc: |
830 Set the designated output directory to a specific location inside the
831 Docker container.
832
833
834 - type: record
835 name: SoftwareRequirement
836 extends: ProcessRequirement
837 doc: |
838 A list of software packages that should be configured in the environment of
839 the defined process.
840 fields:
841 - name: class
842 type: string
843 doc: "Always 'SoftwareRequirement'"
844 jsonldPredicate:
845 "_id": "@type"
846 "_type": "@vocab"
847 - name: packages
848 type: SoftwarePackage[]
849 doc: "The list of software to be configured."
850 jsonldPredicate:
851 mapSubject: package
852 mapPredicate: specs
853
854 - name: SoftwarePackage
855 type: record
856 fields:
857 - name: package
858 type: string
859 doc: |
860 The name of the software to be made available. If the name is
861 common, inconsistent, or otherwise ambiguous it should be combined with
862 one or more identifiers in the `specs` field.
863 - name: version
864 type: string[]?
865 doc: |
866 The (optional) versions of the software that are known to be
867 compatible.
868 - name: specs
869 type: string[]?
870 jsonldPredicate: {_type: "@id", noLinkCheck: true}
871 doc: |
872 One or more [IRI](https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier)s
873 identifying resources for installing or enabling the software named in
874 the `package` field. Implementations may provide resolvers which map
875 these software identifer IRIs to some configuration action; or they can
876 use only the name from the `package` field on a best effort basis.
877
878 For example, the IRI https://packages.debian.org/bowtie could
879 be resolved with `apt-get install bowtie`. The IRI
880 https://anaconda.org/bioconda/bowtie could be resolved with `conda
881 install -c bioconda bowtie`.
882
883 IRIs can also be system independent and used to map to a specific
884 software installation or selection mechanism.
885 Using [RRID](https://www.identifiers.org/rrid/) as an example:
886 https://identifiers.org/rrid/RRID:SCR_005476
887 could be fulfilled using the above mentioned Debian or bioconda
888 package, a local installation managed by [Environement Modules](http://modules.sourceforge.net/),
889 or any other mechanism the platform chooses. IRIs can also be from
890 identifer sources that are discipline specific yet still system
891 independent. As an example, the equivalent [ELIXIR Tools and Data
892 Service Registry](https://bio.tools) IRI to the previous RRID example is
893 https://bio.tools/tool/bowtie2/version/2.2.8.
894 If supported by a given registry, implementations are encouraged to
895 query these system independent sofware identifier IRIs directly for
896 links to packaging systems.
897
898 A site specific IRI can be listed as well. For example, an academic
899 computing cluster using Environement Modules could list the IRI
900 `https://hpc.example.edu/modules/bowtie-tbb/1.22` to indicate that
901 `module load bowtie-tbb/1.1.2` should be executed to make available
902 `bowtie` version 1.1.2 compiled with the TBB library prior to running
903 the accompanying Workflow or CommandLineTool. Note that the example IRI
904 is specific to a particular institution and computing environment as
905 the Environment Modules system does not have a common namespace or
906 standardized naming convention.
907
908 This last example is the least portable and should only be used if
909 mechanisms based off of the `package` field or more generic IRIs are
910 unavailable or unsuitable. While harmless to other sites, site specific
911 software IRIs should be left out of shared CWL descriptions to avoid
912 clutter.
913
914 - name: Dirent
915 type: record
916 doc: |
917 Define a file or subdirectory that must be placed in the designated output
918 directory prior to executing the command line tool. May be the result of
919 executing an expression, such as building a configuration file from a
920 template.
921 fields:
922 - name: entryname
923 type: ["null", string, Expression]
924 jsonldPredicate:
925 _id: cwl:entryname
926 doc: |
927 The name of the file or subdirectory to create in the output directory.
928 If `entry` is a File or Directory, the `entryname` field overrides the value
929 of `basename` of the File or Directory object. Optional.
930 - name: entry
931 type: [string, Expression]
932 jsonldPredicate:
933 _id: cwl:entry
934 doc: |
935 If the value is a string literal or an expression which evaluates to a
936 string, a new file must be created with the string as the file contents.
937
938 If the value is an expression that evaluates to a `File` object, this
939 indicates the referenced file should be added to the designated output
940 directory prior to executing the tool.
941
942 If the value is an expression that evaluates to a `Dirent` object, this
943 indicates that the File or Directory in `entry` should be added to the
944 designated output directory with the name in `entryname`.
945
946 If `writable` is false, the file may be made available using a bind
947 mount or file system link to avoid unnecessary copying of the input
948 file.
949 - name: writable
950 type: boolean?
951 default: false
952 doc: |
953 If true, the file or directory must be writable by the tool. Changes
954 to the file or directory must be isolated and not visible by any other
955 CommandLineTool process. This may be implemented by making a copy of
956 the original file or directory. Default false (files and directories
957 read-only by default).
958
959 A directory marked as `writable: true` implies that all files and
960 subdirectories are recursively writable as well.
961
962
963 - name: InitialWorkDirRequirement
964 type: record
965 extends: ProcessRequirement
966 doc:
967 Define a list of files and subdirectories that must be created by the
968 workflow platform in the designated output directory prior to executing the
969 command line tool.
970 fields:
971 - name: class
972 type: string
973 doc: InitialWorkDirRequirement
974 jsonldPredicate:
975 "_id": "@type"
976 "_type": "@vocab"
977 - name: listing
978 type:
979 - type: array
980 items:
981 - "null"
982 - File
983 - type: array
984 items:
985 - File
986 - Directory
987 - Directory
988 - Dirent
989 - string
990 - Expression
991 - string
992 - Expression
993 jsonldPredicate:
994 _id: "cwl:listing"
995 doc: |
996 The list of files or subdirectories that must be placed in the
997 designated output directory prior to executing the command line tool.
998
999 May be an expression. If so, the expression return value must validate as
1000 `{type: array, items: ["null", File, File[], Directory, Directory[], Dirent]}`.
1001
1002 Files or Directories which are listed in the input parameters and
1003 appear in the `InitialWorkDirRequirement` listing must have their
1004 `path` set to their staged location in the designated output directory.
1005 If the same File or Directory appears more than once in the
1006 `InitialWorkDirRequirement` listing, the implementation must choose
1007 exactly one value for `path`; how this value is chosen is undefined.
1008
1009
1010 - name: EnvVarRequirement
1011 type: record
1012 extends: ProcessRequirement
1013 doc: |
1014 Define a list of environment variables which will be set in the
1015 execution environment of the tool. See `EnvironmentDef` for details.
1016 fields:
1017 - name: class
1018 type: string
1019 doc: "Always 'EnvVarRequirement'"
1020 jsonldPredicate:
1021 "_id": "@type"
1022 "_type": "@vocab"
1023 - name: envDef
1024 type: EnvironmentDef[]
1025 doc: The list of environment variables.
1026 jsonldPredicate:
1027 mapSubject: envName
1028 mapPredicate: envValue
1029
1030
1031 - type: record
1032 name: ShellCommandRequirement
1033 extends: ProcessRequirement
1034 doc: |
1035 Modify the behavior of CommandLineTool to generate a single string
1036 containing a shell command line. Each item in the argument list must be
1037 joined into a string separated by single spaces and quoted to prevent
1038 intepretation by the shell, unless `CommandLineBinding` for that argument
1039 contains `shellQuote: false`. If `shellQuote: false` is specified, the
1040 argument is joined into the command string without quoting, which allows
1041 the use of shell metacharacters such as `|` for pipes.
1042 fields:
1043 - name: class
1044 type: string
1045 doc: "Always 'ShellCommandRequirement'"
1046 jsonldPredicate:
1047 "_id": "@type"
1048 "_type": "@vocab"
1049
1050
1051 - type: record
1052 name: ResourceRequirement
1053 extends: ProcessRequirement
1054 doc: |
1055 Specify basic hardware resource requirements.
1056
1057 "min" is the minimum amount of a resource that must be reserved to schedule
1058 a job. If "min" cannot be satisfied, the job should not be run.
1059
1060 "max" is the maximum amount of a resource that the job shall be permitted
1061 to use. If a node has sufficient resources, multiple jobs may be scheduled
1062 on a single node provided each job's "max" resource requirements are
1063 met. If a job attempts to exceed its "max" resource allocation, an
1064 implementation may deny additional resources, which may result in job
1065 failure.
1066
1067 If "min" is specified but "max" is not, then "max" == "min"
1068 If "max" is specified by "min" is not, then "min" == "max".
1069
1070 It is an error if max < min.
1071
1072 It is an error if the value of any of these fields is negative.
1073
1074 If neither "min" nor "max" is specified for a resource, an implementation may provide a default.
1075
1076 fields:
1077 - name: class
1078 type: string
1079 doc: "Always 'ResourceRequirement'"
1080 jsonldPredicate:
1081 "_id": "@type"
1082 "_type": "@vocab"
1083 - name: coresMin
1084 type: ["null", long, string, Expression]
1085 doc: Minimum reserved number of CPU cores
1086
1087 - name: coresMax
1088 type: ["null", int, string, Expression]
1089 doc: Maximum reserved number of CPU cores
1090
1091 - name: ramMin
1092 type: ["null", long, string, Expression]
1093 doc: Minimum reserved RAM in mebibytes (2**20)
1094
1095 - name: ramMax
1096 type: ["null", long, string, Expression]
1097 doc: Maximum reserved RAM in mebibytes (2**20)
1098
1099 - name: tmpdirMin
1100 type: ["null", long, string, Expression]
1101 doc: Minimum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20)
1102
1103 - name: tmpdirMax
1104 type: ["null", long, string, Expression]
1105 doc: Maximum reserved filesystem based storage for the designated temporary directory, in mebibytes (2**20)
1106
1107 - name: outdirMin
1108 type: ["null", long, string, Expression]
1109 doc: Minimum reserved filesystem based storage for the designated output directory, in mebibytes (2**20)
1110
1111 - name: outdirMax
1112 type: ["null", long, string, Expression]
1113 doc: Maximum reserved filesystem based storage for the designated output directory, in mebibytes (2**20)
1114
1115
1116 - type: record
1117 name: WorkReuse
1118 extends: ProcessRequirement
1119 doc: |
1120 For implementations that support reusing output from past work (on
1121 the assumption that same code and same input produce same
1122 results), control whether to enable or disable the reuse behavior
1123 for a particular tool or step (to accomodate situations where that
1124 assumption is incorrect). A reused step is not executed but
1125 instead returns the same output as the original execution.
1126
1127 If `enableReuse` is not specified, correct tools should assume it
1128 is enabled by default.
1129 fields:
1130 - name: class
1131 type: string
1132 doc: "Always 'WorkReuse'"
1133 jsonldPredicate:
1134 "_id": "@type"
1135 "_type": "@vocab"
1136 - name: enableReuse
1137 type: [boolean, string, Expression]
1138 default: true
1139
1140
1141 - type: record
1142 name: NetworkAccess
1143 extends: ProcessRequirement
1144 doc: |
1145 Indicate whether a process requires outgoing IPv4/IPv6 network
1146 access. Choice of IPv4 or IPv6 is implementation and site
1147 specific, correct tools must support both.
1148
1149 If `networkAccess` is false or not specified, tools must not
1150 assume network access, except for localhost (the loopback device).
1151
1152 If `networkAccess` is true, the tool must be able to make outgoing
1153 connections to network resources. Resources may be on a private
1154 subnet or the public Internet. However, implementations and sites
1155 may apply their own security policies to restrict what is
1156 accessible by the tool.
1157
1158 Enabling network access does not imply a publically routable IP
1159 address or the ability to accept inbound connections.
1160
1161 fields:
1162 - name: class
1163 type: string
1164 doc: "Always 'NetworkAccess'"
1165 jsonldPredicate:
1166 "_id": "@type"
1167 "_type": "@vocab"
1168 - name: networkAccess
1169 type: [boolean, string, Expression]
1170
1171 - name: InplaceUpdateRequirement
1172 type: record
1173 extends: cwl:ProcessRequirement
1174 doc: |
1175
1176 If `inplaceUpdate` is true, then an implementation supporting this
1177 feature may permit tools to directly update files with `writable:
1178 true` in InitialWorkDirRequirement. That is, as an optimization,
1179 files may be destructively modified in place as opposed to copied
1180 and updated.
1181
1182 An implementation must ensure that only one workflow step may
1183 access a writable file at a time. It is an error if a file which
1184 is writable by one workflow step file is accessed (for reading or
1185 writing) by any other workflow step running independently.
1186 However, a file which has been updated in a previous completed
1187 step may be used as input to multiple steps, provided it is
1188 read-only in every step.
1189
1190 Workflow steps which modify a file must produce the modified file
1191 as output. Downstream steps which futher process the file must
1192 use the output of previous steps, and not refer to a common input
1193 (this is necessary for both ordering and correctness).
1194
1195 Workflow authors should provide this in the `hints` section. The
1196 intent of this feature is that workflows produce the same results
1197 whether or not InplaceUpdateRequirement is supported by the
1198 implementation, and this feature is primarily available as an
1199 optimization for particular environments.
1200
1201 Users and implementers should be aware that workflows that
1202 destructively modify inputs may not be repeatable or reproducible.
1203 In particular, enabling this feature implies that WorkReuse should
1204 not be enabled.
1205
1206 fields:
1207 class:
1208 type: string
1209 doc: "Always 'InplaceUpdateRequirement'"
1210 jsonldPredicate:
1211 "_id": "@type"
1212 "_type": "@vocab"
1213 inplaceUpdate:
1214 type: boolean
1215
1216 - type: record
1217 name: ToolTimeLimit
1218 extends: ProcessRequirement
1219 doc: |
1220 Set an upper limit on the execution time of a CommandLineTool.
1221 A CommandLineTool whose execution duration exceeds the time
1222 limit may be preemptively terminated and considered failed.
1223 May also be used by batch systems to make scheduling decisions.
1224 The execution duration excludes external operations, such as
1225 staging of files, pulling a docker image etc, and only counts
1226 wall-time for the execution of the command line itself.
1227 fields:
1228 - name: class
1229 type: string
1230 doc: "Always 'ToolTimeLimit'"
1231 jsonldPredicate:
1232 "_id": "@type"
1233 "_type": "@vocab"
1234 - name: timelimit
1235 type: [long, string, Expression]
1236 doc: |
1237 The time limit, in seconds. A time limit of zero means no
1238 time limit. Negative time limits are an error.