Mercurial > repos > artbio > concatenate_multiple_datasets

comparison catWrapper.xml @ 6:4554fa330d3d draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/ARTbio/tools-artbio/tree/master/tools/concat_multi_datasets commit 4df03fd2d6fbf17a451256c0fb9d30932fc9d637
author artbio
date Mon, 24 Jun 2019 04:02:53 -0400
parents 99a5ed06b86c
children 55cf9d9defd1
comparison
equal deleted inserted replaced
5:99a5ed06b86c 6:4554fa330d3d
1 <tool id="cat_multi_datasets" name="Concatenate multiple datasets" version="1.2.0"> 1 <tool id="cat_multi_datasets" name="Concatenate multiple datasets" version="1.4.0">
2 <description>tail-to-head by specifying how</description> 2 <description>tail-to-head by specifying how</description>
3 <command><![CDATA[ 3 <command><![CDATA[
4 #if $headers == 0: 4 #if $headers == 0:
5 #set $concat_command = "cat" 5 #set $concat_command = "cat"
6 #else: 6 #else:
7 #set $concat_command = 'tail -q -n +'+ str(int($headers)+1) 7 #set $concat_command = 'tail -q -n +'+ str(int($headers)+1)
8 #end if 8 #end if
9 printf "Concatenating files:\n" &&
9 #if $global_condition.input_type == "singles": 10 #if $global_condition.input_type == "singles":
10 #if $dataset_names == "No": 11 #if $dataset_names == "No":
11 $concat_command
12 #for $file in $global_condition.inputs 12 #for $file in $global_condition.inputs
13 '$file' 13 printf "${file.element_identifier}..." &&
14 #if $file.ext[-2:] == "gz" and $headers != 0:
15 gzip -dc '$file' | $concat_command | gzip -c >> '$out_file1' &&
16 #else:
17 $concat_command '$file' >> '$out_file1' &&
18 #end if
19 printf "Done\n" &&
14 #end for 20 #end for
15 > '$out_file1' 21 sleep 1
16 #else: 22 #else:
17 #for $file in $global_condition.inputs 23 #for $file in $global_condition.inputs
18 #if $file.ext[-2:] == "gz": 24 printf "${file.element_identifier}..." &&
25 #if $file.ext[-2:] == "gz" and $headers != 0:
19 printf "# ${file.element_identifier}\n" | gzip -c >> '$out_file1' && 26 printf "# ${file.element_identifier}\n" | gzip -c >> '$out_file1' &&
20 gzip -dc "$file" | $concat_command |gzip -c >> '$out_file1' && 27 gzip -dc "$file" | $concat_command |gzip -c >> '$out_file1' &&
21 #else: 28 #else:
22 printf "# ${file.element_identifier}\n" >> '$out_file1' && 29 printf "# ${file.element_identifier}\n" >> '$out_file1' &&
23 $concat_command "$file" >> '$out_file1' && 30 $concat_command "$file" >> '$out_file1' &&
24 #end if 31 #end if
32 printf "Done\n" &&
25 #end for 33 #end for
26 sleep 1 34 sleep 1
27 #end if 35 #end if
28 #else if $global_condition.input_type == "simple_collections": 36 #else if $global_condition.input_type == "simple_collections":
29 #if $global_condition.collections_condition.collection_cat_type == "two_collections":
30 mkdir concatenated && 37 mkdir concatenated &&
31 #if $dataset_names == "No": 38 #if $dataset_names == "No":
32 #for $x, $y in zip($global_condition.collections_condition.input_1, $global_condition.collections_condition.input_2) 39 #for $x, $y in zip($global_condition.input_1, $global_condition.input_2):
33 $concat_command '$x' '$y' > concatenated/'${x.element_identifier}.listed.${x.ext}.listed' && 40 printf "${x.element_identifier} and ${y.element_identifier}..." &&
41 #if $x.ext[-2:] == "gz" and $headers != 0:
42 gzip -dc '$x' | $concat_command | gzip -c > concatenated/'${x.element_identifier}.listed.${x.ext}.listed' &&
43 gzip -dc '$y' | $concat_command | gzip -c >> concatenated/'${x.element_identifier}.listed.${x.ext}.listed' &&
44 #else:
45 $concat_command '$x' '$y' > concatenated/'${x.element_identifier}.listed.${x.ext}.listed' &&
46 #end if
47 printf "Done\n" &&
34 #end for 48 #end for
35 sleep 1 49 sleep 1
36 #else: 50 #else:
37 #for $x, $y in zip($global_condition.collections_condition.input_1, $global_condition.collections_condition.input_2) 51 #for $x, $y in zip($global_condition.input_1, $global_condition.input_2)
38 #if $x.ext[-2:] == "gz": 52 printf "${x.element_identifier} and ${y.element_identifier}..." &&
53 #if $x.ext[-2:] == "gz" and $headers != 0:
39 printf "# ${x.element_identifier}\n" | gzip -c > concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 54 printf "# ${x.element_identifier}\n" | gzip -c > concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
40 gzip -dc '$x' | $concat_command | gzip -c >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 55 gzip -dc '$x' | $concat_command | gzip -c >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
41 printf "# ${y.element_identifier}\n" | gzip -c >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 56 printf "# ${y.element_identifier}\n" | gzip -c >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
42 gzip -dc '$y' | $concat_command | gzip -c >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 57 gzip -dc '$y' | $concat_command | gzip -c >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
43 #else: 58 #else:
44 printf "# ${x.element_identifier}\n" > concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 59 printf "# ${x.element_identifier}\n" > concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
45 $concat_command '$x'>> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 60 $concat_command '$x'>> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
46 printf "# ${y.element_identifier}\n" >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 61 printf "# ${y.element_identifier}\n" >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
47 $concat_command '$y' >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' && 62 $concat_command '$y' >> concatenated/'${x.element_identifier}_${y.element_identifier}.listed.${x.ext}.listed' &&
48 #end if 63 #end if
64 printf "Done\n" &&
49 #end for 65 #end for
50 sleep 1 66 sleep 1
51 #end if 67 #end if
52 #end if
53 #else if $global_condition.input_type == "paired_collection": 68 #else if $global_condition.input_type == "paired_collection":
54 #if $global_condition.paired_cat_type == "by_strand": 69 #if $global_condition.paired_cat_type == "by_strand":
70 mkdir concatenated &&
55 #if $dataset_names == "No": 71 #if $dataset_names == "No":
56 #for $file in $global_condition.inputs 72 #for $file in $global_condition.inputs
57 $concat_command 73 printf "${file.element_identifier}- forward and reverse..." &&
58 $file['forward'] 74 #if $file['forward'].ext[-2:] == "gz" and $headers != 0:
59 >> '$forward' && 75 gzip -dc $file['forward'] | $concat_command | gzip -c >> concatenated/forward.listed.${file['forward'].ext}.listed &&
60 $concat_command 76 gzip -dc $file['reverse'] | $concat_command | gzip -c >> concatenated/reverse.listed.${file['reverse'].ext}.listed &&
61 $file['reverse'] 77 #else:
62 >> '$reverse' && 78 $concat_command $file['forward'] >> concatenated/forward.listed.${file['forward'].ext}.listed &&
79 $concat_command $file['reverse'] >> concatenated/reverse.listed.${file['reverse'].ext}.listed &&
80 #end if
81 printf "Done\n" &&
63 #end for 82 #end for
64 sleep 1 83 sleep 1
65 #else: 84 #else:
66 #for $file in $global_condition.inputs.keys() 85 #for $file in $global_condition.inputs.keys():
67 printf "# ${file}_forward\n" >> '$forward' && 86 printf "$file - forward and reverse..." &&
68 $concat_command 87 #if $global_condition.inputs[$file]['forward'].ext[-2:] == "gz" and $headers != 0:
69 $global_condition.inputs[$file]['forward'] 88 printf "# ${file}_forward\n" | gzip -c >> concatenated/forward.listed.${global_condition.inputs[$file]['forward'].ext}.listed &&
70 >> '$forward' && 89 gzip -dc $global_condition.inputs[$file]['forward'] | $concat_command | gzip -c >> concatenated/forward.listed.${global_condition.inputs[$file]['forward'].ext}.listed &&
71 printf "# ${file}_reverse\n" >> '$reverse' && 90 printf "# ${file}_reverse\n" | gzip -c >> concatenated/reverse.listed.${global_condition.inputs[$file]['reverse'].ext}.listed &&
72 $concat_command 91 gzip -dc $global_condition.inputs[$file]['reverse'] | $concat_command | gzip -c >> concatenated/reverse.listed.${global_condition.inputs[$file]['reverse'].ext}.listed &&
73 $global_condition.inputs[$file]['reverse'] 92 #else:
74 >> '$reverse' && 93 printf "# ${file}_forward\n" >> concatenated/forward.listed.${global_condition.inputs[$file]['forward'].ext}.listed &&
94 $concat_command $global_condition.inputs[$file]['forward'] >> concatenated/forward.listed.${global_condition.inputs[$file]['forward'].ext}.listed &&
95 printf "# ${file}_reverse\n" >> concatenated/reverse.listed.${global_condition.inputs[$file]['reverse'].ext}.listed &&
96 $concat_command $global_condition.inputs[$file]['reverse'] >> concatenated/reverse.listed.${global_condition.inputs[$file]['reverse'].ext}.listed &&
97 #end if
98 printf "Done\n" &&
75 #end for 99 #end for
76 sleep 1 100 sleep 1
77 #end if 101 #end if
78 #else if $global_condition.paired_cat_type == "by_pair": 102 #else if $global_condition.paired_cat_type == "by_pair":
79 mkdir concatenated && 103 mkdir concatenated &&
80 #if $dataset_names == "No": 104 #if $dataset_names == "No":
81 #for $file in $global_condition.inputs.keys() 105 #for $file in $global_condition.inputs.keys():
82 $concat_command 106 printf "$file - forward and reverse..." &&
83 $global_condition.inputs[$file]['forward'] 107 #if $global_condition.inputs[$file]['forward'].ext[-2:] == "gz" and $headers != 0:
84 > concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' && 108 gzip -dc $global_condition.inputs[$file]['forward'] | $concat_command | gzip -c
85 $concat_command 109 > concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
86 $global_condition.inputs[$file]['reverse'] 110 gzip -dc $global_condition.inputs[$file]['reverse'] | $concat_command | gzip -c
87 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' && 111 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
112 #else:
113 $concat_command $global_condition.inputs[$file]['forward']
114 > concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
115 $concat_command $global_condition.inputs[$file]['reverse']
116 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
117 #end if
118 printf "Done\n" &&
88 #end for 119 #end for
89 sleep 1 120 sleep 1
90 #else: 121 #else:
91 #for $file in $global_condition.inputs.keys() 122 #for $file in $global_condition.inputs.keys():
92 printf "# ${file}_forward\n" > concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' && 123 printf "$file - forward and reverse..." &&
93 $concat_command 124 #if $global_condition.inputs[$file]['reverse'].ext[-2:] == "gz" and $headers != 0:
94 $global_condition.inputs[$file]['forward'] 125 printf "# ${file}_forward\n" | gzip -c > concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
95 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' && 126 gzip -dc $global_condition.inputs[$file]['forward'] | $concat_command | gzip -c
96 printf "# ${file}_reverse\n" >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' && 127 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
97 $concat_command 128 printf "# ${file}_reverse\n" | gzip -c >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
98 $global_condition.inputs[$file]['reverse'] 129 gzip -dc $global_condition.inputs[$file]['reverse'] | $concat_command | gzip -c
99 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' && 130 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
131 #else:
132 printf "# ${file}_forward\n" > concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
133 $concat_command $global_condition.inputs[$file]['forward']
134 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
135 printf "# ${file}_reverse\n" >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
136 $concat_command $global_condition.inputs[$file]['reverse']
137 >> concatenated/'${file}.listed.${global_condition.inputs[$file]['reverse'].ext}.listed' &&
138 #end if
139 printf "Done\n" &&
100 #end for 140 #end for
101 sleep 1 141 sleep 1
102 #end if 142 #end if
103 #else if $global_condition.paired_cat_type == "all": 143 #else if $global_condition.paired_cat_type == "all":
144 mkdir concatenated &&
145 #set $base_name=$global_condition.inputs.element_identifier
146 #set $extention=$global_condition.inputs[$global_condition.inputs.keys()[0]]['forward'].ext
104 #if $dataset_names == "No": 147 #if $dataset_names == "No":
105 #for $file in $global_condition.inputs.keys() 148 #for $file in $global_condition.inputs.keys():
106 $concat_command 149 printf "$file - forward and reverse..." &&
107 $global_condition.inputs[$file]['forward'] 150 #if $global_condition.inputs[$file]['forward'].ext[-2:] == "gz" and $headers != 0:
108 >> $out_file1 && 151 gzip -dc $global_condition.inputs[$file]['forward'] | $concat_command | gzip -c >> c'$paired_out_file' &&
109 $concat_command 152 gzip -dc $global_condition.inputs[$file]['reverse'] | $concat_command | gzip -c >> '$paired_out_file' &&
110 $global_condition.inputs[$file]['reverse'] 153 #else:
111 >> $out_file1 && 154 $concat_command
155 $global_condition.inputs[$file]['forward']
156 >> '$paired_out_file' &&
157 $concat_command
158 $global_condition.inputs[$file]['reverse']
159 >> '$paired_out_file' &&
160 #end if
161 printf "Done\n" &&
112 #end for 162 #end for
113 sleep 1 163 sleep 1
114 #else: 164 #else:
115 #for $file in $global_condition.inputs.keys() 165 #for $file in $global_condition.inputs.keys():
116 printf "# ${file}_forward\n" > $out_file1 && 166 printf "$file - forward and reverse..." &&
117 $concat_command 167 #if $global_condition.inputs[$file]['forward'].ext[-2:] == "gz" and $headers != 0:
118 $global_condition.inputs[$file]['forward'] 168 printf "# ${file}_forward\n" | gzip -c > '$paired_out_file' &&
119 >> $out_file1 && 169 gzip -dc $global_condition.inputs[$file]['forward'] | $concat_command | gzip -c >> '$paired_out_file' &&
120 printf "# ${file}_reverse\n" >> $out_file1 && 170 printf "# ${file}_reverse\n" | gzip -c >> '$paired_out_file' &&
121 $concat_command 171 gzip -dc $global_condition.inputs[$file]['reverse'] | $concat_command | gzip -c >> '$paired_out_file' &&
122 $global_condition.inputs[$file]['reverse'] 172 #else:
123 >> $out_file1 && 173 printf "# ${file}_forward\n" > '$paired_out_file' &&
174 $concat_command
175 $global_condition.inputs[$file]['forward']
176 >> '$paired_out_file' &&
177 printf "# ${file}_reverse\n" >> '$paired_out_file' &&
178 $concat_command
179 $global_condition.inputs[$file]['reverse']
180 >> '$paired_out_file' &&
181 #end if
182 printf "Done\n" &&
124 #end for 183 #end for
125 sleep 1 184 sleep 1
126 #end if 185 #end if
186 #end if
187 #else if $global_condition.input_type == "nested_collection":
188 mkdir concatenated &&
189 #if $dataset_names == "No":
190 #for $sub_list in $global_condition.inputs:
191 #set $file_base_name=$sub_list.element_identifier
192 #for $sub_list_element in $sub_list:
193 printf "${file_base_name} - ${sub_list_element.element_identifier}..." &&
194 #if $sub_list_element.ext[-2:] == "gz" and $headers != 0:
195 gzip -dc ${sub_list_element} | $concat_command | gzip -c >> concatenated/'${file_base_name}.listed.${sub_list_element.ext}.listed' &&
196 #else:
197 $concat_command ${sub_list_element} >> concatenated/'${file_base_name}.listed.${sub_list_element.ext}.listed' &&
198 #end if
199 printf "Done\n" &&
200 #end for
201 #end for
202 sleep 1
203 #else:
204 #for $sub_list in $global_condition.inputs:
205 #set $file_base_name=$sub_list.element_identifier
206 #for $sub_list_element in $sub_list:
207 printf "${file_base_name} - ${sub_list_element.element_identifier}..." &&
208 #if $sub_list_element.ext[-2:] == "gz" and $headers != 0:
209 printf "# ${sub_list_element.element_identifier}\n" | gzip -c >> concatenated/'${file_base_name}.listed.${sub_list_element.ext}.listed' &&
210 gzip -dc ${sub_list_element} | $concat_command | gzip -c >> concatenated/'${file_base_name}.listed.${sub_list_element.ext}.listed' &&
211 #else:
212 printf "# ${sub_list_element.element_identifier}\n" >> concatenated/'${file_base_name}.listed.${sub_list_element.ext}.listed' &&
213 $concat_command ${sub_list_element} >> concatenated/'${file_base_name}.listed.${sub_list_element.ext}.listed' &&
214 #end if
215 printf "Done\n" &&
216 #end for
217 #end for
218 sleep 1
127 #end if 219 #end if
128 #end if 220 #end if
129 ]]> 221 ]]>
130 </command> 222 </command>
131 <inputs> 223 <inputs>
132 <conditional name="global_condition"> 224 <conditional name="global_condition">
133 <param name="input_type" type="select" label="What type of data do you wish to concatenate?" help="Depending on the type of input selected the concatenation options will differ"> 225 <param name="input_type" type="select" label="What type of data do you wish to concatenate?" help="Depending on the type of input selected the concatenation options will differ">
134 <option value="singles">Single datasets</option> 226 <option value="singles">Single datasets</option>
135 <option value="simple_collections">Collections</option> 227 <option value="simple_collections">2 Collections</option>
136 <option value="paired_collection">Paired collection</option> 228 <option value="paired_collection">Paired collection</option>
229 <option value="nested_collection">Nested collection</option>
137 </param> 230 </param>
138 <when value="singles"> 231 <when value="singles">
139 <param name="inputs" type="data" label="Concatenate Datasets" multiple="True" help="All inputed datasets will be concatenated tail-to-head."/> 232 <param name="inputs" type="data" label="Concatenate Datasets" multiple="True" help="All inputed datasets will be concatenated tail-to-head."/>
140 </when> 233 </when>
141 <when value="paired_collection"> 234 <when value="paired_collection">
145 <option value="by_pair">Concatenate pairs of datasets (outputs an unpaired collection of datasets)</option> 238 <option value="by_pair">Concatenate pairs of datasets (outputs an unpaired collection of datasets)</option>
146 <option value="all">Concatenate all datasets into a single file regardless of strand (outputs a single file)</option> 239 <option value="all">Concatenate all datasets into a single file regardless of strand (outputs a single file)</option>
147 </param> 240 </param>
148 </when> 241 </when>
149 <when value="simple_collections"> 242 <when value="simple_collections">
150 <conditional name="collections_condition"> 243 <param name="input_1" type="data_collection" collection_type="list" label="Input first collection" help="The first collection contains the datasets that will be written first in the concatenated file" />
151 <param name="collection_cat_type" type="select" label="What type of concatenation do you wish to perform?"> 244 <param name="input_2" type="data_collection" collection_type="list" label="Input second collection" help="The second collection contains the datasets that will be written last in the concatenated file" />
152 <option value="two_collections">Concatenate datasets of 2 collections (outputs a simple collection)</option> 245 </when>
153 </param> 246 <when value="nested_collection">
154 <when value="two_collections"> 247 <param name="inputs" type="data_collection" collection_type="list:list" label="Input nested collection" help="The Nested collection which items you want to concatenate." />
155 <param name="input_1" type="data_collection" collection_type="list" label="Input first collection" help="The first collection contains the datasets that will be written first in the concatenated file" />
156 <param name="input_2" type="data_collection" collection_type="list" label="Input second collection" help="The second collection contains the datasets that will be written last in the concatenated file" />
157 </when>
158 </conditional>
159 </when> 248 </when>
160 </conditional> 249 </conditional>
161 <param name="dataset_names" type="boolean" label="Include dataset names?" truevalue="Yes" falsevalue="No" checked="false" help="If 'Yes' is selected '#name of dataset' will be added when concatenating."/> 250 <param name="dataset_names" type="boolean" label="Include dataset names?" truevalue="Yes" falsevalue="No" checked="false" help="If 'Yes' is selected '#name of dataset' will be added when concatenating."/>
162 <param name="headers" type="integer" label="Number of lines to skip at the beginning of each concatenation:" value="0" help="This paremeter exists so as to not concatenate comments or headers contained at the start of the files."/> 251 <param name="headers" type="integer" label="Number of lines to skip at the beginning of each concatenation:" value="0" help="This paremeter exists so as to not concatenate comments or headers contained at the start of the files."/>
163 </inputs> 252 </inputs>
164 <outputs> 253 <outputs>
165 <data name="out_file1" format_source="inputs" metadata_source="inputs" label="Concatenated datasets"> 254 <data name="out_file1" format_source="inputs" metadata_source="inputs" label="Concatenated datasets">
166 <filter>global_condition['input_type'] == 'singles' or (global_condition['input_type'] == 'paired_collection' and global_condition['paired_cat_type'] == 'all')</filter> 255 <filter>global_condition['input_type'] == 'singles'</filter>
256 </data>
257 <data name="paired_out_file" label="${global_condition.inputs.element_identifier}" auto_format="true">
258 <filter>global_condition['input_type'] == 'paired_collection' and global_condition['paired_cat_type'] == 'all'</filter>
167 </data> 259 </data>
168 <collection name="paired_output" type="paired" label="Concatenation by strtand"> 260 <collection name="paired_output" type="paired" label="Concatenation by strtand">
169 <data name="forward" /> 261 <discover_datasets pattern="(?P&lt;name&gt;.*)\.listed\.(?P&lt;ext&gt;.*)\.listed" visible="false" directory="concatenated"/>
170 <data name="reverse" />
171 <filter>global_condition['input_type'] == 'paired_collection' and global_condition['paired_cat_type'] == 'by_strand'</filter> 262 <filter>global_condition['input_type'] == 'paired_collection' and global_condition['paired_cat_type'] == 'by_strand'</filter>
172 </collection> 263 </collection>
173 <collection name="list_output" type="list" label="Concatenation by pairs"> 264 <collection name="list_output" type="list" label="Concatenation by pairs">
174 <discover_datasets pattern="(?P&lt;name&gt;.*)\.listed\.(?P&lt;ext&gt;.*)\.listed" visible="false" directory="concatenated"/> 265 <discover_datasets pattern="(?P&lt;identifier_0&gt;.*)\.listed\.(?P&lt;ext&gt;.*)\.listed" visible="false" directory="concatenated"/>
175 <filter>(global_condition['input_type'] == 'paired_collection' and global_condition['paired_cat_type'] == 'by_pair') or (global_condition['input_type'] == 'simple_collections' and global_condition['collections_condition']['collection_cat_type'] == 'two_collections')</filter> 266 <filter>(global_condition['input_type'] == 'paired_collection' and global_condition['paired_cat_type'] == 'by_pair') or (global_condition['input_type'] == 'simple_collections') or (global_condition['input_type'] == 'nested_collection')</filter>
176 </collection> 267 </collection>
177 </outputs> 268 </outputs>
178 <tests> 269 <tests>
179 <!-- Single files concatenation --> 270 <!-- Single files concatenation -->
180 <test> <!-- Test 2 single files concatenation with no other option --> 271 <test> <!-- Test 2 single files concatenation with no other option -->
320 <param name="headers" value="4" /> 411 <param name="headers" value="4" />
321 <output_collection name="list_output" type="list" count="1" > 412 <output_collection name="list_output" type="list" count="1" >
322 <element name="1_f.fastq_1_r.fastq" file="1_options.fastq.gz" decompress="True"/> 413 <element name="1_f.fastq_1_r.fastq" file="1_options.fastq.gz" decompress="True"/>
323 </output_collection> 414 </output_collection>
324 </test> 415 </test>
416 <test> <!-- Test nested collections concatenation -->
417 <param name="input_type" value="nested_collection" />
418 <param name="inputs">
419 <collection type="list:list">
420 <element name="2">
421 <collection type="list">
422 <element name="2_f" value="2_f.fastq" ftype="fastq"/>
423 <element name="2_r" value="2_r.fastq" ftype="fastq"/>
424 </collection>
425 </element>
426 <element name="3">
427 <collection type="list">
428 <element name="3" value="3.fastq" ftype="fastq"/>
429 </collection>
430 </element>
431 </collection>
432 </param>
433 <param name="dataset_names" value="No" />
434 <param name="headers" value="0" />
435 <output_collection name="list_output" type="list" count="2" >
436 <element name="2" file="2.fastq"/>
437 <element name="3" file="3.fastq"/>
438 </output_collection>
439 </test>
440 <test> <!-- Test nested collections concatenation with options and gzip-->
441 <param name="input_type" value="nested_collection" />
442 <param name="inputs">
443 <collection type="list:list">
444 <element name="1">
445 <collection type="list">
446 <element name="1_f.fastq" value="1_f.fastq.gz" ftype="fastq.gz"/>
447 <element name="1_r.fastq" value="1_r.fastq.gz" ftype="fastq.gz"/>
448 </collection>
449 </element>
450 </collection>
451 </param>
452 <param name="dataset_names" value="Yes" />
453 <param name="headers" value="4" />
454 <output_collection name="list_output" type="list" count="1" >
455 <element name="1" file="1_options.fastq.gz" decompress="True"/>
456 </output_collection>
457 </test>
325 </tests> 458 </tests>
326 <help> 459 <help>
327 460
328 .. class:: warningmark 461 .. class:: warningmark
329 462
330 **WARNING:** This tool does not check if the datasets being concatenated are in the same format. 463 **WARNING:** This tool does not check if the datasets being concatenated are in the same format.
331 464
332 **WARNING:** The paired collection operations do not handle gziped files.
333
334 **WARNING:** When concatenating 2 collections make sure the first collection is the one with the most items. 465 **WARNING:** When concatenating 2 collections make sure the first collection is the one with the most items.
335 466
467 **WARNING:** This tool can't handle nested collection deeper than list:list.
468
336 ----- 469 -----
337 470
338 **What it does** 471 **What it does**
339 472
340 Concatenates datasets and paired collections with multiple options: 473 Concatenates datasets and paired collections with multiple options:
346 - concatenation by pair : forward - reverse dataset pairs are concatenated and a simple dataset collection is returned 479 - concatenation by pair : forward - reverse dataset pairs are concatenated and a simple dataset collection is returned
347 480
348 - whole collection concatenation : all datasets in the collection are concatenated and a single dataset is returned 481 - whole collection concatenation : all datasets in the collection are concatenated and a single dataset is returned
349 482
350 - When the inputs are 2 collections: datasets are concatenated in a pairwise combination and a single dataset collection is returned 483 - When the inputs are 2 collections: datasets are concatenated in a pairwise combination and a single dataset collection is returned
484
485 - When nested collection concatenation: datasets in each sub-collection are concatenated and a simple dataset collection is returned
351 486
352 - Skipping lines before concatenation to avoid headers 487 - Skipping lines before concatenation to avoid headers
353 488
354 - Add the name of the concatenated files as separator 489 - Add the name of the concatenated files as separator
355 490
454 589
455 1 dataset 590 1 dataset
456 591
457 ----- 592 -----
458 593
594 **Nested collection concatenation example**
595
596 Nested collection:
597
598 - Experiment
599
600 - Sample_1
601
602 - Sample_1_file_1
603 - Sample_1_file_2
604
605 - Sample_2
606
607 - Sample_2_file_1
608 - Sample_2_file_2
609 - Sample_2_file_3
610
611 Concatenation result::
612
613 A single collection containing:
614
615 - Sample_1: (Sample_1_file_1 + Sample_1_file_2)
616 - Sample_2: (Sample_2_file_1 + Sample_2_file_2 + Sample_2_file_3)
617
618 -----
619
459 **When selecting "Include dataset names" when concatenating files**: 620 **When selecting "Include dataset names" when concatenating files**:
460 621
461 1rst file name="first_tabular":: 622 1rst file name="first_tabular"::
462 623
463 chrX 151087187 151087355 A 0 - 624 chrX 151087187 151087355 A 0 -