annotate toolfactory_docker/test-data/input1_sample @ 2:a5c5652823a6 draft

Uploaded
author fubar
date Tue, 05 Jan 2021 00:35:40 +0000
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
1 *WARNING before you start*
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
2
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
3 Install this tool on a private Galaxy ONLY
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
4 Please NEVER on a public or production instance
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
5
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
6 Updated august 2014 by John Chilton adding citation support
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
7
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
8 Updated august 8 2014 to fix bugs reported by Marius van den Beek
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
9
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
10 Please cite the resource at
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
11 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
12 if you use this tool in your published work.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
13
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
14 **Short Story**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
15
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
16 This is an unusual Galaxy tool capable of generating new Galaxy tools.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
17 It works by exposing *unrestricted* and therefore extremely dangerous scripting
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
18 to all designated administrators of the host Galaxy server, allowing them to
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
19 run scripts in R, python, sh and perl over multiple selected input data sets,
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
20 writing a single new data set as output.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
21
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
22 *You have a working r/python/perl/bash script or any executable with positional or argparse style parameters*
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
23
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
24 It can be turned into an ordinary Galaxy tool in minutes, using a Galaxy tool.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
25
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
26
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
27 **Automated generation of new Galaxy tools for installation into any Galaxy**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
28
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
29 A test is generated using small sample test data inputs and parameter settings you supply.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
30 Once the test case outputs have been produced, they can be used to build a
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
31 new Galaxy tool. The supplied script or executable is baked as a requirement
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
32 into a new, ordinary Galaxy tool, fully workflow compatible out of the box.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
33 Generated tools are installed via a tool shed by an administrator
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
34 and work exactly like all other Galaxy tools for your users.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
35
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
36 **More Detail**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
37
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
38 To use the ToolFactory, you should have prepared a script to paste into a
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
39 text box, or have a package in mind and a small test input example ready to select from your history
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
40 to test your new script.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
41
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
42 ```planemo test rgToolFactory2.xml --galaxy_root ~/galaxy --test_data ~/galaxy/tools/tool_makers/toolfactory/test-data``` works for me
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
43
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
44 There is an example in each scripting language on the Tool Factory form. You
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
45 can just cut and paste these to try it out - remember to select the right
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
46 interpreter please. You'll also need to create a small test data set using
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
47 the Galaxy history add new data tool.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
48
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
49 If the script fails somehow, use the "redo" button on the tool output in
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
50 your history to recreate the form complete with broken script. Fix the bug
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
51 and execute again. Rinse, wash, repeat.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
52
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
53 Once the script runs sucessfully, a new Galaxy tool that runs your script
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
54 can be generated. Select the "generate" option and supply some help text and
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
55 names. The new tool will be generated in the form of a new Galaxy datatype
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
56 *toolshed.gz* - as the name suggests, it's an archive ready to upload to a
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
57 Galaxy ToolShed as a new tool repository.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
58
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
59 Once it's in a ToolShed, it can be installed into any local Galaxy server
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
60 from the server administrative interface.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
61
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
62 Once the new tool is installed, local users can run it - each time, the script
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
63 that was supplied when it was built will be executed with the input chosen
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
64 from the user's history. In other words, the tools you generate with the
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
65 ToolFactory run just like any other Galaxy tool,but run your script every time.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
66
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
67 Tool factory tools are perfect for workflow components. One input, one output,
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
68 no variables.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
69
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
70 *To fully and safely exploit the awesome power* of this tool,
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
71 Galaxy and the ToolShed, you should be a developer installing this
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
72 tool on a private/personal/scratch local instance where you are an
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
73 admin_user. Then, if you break it, you get to keep all the pieces see
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
74 https://bitbucket.org/fubar/galaxytoolfactory/wiki/Home
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
75
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
76 **Installation**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
77 This is a Galaxy tool. You can install it most conveniently using the
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
78 administrative "Search and browse tool sheds" link. Find the Galaxy Main
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
79 toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
80 repository. Open it and review the code and select the option to install it.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
81
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
82 If you can't get the tool that way, the xml and py files here need to be
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
83 copied into a new tools
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
84 subdirectory such as tools/toolfactory Your tool_conf.xml needs a new entry
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
85 pointing to the xml
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
86 file - something like::
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
87
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
88 <section name="Tool building tools" id="toolbuilders">
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
89 <tool file="toolfactory/rgToolFactory.xml"/>
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
90 </section>
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
91
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
92 If not already there,
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
93 please add:
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
94 <datatype extension="toolshed.gz" type="galaxy.datatypes.binary:Binary"
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
95 mimetype="multipart/x-gzip" subclass="True" />
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
96 to your local data_types_conf.xml.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
97
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
98
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
99 **Restricted execution**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
100
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
101 The tool factory tool itself will then be usable ONLY by admin users -
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
102 people with IDs in admin_users in universe_wsgi.ini **Yes, that's right. ONLY
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
103 admin_users can run this tool** Think about it for a moment. If allowed to
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
104 run any arbitrary script on your Galaxy server, the only thing that would
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
105 impede a miscreant bent on destroying all your Galaxy data would probably
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
106 be lack of appropriate technical skills.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
107
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
108 **What it does**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
109
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
110 This is a tool factory for simple scripts in python, R and
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
111 perl currently. Functional tests are automatically generated. How cool is that.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
112
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
113 LIMITED to simple scripts that read one input from the history. Optionally can
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
114 write one new history dataset, and optionally collect any number of outputs
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
115 into links on an autogenerated HTML index page for the user to navigate -
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
116 useful if the script writes images and output files - pdf outputs are shown
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
117 as thumbnails and R's bloated pdf's are shrunk with ghostscript so that and
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
118 imagemagik need to be available.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
119
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
120 Generated tools can be edited and enhanced like any Galaxy tool, so start
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
121 small and build up since a generated script gets you a serious leg up to a
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
122 more complex one.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
123
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
124 **What you do**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
125
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
126 You paste and run your script, you fix the syntax errors and
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
127 eventually it runs. You can use the redo button and edit the script before
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
128 trying to rerun it as you debug - it works pretty well.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
129
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
130 Once the script works on some test data, you can generate a toolshed compatible
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
131 gzip file containing your script ready to run as an ordinary Galaxy tool in
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
132 a repository on your local toolshed. That means safe and largely automated
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
133 installation in any production Galaxy configured to use your toolshed.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
134
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
135 **Generated tool Security**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
136
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
137 Once you install a generated tool, it's just
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
138 another tool - assuming the script is safe. They just run normally and their
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
139 user cannot do anything unusually insecure but please, practice safe toolshed.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
140 Read the code before you install any tool. Especially this one - it is really scary.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
141
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
142 **Send Code**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
143
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
144 Patches and suggestions welcome as bitbucket issues please?
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
145
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
146 **Attribution**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
147
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
148 Creating re-usable tools from scripts: The Galaxy Tool Factory
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
149 Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
150 Bioinformatics 2012; doi: 10.1093/bioinformatics/bts573
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
151
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
152 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
153
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
154 **Licensing**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
155
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
156 Copyright Ross Lazarus 2010
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
157 ross lazarus at g mail period com
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
158
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
159 All rights reserved.
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
160
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
161 Licensed under the LGPL
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
162
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
163 **Obligatory screenshot**
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
164
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
165 http://bitbucket.org/fubar/galaxytoolmaker/src/fda8032fe989/images/dynamicScriptTool.png
a5c5652823a6 Uploaded
fubar
parents:
diff changeset
166