annotate toolfactory/test-data/input1_sample @ 79:2d0272e730d1 draft

Uploaded
author fubar
date Tue, 17 Nov 2020 06:21:07 +0000
parents 4d578c8c1613
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
32
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
1 *WARNING before you start*
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
2
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
3 Install this tool on a private Galaxy ONLY
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
4 Please NEVER on a public or production instance
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
5
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
6 Updated august 2014 by John Chilton adding citation support
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
7
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
8 Updated august 8 2014 to fix bugs reported by Marius van den Beek
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
9
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
10 Please cite the resource at
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
11 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
12 if you use this tool in your published work.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
13
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
14 **Short Story**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
15
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
16 This is an unusual Galaxy tool capable of generating new Galaxy tools.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
17 It works by exposing *unrestricted* and therefore extremely dangerous scripting
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
18 to all designated administrators of the host Galaxy server, allowing them to
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
19 run scripts in R, python, sh and perl over multiple selected input data sets,
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
20 writing a single new data set as output.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
21
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
22 *You have a working r/python/perl/bash script or any executable with positional or argparse style parameters*
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
23
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
24 It can be turned into an ordinary Galaxy tool in minutes, using a Galaxy tool.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
25
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
26
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
27 **Automated generation of new Galaxy tools for installation into any Galaxy**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
28
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
29 A test is generated using small sample test data inputs and parameter settings you supply.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
30 Once the test case outputs have been produced, they can be used to build a
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
31 new Galaxy tool. The supplied script or executable is baked as a requirement
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
32 into a new, ordinary Galaxy tool, fully workflow compatible out of the box.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
33 Generated tools are installed via a tool shed by an administrator
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
34 and work exactly like all other Galaxy tools for your users.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
35
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
36 **More Detail**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
37
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
38 To use the ToolFactory, you should have prepared a script to paste into a
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
39 text box, or have a package in mind and a small test input example ready to select from your history
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
40 to test your new script.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
41
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
42 ```planemo test rgToolFactory2.xml --galaxy_root ~/galaxy --test_data ~/galaxy/tools/tool_makers/toolfactory/test-data``` works for me
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
43
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
44 There is an example in each scripting language on the Tool Factory form. You
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
45 can just cut and paste these to try it out - remember to select the right
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
46 interpreter please. You'll also need to create a small test data set using
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
47 the Galaxy history add new data tool.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
48
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
49 If the script fails somehow, use the "redo" button on the tool output in
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
50 your history to recreate the form complete with broken script. Fix the bug
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
51 and execute again. Rinse, wash, repeat.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
52
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
53 Once the script runs sucessfully, a new Galaxy tool that runs your script
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
54 can be generated. Select the "generate" option and supply some help text and
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
55 names. The new tool will be generated in the form of a new Galaxy datatype
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
56 *toolshed.gz* - as the name suggests, it's an archive ready to upload to a
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
57 Galaxy ToolShed as a new tool repository.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
58
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
59 Once it's in a ToolShed, it can be installed into any local Galaxy server
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
60 from the server administrative interface.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
61
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
62 Once the new tool is installed, local users can run it - each time, the script
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
63 that was supplied when it was built will be executed with the input chosen
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
64 from the user's history. In other words, the tools you generate with the
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
65 ToolFactory run just like any other Galaxy tool,but run your script every time.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
66
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
67 Tool factory tools are perfect for workflow components. One input, one output,
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
68 no variables.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
69
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
70 *To fully and safely exploit the awesome power* of this tool,
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
71 Galaxy and the ToolShed, you should be a developer installing this
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
72 tool on a private/personal/scratch local instance where you are an
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
73 admin_user. Then, if you break it, you get to keep all the pieces see
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
74 https://bitbucket.org/fubar/galaxytoolfactory/wiki/Home
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
75
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
76 **Installation**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
77 This is a Galaxy tool. You can install it most conveniently using the
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
78 administrative "Search and browse tool sheds" link. Find the Galaxy Main
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
79 toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
80 repository. Open it and review the code and select the option to install it.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
81
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
82 If you can't get the tool that way, the xml and py files here need to be
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
83 copied into a new tools
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
84 subdirectory such as tools/toolfactory Your tool_conf.xml needs a new entry
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
85 pointing to the xml
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
86 file - something like::
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
87
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
88 <section name="Tool building tools" id="toolbuilders">
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
89 <tool file="toolfactory/rgToolFactory.xml"/>
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
90 </section>
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
91
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
92 If not already there,
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
93 please add:
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
94 <datatype extension="toolshed.gz" type="galaxy.datatypes.binary:Binary"
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
95 mimetype="multipart/x-gzip" subclass="True" />
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
96 to your local data_types_conf.xml.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
97
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
98
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
99 **Restricted execution**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
100
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
101 The tool factory tool itself will then be usable ONLY by admin users -
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
102 people with IDs in admin_users in universe_wsgi.ini **Yes, that's right. ONLY
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
103 admin_users can run this tool** Think about it for a moment. If allowed to
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
104 run any arbitrary script on your Galaxy server, the only thing that would
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
105 impede a miscreant bent on destroying all your Galaxy data would probably
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
106 be lack of appropriate technical skills.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
107
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
108 **What it does**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
109
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
110 This is a tool factory for simple scripts in python, R and
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
111 perl currently. Functional tests are automatically generated. How cool is that.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
112
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
113 LIMITED to simple scripts that read one input from the history. Optionally can
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
114 write one new history dataset, and optionally collect any number of outputs
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
115 into links on an autogenerated HTML index page for the user to navigate -
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
116 useful if the script writes images and output files - pdf outputs are shown
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
117 as thumbnails and R's bloated pdf's are shrunk with ghostscript so that and
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
118 imagemagik need to be available.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
119
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
120 Generated tools can be edited and enhanced like any Galaxy tool, so start
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
121 small and build up since a generated script gets you a serious leg up to a
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
122 more complex one.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
123
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
124 **What you do**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
125
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
126 You paste and run your script, you fix the syntax errors and
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
127 eventually it runs. You can use the redo button and edit the script before
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
128 trying to rerun it as you debug - it works pretty well.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
129
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
130 Once the script works on some test data, you can generate a toolshed compatible
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
131 gzip file containing your script ready to run as an ordinary Galaxy tool in
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
132 a repository on your local toolshed. That means safe and largely automated
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
133 installation in any production Galaxy configured to use your toolshed.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
134
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
135 **Generated tool Security**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
136
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
137 Once you install a generated tool, it's just
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
138 another tool - assuming the script is safe. They just run normally and their
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
139 user cannot do anything unusually insecure but please, practice safe toolshed.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
140 Read the code before you install any tool. Especially this one - it is really scary.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
141
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
142 **Send Code**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
143
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
144 Patches and suggestions welcome as bitbucket issues please?
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
145
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
146 **Attribution**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
147
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
148 Creating re-usable tools from scripts: The Galaxy Tool Factory
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
149 Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
150 Bioinformatics 2012; doi: 10.1093/bioinformatics/bts573
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
151
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
152 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
153
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
154 **Licensing**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
155
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
156 Copyright Ross Lazarus 2010
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
157 ross lazarus at g mail period com
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
158
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
159 All rights reserved.
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
160
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
161 Licensed under the LGPL
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
162
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
163 **Obligatory screenshot**
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
164
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
165 http://bitbucket.org/fubar/galaxytoolmaker/src/fda8032fe989/images/dynamicScriptTool.png
4d578c8c1613 passes planemo test
fubar
parents:
diff changeset
166