annotate README.txt @ 3:d75fd880fce2 draft

Uploaded
author fubar
date Sun, 11 Jan 2015 22:01:27 -0500
parents c34063ab3735
children dd6cf2ddaac7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
1 # WARNING before you start
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
2 # Install this tool on a private Galaxy ONLY
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
3 # Please NEVER on a public or production instance
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
4 # updated august 2014 by John Chilton adding citation support
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
5 #
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
6 # updated august 8 2014 to fix bugs reported by Marius van den Beek
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
7 # please cite the resource at http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
8 # if you use this tool in your published work.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
9
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
10 *Short Story*
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
11
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
12 This is an unusual Galaxy tool capable of generating new Galaxy tools.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
13 It works by exposing *unrestricted* and therefore extremely dangerous
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
14 scripting to all designated administrators of the host Galaxy server, allowing them to run scripts
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
15 in R, python, sh and perl over multiple selected input data sets, writing a single new data set as output.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
16
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
17 *Automated outputs in named sections*
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
18
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
19 If your script writes to the current directory path, arbitrary mix of (eg) pdfs, tabular analysis results and run logs,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
20 the tool factory can optionally auto-generate a linked Html page with separate sections showing a thumbnail grid
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
21 for all pdfs and the log text, grouping all artifacts sharing a file name and log name prefix::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
22
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
23 eg: if "foo.log" is emitted then *all* other outputs matching foo_* will all be grouped together - eg
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
24 foo_baz.pdf
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
25 foo_bar.pdf and
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
26 foo_zot.xls
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
27 would all be displayed and linked in the same section with foo.log's contents - to form the "Foo" section of the Html page.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
28 Sections appear in alphabetic order and there are no limits on the number of files or sections.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
29
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
30 *Automated generation of new Galaxy tool shed tools for installation into any Galaxy*
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
31
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
32 Once a script is working correctly, this tool optionally generates a new Galaxy tool, effectively
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
33 freezing the supplied script into a new, ordinary Galaxy tool that runs it over one or more input files
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
34 selected by the user. Generated tools are installed via a tool shed by an administrator and work exactly like all other Galaxy tools for your users.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
35
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
36 If you use the Html output option, please ensure that sanitize_all_html is set to False and
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
37 uncommented in universe_wsgi.ini - it should show::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
38
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
39 # By default, all tool output served as 'text/html' will be sanitized
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
40 sanitize_all_html = False
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
41
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
42 This opens potential security risks and may not be acceptable for public sites where the lack of stylesheets
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
43 may make Html pages damage onlookers' eyeballs but should still be correct.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
44
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
45
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
46 *More Detail*
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
47
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
48 To use the ToolFactory, you should have prepared a script to paste into a text box,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
49 and a small test input example ready to select from your history to test your new script.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
50 There is an example in each scripting language on the Tool Factory form. You can just
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
51 cut and paste these to try it out - remember to select the right interpreter please. You'll
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
52 also need to create a small test data set using the Galaxy history add new data tool.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
53
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
54 If the script fails somehow, use the "redo" button on the tool output in your history to
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
55 recreate the form complete with broken script. Fix the bug and execute again. Rinse, wash, repeat.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
56
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
57 Once the script runs sucessfully, a new Galaxy tool that runs your script can be generated.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
58 Select the "generate" option and supply some help text and names. The new tool will be
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
59 generated in the form of a new Galaxy datatype - toolshed.gz - as the name suggests,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
60 it's an archive ready to upload to a Galaxy ToolShed as a new tool repository.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
61
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
62 Once it's in a ToolShed, it can be installed into any local Galaxy server from
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
63 the server administrative interface.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
64
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
65 Once the new tool is installed, local users can run it - each time, the script that was supplied
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
66 when it was built will be executed with the input chosen from the user's history. In other words,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
67 the tools you generate with the ToolFactory run just like any other Galaxy tool,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
68 but run your script every time.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
69
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
70 Tool factory tools are perfect for workflow components. One input, one output, no variables.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
71
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
72 *To fully and safely exploit the awesome power* of this tool, Galaxy and the ToolShed,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
73 you should be a developer installing this tool on a private/personal/scratch local instance where you
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
74 are an admin_user. Then, if you break it, you get to keep all the pieces
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
75 see https://bitbucket.org/fubar/galaxytoolfactory/wiki/Home
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
76
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
77 ** Installation **
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
78 This is a Galaxy tool. You can install it most conveniently using the administrative "Search and browse tool sheds" link.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
79 Find the Galaxy Main toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory repository.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
80 Open it and review the code and select the option to install it.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
81
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
82 (
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
83 If you can't get the tool that way, the xml and py files here need to be copied into a new tools
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
84 subdirectory such as tools/toolfactory Your tool_conf.xml needs a new entry pointing to the xml
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
85 file - something like::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
86
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
87 <section name="Tool building tools" id="toolbuilders">
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
88 <tool file="toolfactory/rgToolFactory.xml"/>
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
89 </section>
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
90
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
91 If not already there (I just added it to datatypes_conf.xml.sample), please add:
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
92 <datatype extension="toolshed.gz" type="galaxy.datatypes.binary:Binary" mimetype="multipart/x-gzip" subclass="True" />
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
93 to your local data_types_conf.xml.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
94 )
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
95
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
96 Of course, R, python, perl etc are needed on your path if you want to test scripts using those interpreters.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
97 Adding new ones to this tool code should be easy enough. Please make suggestions as bitbucket issues and code.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
98 The HTML file code automatically shrinks R's bloated pdfs, and depends on ghostscript. The thumbnails require imagemagick .
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
99
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
100 * Restricted execution *
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
101 The tool factory tool itself will then be usable ONLY by admin users - people with IDs in admin_users in universe_wsgi.ini
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
102 **Yes, that's right. ONLY admin_users can run this tool** Think about it for a moment. If allowed to run any
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
103 arbitrary script on your Galaxy server, the only thing that would impede a miscreant bent on destroying all your
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
104 Galaxy data would probably be lack of appropriate technical skills.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
105
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
106 *What it does* This is a tool factory for simple scripts in python, R and perl currently.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
107 Functional tests are automatically generated. How cool is that.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
108
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
109 LIMITED to simple scripts that read one input from the history.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
110 Optionally can write one new history dataset,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
111 and optionally collect any number of outputs into links on an autogenerated HTML
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
112 index page for the user to navigate - useful if the script writes images and output files - pdf outputs
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
113 are shown as thumbnails and R's bloated pdf's are shrunk with ghostscript so that and imagemagik need to
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
114 be avaailable.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
115
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
116 Generated tools can be edited and enhanced like any Galaxy tool, so start small and build up since
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
117 a generated script gets you a serious leg up to a more complex one.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
118
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
119 *What you do* You paste and run your script
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
120 you fix the syntax errors and eventually it runs
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
121 You can use the redo button and edit the script before
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
122 trying to rerun it as you debug - it works pretty well.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
123
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
124 Once the script works on some test data, you can
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
125 generate a toolshed compatible gzip file
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
126 containing your script ready to run as an ordinary Galaxy tool in a
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
127 repository on your local toolshed. That means safe and largely automated installation in any
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
128 production Galaxy configured to use your toolshed.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
129
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
130 *Generated tool Security* Once you install a generated tool, it's just
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
131 another tool - assuming the script is safe. They just run normally and their user cannot do anything unusually insecure
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
132 but please, practice safe toolshed.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
133 Read the fucking code before you install any tool.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
134 Especially this one - it is really scary.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
135
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
136 If you opt for an HTML output, you get all the script outputs arranged
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
137 as a single Html history item - all output files are linked, thumbnails for all the pdfs.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
138 Ugly but really inexpensive.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
139
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
140 Patches and suggestions welcome as bitbucket issues please?
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
141
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
142 copyright ross lazarus (ross stop lazarus at gmail stop com) May 2012
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
143
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
144 all rights reserved
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
145 Licensed under the LGPL if you want to improve it, feel free https://bitbucket.org/fubar/galaxytoolfactory/wiki/Home
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
146
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
147 Material for our more enthusiastic and voracious readers continues below - we salute you.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
148
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
149 **Motivation** Simple transformation, filtering or reporting scripts get written, run and lost every day in most busy labs
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
150 - even ours where Galaxy is in use. This 'dark script matter' is pervasive and generally not reproducible.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
151
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
152 **Benefits** For our group, this allows Galaxy to fill that important dark script gap - all those "small" bioinformatics
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
153 tasks. Once a user has a working R (or python or perl) script that does something Galaxy cannot currently do (eg transpose a
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
154 tabular file) and takes parameters the way Galaxy supplies them (see example below), they:
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
155
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
156 1. Install the tool factory on a personal private instance
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
157
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
158 2. Upload a small test data set
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
159
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
160 3. Paste the script into the 'script' text box and iteratively run the insecure tool on test data until it works right -
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
161 there is absolutely no reason to do this anywhere other than on a personal private instance.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
162
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
163 4. Once it works right, set the 'Generate toolshed gzip' option and run it again.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
164
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
165 5. A toolshed style gzip appears ready to upload and install like any other Toolshed entry.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
166
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
167 6. Upload the new tool to the toolshed
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
168
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
169 7. Ask the local admin to check the new tool to confirm it's not evil and install it in the local production galaxy
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
170
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
171 **Simple examples on the tool form**
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
172
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
173 A simple Rscript "filter" showing how the command line parameters can be handled, takes an input file,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
174 does something (transpose in this case) and writes the results to a new tabular file::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
175
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
176 # transpose a tabular input file and write as a tabular output file
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
177 ourargs = commandArgs(TRUE)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
178 inf = ourargs[1]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
179 outf = ourargs[2]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
180 inp = read.table(inf,head=F,row.names=NULL,sep='\t')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
181 outp = t(inp)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
182 write.table(outp,outf, quote=FALSE, sep="\t",row.names=F,col.names=F)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
183
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
184 Calculate a multiple test adjusted p value from a column of p values - for this script to be useful,
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
185 it needs the right column for the input to be specified in the code for the
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
186 given input file type(s) specified when the tool is generated ::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
187
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
188 # use p.adjust - assumes a HEADER row and column 1 - please fix for any real use
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
189 column = 1 # adjust if necessary for some other kind of input
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
190 fdrmeth = 'BH'
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
191 ourargs = commandArgs(TRUE)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
192 inf = ourargs[1]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
193 outf = ourargs[2]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
194 inp = read.table(inf,head=T,row.names=NULL,sep='\t')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
195 p = inp[,column]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
196 q = p.adjust(p,method=fdrmeth)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
197 newval = paste(fdrmeth,'p-value',sep='_')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
198 q = data.frame(q)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
199 names(q) = newval
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
200 outp = cbind(inp,newval=q)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
201 write.table(outp,outf, quote=FALSE, sep="\t",row.names=F,col.names=T)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
202
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
203
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
204
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
205 Another Rscript example without any input file - generates a random heatmap pdf - you must make sure the option to create an HTML output file is
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
206 turned on for this to work. The heatmap will be presented as a thumbnail linked to the pdf in the resulting HTML page::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
207
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
208 # note this script takes NO input or output because it generates random data
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
209 foo = data.frame(a=runif(100),b=runif(100),c=runif(100),d=runif(100),e=runif(100),f=runif(100))
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
210 bar = as.matrix(foo)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
211 pdf( "heattest.pdf" )
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
212 heatmap(bar,main='Random Heatmap')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
213 dev.off()
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
214
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
215 A Python example that reverses each row of a tabular file. You'll need to remove the leading spaces for this to work if cut
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
216 and pasted into the script box. Note that you can already do this in Galaxy by setting up the cut columns tool with the
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
217 correct number of columns in reverse order,but this script will work for any number of columns so is completely generic::
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
218
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
219 # reverse order of columns in a tabular file
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
220 import sys
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
221 inp = sys.argv[1]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
222 outp = sys.argv[2]
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
223 i = open(inp,'r')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
224 o = open(outp,'w')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
225 for row in i:
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
226 rs = row.rstrip().split('\t')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
227 rs.reverse()
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
228 o.write('\t'.join(rs))
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
229 o.write('\n')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
230 i.close()
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
231 o.close()
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
232
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
233
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
234 Galaxy as an IDE for developing API scripts
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
235 If you need to develop Galaxy API scripts and you like to live dangerously, please read on.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
236
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
237 Galaxy as an IDE?
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
238 Amazingly enough, blend-lib API scripts run perfectly well *inside* Galaxy when pasted into a Tool Factory form. No need to generate a new tool. Galaxy+Tool_Factory = IDE I think we need a new t-shirt. Seriously, it is actually quite useable.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
239
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
240 Why bother - what's wrong with Eclipse
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
241 Nothing. But, compared with developing API scripts in the usual way outside Galaxy, you get persistence and other framework benefits plus at absolutely no extra charge, a ginormous security problem if you share the history or any outputs because they contain the api script with key so development servers only please!
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
242
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
243 Workflow
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
244 Fire up the Tool Factory in Galaxy.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
245
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
246 Leave the input box empty, set the interpreter to python, paste and run an api script - eg working example (substitute the url and key) below.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
247
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
248 It took me a few iterations to develop the example below because I know almost nothing about the API. I started with very simple code from one of the samples and after each run, the (edited..) api script is conveniently recreated using the redo button on the history output item. So each successive version of the developing api script you run is persisted - ready to be edited and rerun easily. It is ''very'' handy to be able to add a line of code to the script and run it, then view the output to (eg) inspect dicts returned by API calls to help move progressively deeper iteratively.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
249
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
250 Give the below a whirl on a private clone (install the tool factory from the main toolshed) and try adding complexity with few rerun/edit/rerun cycles.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
251
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
252 Eg tool factory api script
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
253 import sys
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
254 from blend.galaxy import GalaxyInstance
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
255 ourGal = 'http://x.x.x.x:xxxx'
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
256 ourKey = 'xxx'
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
257 gi = GalaxyInstance(ourGal, key=ourKey)
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
258 libs = gi.libraries.get_libraries()
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
259 res = []
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
260 # libs looks like
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
261 # u'url': u'/galaxy/api/libraries/441d8112651dc2f3', u'id': u'441d8112651dc2f3', u'name':.... u'Demonstration sample RNA data',
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
262 for lib in libs:
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
263 res.append('%s:\n' % lib['name'])
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
264 res.append(str(gi.libraries.show_library(lib['id'],contents=True)))
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
265 outf=open(sys.argv[2],'w')
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
266 outf.write('\n'.join(res))
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
267 outf.close()
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
268
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
269 **Attribution**
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
270 Creating re-usable tools from scripts: The Galaxy Tool Factory
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
271 Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
272 Bioinformatics 2012; doi: 10.1093/bioinformatics/bts573
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
273
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
274 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
275
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
276 **Licensing**
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
277 Copyright Ross Lazarus 2010
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
278 ross lazarus at g mail period com
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
279
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
280 All rights reserved.
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
281
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
282 Licensed under the LGPL
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
283
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
284 **Obligatory screenshot**
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
285
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
286 http://bitbucket.org/fubar/galaxytoolmaker/src/fda8032fe989/images/dynamicScriptTool.png
c34063ab3735 Initial commit of code in iuc github repository
fubar
parents:
diff changeset
287