| Previous changeset 25:9fe74bd23af2 (2015-03-02) Next changeset 27:c4e3cf28545f (2020-07-31) |
|
Commit message:
Passes planemo test Uses galaxyxml to generate new tool. More outputs will be added... |
|
modified:
README.md README.txt rgToolFactory2.py rgToolFactory2.xml |
|
added:
test-data/infile.tabular test-data/reverseargp2.toolshed.gz test-data/reverseargp2_test1_output.xls test-data/runme.py |
|
removed:
getlocalrpackages.py test-data/tf2_test.html test-data/tf2_test.toolshed.gz test-data/tf2_test_in.xls test-data/tf2_test_out.xls test-data/tf2_test_runme.py tool_dependencies.xml |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 README.md --- a/README.md Mon Mar 02 05:18:21 2015 -0500 +++ b/README.md Thu Jul 30 06:48:45 2020 -0400 |
| [ |
| @@ -34,20 +34,14 @@ A copy of the actual script is provided for provenance - be warned, it exposes real file paths. - -tldr; - -``` +**WARNING before you start** -# WARNING before you start -# Install this tool on a private Galaxy ONLY -# Please NEVER on a public or production instance -# updated august 2014 by John Chilton adding citation support -# -# updated august 8 2014 to fix bugs reported by Marius van den Beek -# please cite the resource at +Install this tool on a private Galaxy ONLY +Please NEVER on a public or production instance +Please cite the resource at http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref -# if you use this tool in your published work. +if you use this tool in your published work. + *Short Story* @@ -78,16 +72,14 @@ pdfs, tabular analysis results and run logs,the tool factory can optionally auto-generate a linked Html page with separate sections showing a thumbnail grid for all pdfs and the log text, grouping all artifacts sharing a file -name and log name prefix:: - - eg: if "foo.log" is emitted then *all* other outputs matching foo_* will - all be grouped together - eg - foo_baz.pdf - foo_bar.pdf and - foo_zot.xls - would all be displayed and linked in the same section with foo.log's contents - - to form the "Foo" section of the Html page. Sections appear in alphabetic - order and there are no limits on the number of files or sections. +name and log name prefix.if "foo.log" is emitted then *all* other outputs matching foo_* will +all be grouped together - eg +- foo_baz.pdf +- foo_bar.pdf and +- foo_zot.xls + +would all be displayed and linked in the same section with foo.log's contents to form the "Foo" section of the Html page. +Sections appear in alphabetic order and there are no limits on the number of files or sections. *Automated generation of new Galaxy tools for installation into any Galaxy* @@ -98,16 +90,15 @@ and work exactly like all other Galaxy tools for your users. If you use the Html output option, please ensure that sanitize_all_html is -set to False and uncommented in universe_wsgi.ini - it should show:: +set to False and uncommented in universe_wsgi.ini - it should show - # By default, all tool output served as 'text/html' will be sanitized - sanitize_all_html = False +By default, all tool output served as 'text/html' will be sanitized +Change ```sanitize_all_html = False``` This opens potential security risks and may not be acceptable for public sites where the lack of stylesheets may make Html pages damage onlookers' eyeballs but should still be correct. - *More Detail* To use the ToolFactory, you should have prepared a script to paste into a @@ -152,23 +143,24 @@ toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory repository. Open it and review the code and select the option to install it. -( + If you can't get the tool that way, the xml and py files here need to be -copied into a new tools -subdirectory such as tools/toolfactory Your tool_conf.xml needs a new entry -pointing to the xml -file - something like:: - +copied into a new tools subdirectory such as tools/toolfactory +Your tool_conf.xml needs a new entry pointing to the xml \file - something like +``` <section name="Tool building tools" id="toolbuilders"> <tool file="toolfactory/rgToolFactory.xml"/> </section> - +``` If not already there (I just added it to datatypes_conf.xml.sample), please add: + +``` <datatype extension="toolshed.gz" type="galaxy.datatypes.binary:Binary" mimetype="multipart/x-gzip" subclass="True" /> +``` to your local data_types_conf.xml. -) + Of course, R, python, perl etc are needed on your path if you want to test scripts using those interpreters. Adding new ones to this tool code should @@ -275,7 +267,7 @@ Bash and sh will see any additional parameters on their command lines and the 3 named parameters in their environment magically - well, using env on the CL - +``` ***python***:: # argparse for 3 possible comma separated lists @@ -345,7 +337,7 @@ print $fh "INPATHS=$INPATHS\n INNAMES=$INNAMES\n OUTPATH=$OUTPATH\n"; close $fh; - +``` Galaxy as an IDE for developing API scripts If you need to develop Galaxy API scripts and you like to live dangerously, @@ -383,6 +375,7 @@ the main toolshed) and try adding complexity with few rerun/edit/rerun cycles. Eg tool factory api script +``` import sys from blend.galaxy import GalaxyInstance ourGal = 'http://x.x.x.x:xxxx' @@ -399,6 +392,7 @@ outf=open(sys.argv[2],'w') outf.write('\n'.join(res)) outf.close() +``` **Attribution** Creating re-usable tools from scripts: The Galaxy Tool Factory @@ -415,9 +409,9 @@ Licensed under the LGPL -**Obligatory screenshot** +**screenshot** -http://bitbucket.org/fubar/galaxytoolmaker/src/fda8032fe989/images/dynamicScriptTool.png + ``` |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 README.txt --- a/README.txt Mon Mar 02 05:18:21 2015 -0500 +++ b/README.txt Thu Jul 30 06:48:45 2020 -0400 |
| [ |
| b'@@ -16,63 +16,28 @@\n run scripts in R, python, sh and perl over multiple selected input data sets,\n writing a single new data set as output.\n \n-*Differences between TF2 and the original Tool Factory*\n-\n-1. TF2 (this one) allows any number of either fixed or user-editable parameters to be defined\n-for the new tool. If these are editable, the user can change them but otherwise, they are passed\n-as fixed and invisible parameters for each execution. Obviously, there are substantial security\n-implications with editable parameters, but these are always sanitized by Galaxy\'s inbuilt \n-parameter sanitization so you may need to "unsanitize" characters - eg translate all "__lt__" \n-into "<" for certain parameters where that is needed. Please practise safe toolshed.\n-\n-2. Any number of (the same datatype) of input files may be defined.\n-\n-These changes substantially complicate the way your supplied script is supplied with\n-all the new and variable parameters. Examples in each scripting language are shown\n-in the tool help\n+*You have a working r/python/perl/bash script or any executable with positional or argparse style parameters*\n \n-*Automated outputs in named sections*\n-\n-If your script writes to the current directory path, arbitrary mix of (eg)\n-pdfs, tabular analysis results and run logs,the tool factory can optionally\n-auto-generate a linked Html page with separate sections showing a thumbnail\n-grid for all pdfs and the log text, grouping all artifacts sharing a file\n-name and log name prefix::\n+It can be turned into an ordinary Galaxy tool in minutes, using a Galaxy tool.\n \n- eg: if "foo.log" is emitted then *all* other outputs matching foo_* will\n- all be grouped together - eg\n- foo_baz.pdf\n- foo_bar.pdf and\n- foo_zot.xls\n- would all be displayed and linked in the same section with foo.log\'s contents\n- - to form the "Foo" section of the Html page. Sections appear in alphabetic\n- order and there are no limits on the number of files or sections.\n \n *Automated generation of new Galaxy tools for installation into any Galaxy*\n \n-Once a script is working correctly, this tool optionally generates a\n-new Galaxy tool, effectively freezing the supplied script into a new,\n-ordinary Galaxy tool that runs it over one or more input files selected by\n-the user. Generated tools are installed via a tool shed by an administrator\n+A test is generated using small sample test data inputs and parameter settings you supply.\n+Once the test case outputs have been produced, they can be used to build a\n+new Galaxy tool. The supplied script or executable is baked as a requirement\n+into a new, ordinary Galaxy tool, fully workflow compatible out of the box.\n+Generated tools are installed via a tool shed by an administrator\n and work exactly like all other Galaxy tools for your users.\n \n-If you use the Html output option, please ensure that sanitize_all_html is\n-set to False and uncommented in universe_wsgi.ini - it should show::\n-\n- # By default, all tool output served as \'text/html\' will be sanitized\n- sanitize_all_html = False\n-\n-This opens potential security risks and may not be acceptable for public\n-sites where the lack of stylesheets may make Html pages damage onlookers\'\n-eyeballs but should still be correct.\n-\n-\n *More Detail*\n \n To use the ToolFactory, you should have prepared a script to paste into a\n-text box, and a small test input example ready to select from your history\n+text box, or have a package in mind and a small test input example ready to select from your history\n to test your new script.\n \n+```planemo test rgToolFactory2.xml --galaxy_root ~/galaxy --test_data ~/galaxy/tools/tool_makers/toolfactory/test-data``` works for me\n+\n There is an example in each scripting language on the Tool Factory form. You\n can just cut and paste these to try it out - remember to select the right\n interpreter please. You\'ll also need to create a small test data set using\n@@ -129,12 +94,6 @@\n to your local data_types_conf.xml.\n )\n \n-Of course, R, python, perl'..b'if necessary for some other kind of input\n- fdrmeth = \'BH\'\n- ourargs = commandArgs(TRUE)\n- inf = ourargs[1]\n- outf = ourargs[2]\n- inp = read.table(inf,head=T,row.names=NULL,sep=\'\\t\')\n- p = inp[,column]\n- q = p.adjust(p,method=fdrmeth)\n- newval = paste(fdrmeth,\'p-value\',sep=\'_\')\n- q = data.frame(q)\n- names(q) = newval\n- outp = cbind(inp,newval=q)\n- write.table(outp,outf, quote=FALSE, sep="\\t",row.names=F,col.names=T)\n-\n-\n-\n-Another Rscript example without any input file - generates a random heatmap\n-pdf - you must make sure the option to create an HTML output file is\n-turned on for this to work. The heatmap will be presented as a thumbnail\n-linked to the pdf in the resulting HTML page::\n-\n- # note this script takes NO input or output because it generates random data\n- foo = data.frame(a=runif(100),b=runif(100),c=runif(100),d=runif(100),\n- e=runif(100),f=runif(100))\n- bar = as.matrix(foo)\n- pdf( "heattest.pdf" )\n- heatmap(bar,main=\'Random Heatmap\')\n- dev.off()\n-\n-A Python example that reverses each row of a tabular file. You\'ll need\n-to remove the leading spaces for this to work if cut and pasted into the\n-script box. Note that you can already do this in Galaxy by setting up the\n-cut columns tool with the correct number of columns in reverse order,but\n-this script will work for any number of columns so is completely generic::\n-\n-# reverse order of columns in a tabular file\n-import sys\n-inp = sys.argv[1]\n-outp = sys.argv[2]\n-i = open(inp,\'r\')\n-o = open(outp,\'w\')\n-for row in i:\n- rs = row.rstrip().split(\'\\t\')\n- rs.reverse()\n- o.write(\'\\t\'.join(rs))\n- o.write(\'\\n\')\n-i.close()\n-o.close()\n-\n-\n-Galaxy as an IDE for developing API scripts\n-If you need to develop Galaxy API scripts and you like to live dangerously,\n-please read on.\n-\n-Galaxy as an IDE?\n-Amazingly enough, blend-lib API scripts run perfectly well *inside*\n-Galaxy when pasted into a Tool Factory form. No need to generate a new\n-tool. Galaxy+Tool_Factory = IDE I think we need a new t-shirt. Seriously,\n-it is actually quite useable.\n-\n-Why bother - what\'s wrong with Eclipse\n-Nothing. But, compared with developing API scripts in the usual way outside\n-Galaxy, you get persistence and other framework benefits plus at absolutely\n-no extra charge, a ginormous security problem if you share the history or\n-any outputs because they contain the api script with key so development\n-servers only please!\n-\n-Workflow\n-Fire up the Tool Factory in Galaxy.\n-\n-Leave the input box empty, set the interpreter to python, paste and run an\n-api script - eg working example (substitute the url and key) below.\n-\n-It took me a few iterations to develop the example below because I know\n-almost nothing about the API. I started with very simple code from one of the\n-samples and after each run, the (edited..) api script is conveniently recreated\n-using the redo button on the history output item. So each successive version\n-of the developing api script you run is persisted - ready to be edited and\n-rerun easily. It is \'\'very\'\' handy to be able to add a line of code to the\n-script and run it, then view the output to (eg) inspect dicts returned by\n-API calls to help move progressively deeper iteratively.\n-\n-Give the below a whirl on a private clone (install the tool factory from\n-the main toolshed) and try adding complexity with few rerun/edit/rerun cycles.\n-\n-Eg tool factory api script\n-import sys\n-from blend.galaxy import GalaxyInstance\n-ourGal = \'http://x.x.x.x:xxxx\'\n-ourKey = \'xxx\'\n-gi = GalaxyInstance(ourGal, key=ourKey)\n-libs = gi.libraries.get_libraries()\n-res = []\n-# libs looks like\n-# u\'url\': u\'/galaxy/api/libraries/441d8112651dc2f3\', u\'id\':\n-u\'441d8112651dc2f3\', u\'name\':.... u\'Demonstration sample RNA data\',\n-for lib in libs:\n- res.append(\'%s:\\n\' % lib[\'name\'])\n- res.append(str(gi.libraries.show_library(lib[\'id\'],contents=True)))\n-outf=open(sys.argv[2],\'w\')\n-outf.write(\'\\n\'.join(res))\n-outf.close()\n \n **Attribution**\n Creating re-usable tools from scripts: The Galaxy Tool Factory\n' |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 getlocalrpackages.py --- a/getlocalrpackages.py Mon Mar 02 05:18:21 2015 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 |
| [ |
| @@ -1,53 +0,0 @@ -import os -import subprocess -import sys - - -def find_packages(prefix="package_r_"): - """ - """ - #locate env.sh | grep -i package_r_ - #/data/extended/galaxyJune14_2014/tool_dependency/readline/6.2/devteam/package_r_2_15_0/8ab0d08a3da1/env.sh - #/data/home/rlazarus/galaxy/tool_dependency_dir/R_3_1_1/3.1.1/fubar/package_r_3_1_1/5f1b8d22140a/env.sh - #/data/home/rlazarus/galaxy/tool_dependency_dir/R_3_1_1/3.1.1/fubar/package_r_3_1_1/d9964efbfbe3/env.sh - #/data/home/rlazarus/galtest/tool_dependency_dir/R_3_1_1/3.1.1/fubar/package_r_3_1_1/63cdb9b2234c/env.sh - eprefix = prefix - if prefix.find('/') <> -1: - eprefix = prefix.replace('/','\/') # for grep - path = '.' - # fails on nitesh's recent mac - locate not working - # cl = ['locate env.sh | grep -i %s' % eprefix,] - cl = ['find %s -iname "env.sh" | grep -i %s' % (path,eprefix),] - p = subprocess.Popen(cl, stdout=subprocess.PIPE, stderr=subprocess.PIPE,shell=True) - out, err = p.communicate() - fpaths = out.split('\n') - fpaths = [x for x in fpaths if len(x) > 1] - fver = [x.split(os.path.sep)[-4:-1] for x in fpaths] - # >>> foo.split(os.path.sep)[-4:-1] - # ['fubar', 'package_r_3_1_1', '63cdb9b2234c'] - if len(fpaths) > 0: - res = [['%s rev %s owner %s' % (x[1],x[2],x[0]),fpaths[i],False] for i,x in enumerate(fver)] - res.insert(0,['Use default (system) interpreter','system',False]) - else: - res = [['Use default (system) interpreter','system',False], - ['**WARNING** NO package env.sh files found - is the "find" system command working? Are any interpreters installed?','system',True]] - if len(res) > 2: - res[1][2] = True # selected if more than one - # return a triplet - user_sees,value,selected - all unselected if False - return res - -def testapi(): - host_url = 'http://localhost:8080' - new_path = [ os.path.join( os.getcwd(), "lib" ) ] - new_path.extend( sys.path[1:] ) # remove scripts/ from the path - sys.path = new_path - from galaxy import config - aconfig = config.Configuration( ) - M_A_K = aconfig.master_api_key - tooldeps = aconfig.tool_dependency_dir - gi = GalaxyInstance(url=host_url, key=M_A_K) - - -if __name__ == "__main__": - print find_packages() - |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 rgToolFactory2.py --- a/rgToolFactory2.py Mon Mar 02 05:18:21 2015 -0500 +++ b/rgToolFactory2.py Thu Jul 30 06:48:45 2020 -0400 |
| [ |
| b'@@ -1,962 +1,468 @@\n-# rgToolFactoryMultIn.py\n-# see https://bitbucket.org/fubar/galaxytoolfactory/wiki/Home\n+# rgToolFactory.py\n+# see https://github.com/fubar2/toolfactory\n # \n # copyright ross lazarus (ross stop lazarus at gmail stop com) May 2012\n # \n # all rights reserved\n # Licensed under the LGPL\n-# suggestions for improvement and bug fixes welcome at https://bitbucket.org/fubar/galaxytoolfactory/wiki/Home\n-#\n-# January 2015\n-# unified all setups by passing the script on the cl rather than via a PIPE - no need for treat_bash_special so removed\n-#\n-# in the process of building a complex tool\n-# added ability to choose one of the current toolshed package_r or package_perl or package_python dependencies and source that package\n-# add that package to tool_dependencies\n-# Note that once the generated tool is loaded, it will have that package\'s env.sh loaded automagically so there is no\n-# --envshpath in the parameters for the generated tool and it uses the system one which will be first on the adjusted path.\n-#\n-# sept 2014 added additional params from\n-# https://bitbucket.org/mvdbeek/dockertoolfactory/src/d4863bcf7b521532c7e8c61b6333840ba5393f73/DockerToolFactory.py?at=default\n-# passing them is complex\n-# and they are restricted to NOT contain commas or double quotes to ensure that they can be safely passed together on \n-# the toolfactory command line as a comma delimited double quoted string for parsing and passing to the script\n-# see examples on this tool form\n-\n-# august 2014\n-\n-# Allows arbitrary number of input files\n-# NOTE positional parameters are now passed to script\n-# and output (may be "None") is *before* arbitrary number of inputs\n+# suggestions for improvement and bug fixes welcome at https://github.com/fubar2/toolfactory\n #\n-# march 2014\n-# had to remove dependencies because cross toolshed dependencies are not possible - can\'t pre-specify a toolshed url for graphicsmagick and ghostscript\n-# grrrrr - night before a demo\n-# added dependencies to a tool_dependencies.xml if html page generated so generated tool is properly portable\n-#\n-# added ghostscript and graphicsmagick as dependencies \n-# fixed a wierd problem where gs was trying to use the new_files_path from universe (database/tmp) as ./database/tmp\n-# errors ensued\n-#\n-# august 2013\n-# found a problem with GS if $TMP or $TEMP missing - now inject /tmp and warn\n+# July 2020: BCC was fun and I feel like rip van winkle after 5 years. \n+# Decided to \n+# 1. Fix the toolfactory so it works - done for simplest case\n+# 2. Fix planemo so the toolfactory function works\n+# 3. Rewrite bits using galaxyxml functions where that makes sense - done\n #\n-# july 2013\n-# added ability to combine images and individual log files into html output\n-# just make sure there\'s a log file foo.log and it will be output\n-# together with all images named like "foo_*.pdf\n-# otherwise old format for html\n-#\n-# January 2013\n-# problem pointed out by Carlos Borroto\n-# added escaping for <>$ - thought I did that ages ago...\n-#\n-# August 11 2012 \n-# changed to use shell=False and cl as a sequence\n+# removed all the old complications including making the new tool use this same script\n+# galaxyxml now generates the tool xml https://github.com/hexylena/galaxyxml\n+# No support for automatic HTML file creation from arbitrary outputs\n+# TODO: add option to run that code as a post execution hook\n+# TODO: add additional history input parameters - currently only one \n \n-# This is a Galaxy tool factory for simple scripts in python, R or whatever ails ye.\n-# It also serves as the wrapper for the new tool.\n-# \n-# you paste and run your script\n-# Only works for simple scripts that read one input from the history.\n-# Optionally can write one new history dataset,\n-# and optionally collect any number of outputs into links on an autogenerated HTML page.\n-\n-# DO NOT install on a public or important site - please.\n-\n-# installed generated tools are fine if the script is safe.\n-# They just run normally and th'..b',default=None) \n- a(\'--citations\',default=None)\n- a(\'--additional_parameters\', dest=\'additional_parameters\', action=\'append\', default=[])\n- a(\'--edit_additional_parameters\', action="store_true", default=False)\n- a(\'--envshpath\',default="system") \n- opts, args = op.parse_args()\n- assert not opts.bad_user,\'UNAUTHORISED: %s is NOT authorized to use this tool until Galaxy admin adds %s to admin_users in universe_wsgi.ini\' % (opts.bad_user,opts.bad_user)\n- assert opts.tool_name,\'## Tool Factory expects a tool name - eg --tool_name=DESeq\'\n- assert opts.interpreter,\'## Tool Factory wrapper expects an interpreter - eg --interpreter=Rscript\'\n- assert os.path.isfile(opts.script_path),\'## Tool Factory wrapper expects a script path - eg --script_path=foo.R\'\n- if opts.output_dir:\n- try:\n- os.makedirs(opts.output_dir)\n- except:\n- pass\n- opts.input_tab = [x.replace(\'"\',\'\').replace("\'",\'\') for x in opts.input_tab]\n- for i,x in enumerate(opts.additional_parameters): # remove quotes we need to deal with spaces in CL params\n- opts.additional_parameters[i] = opts.additional_parameters[i].replace(\'"\',\'\')\n- r = ScriptRunner(opts)\n- if opts.make_Tool:\n- retcode = r.makeTooltar()\n- else:\n- retcode = r.run()\n- os.unlink(r.sfile)\n- if retcode:\n- sys.exit(retcode) # indicate failure to job runner\n+\tu = """\n+\tThis is a Galaxy wrapper. It expects to be called by a special purpose tool.xml as:\n+\t<command interpreter="python">rgBaseScriptWrapper.py --script_path "$scriptPath" --tool_name "foo" --interpreter "Rscript"\n+\t</command>\n+\t"""\n+\tparser = argparse.ArgumentParser()\n+\ta = parser.add_argument\n+\ta(\'--script_path\',default=\'\')\n+\ta(\'--tool_name\',default=None)\n+\ta(\'--interpreter_name\',default=None)\n+\ta(\'--interpreter_version\',default=None)\n+\ta(\'--exe_package\',default=None)\n+\ta(\'--exe_package_version\',default=None)\n+\ta(\'--output_dir\',default=\'./\')\n+\ta(\'--input_files\',default=[], action="append") # these are "galaxypath,metadataname" pairs\n+\ta("--input_formats",default="tabular")\n+\ta(\'--output_tab\',default=None)\n+\ta(\'--output_format\',default=\'tabular\')\n+\ta(\'--output_cl\',default=None)\n+\ta(\'--user_email\',default=\'Unknown\')\n+\ta(\'--bad_user\',default=None)\n+\ta(\'--make_Tool\',default=None)\n+\ta(\'--help_text\',default=None)\n+\ta(\'--tool_desc\',default=None)\n+\ta(\'--new_tool\',default=None)\n+\ta(\'--tool_version\',default=None)\n+\ta(\'--include_dependencies\',default=None) \n+\ta(\'--citations\',default=None)\n+\ta(\'--additional_parameters\', dest=\'additional_parameters\', action=\'append\', default=[])\n+\ta(\'--edit_additional_parameters\', action="store_true", default=False)\n+\ta(\'--parampass\',default="positional") \n+\targs = parser.parse_args()\n+\tassert not args.bad_user,\'UNAUTHORISED: %s is NOT authorized to use this tool until Galaxy admin adds %s to "admin_users" in the Galaxy configuration file\' % (args.bad_user,args.bad_user)\n+\tassert args.tool_name,\'## Tool Factory expects a tool name - eg --tool_name=DESeq\'\n+\tassert (args.interpreter_name or args.exe_package),\'## Tool Factory wrapper expects an interpreter - eg --interpreter_name=Rscript or an executable package findable by the dependency management package\'\n+\tassert args.exe_package or (len(args.script_path) > 0 and os.path.isfile(args.script_path)),\'## Tool Factory wrapper expects a script path - eg --script_path=foo.R if no executable\'\n+\tif args.output_dir:\n+\t\ttry:\n+\t\t\tos.makedirs(args.output_dir)\n+\t\texcept:\n+\t\t\tpass\n+\targs.input_files = [x.replace(\'"\',\'\').replace("\'",\'\') for x in args.input_files]\n+\tfor i,x in enumerate(args.additional_parameters): # remove quotes we need to deal with spaces in CL params\n+\t\targs.additional_parameters[i] = args.additional_parameters[i].replace(\'"\',\'\')\n+\tr = ScriptRunner(args)\n+\tif args.make_Tool:\n+\t\tretcode = r.makeTooltar()\n+\telse:\n+\t\tretcode = r.run()\n+\tif retcode:\n+\t\tsys.exit(retcode) # indicate failure to job runner\n \n \n if __name__ == "__main__":\n- main()\n+\tmain()\n \n \n' |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 rgToolFactory2.xml --- a/rgToolFactory2.xml Mon Mar 02 05:18:21 2015 -0500 +++ b/rgToolFactory2.xml Thu Jul 30 06:48:45 2020 -0400 |
| [ |
| b'@@ -1,69 +1,178 @@\n-<tool id="rgTF2" name="Tool Factory Two" version="1.17">\n+<tool id="rgTF2" name="ToolFactory" version="2.00">\n <description>Scripts into tools</description>\n+ <macros>\n+ <xml name="addpar">\n+\t\t<param name="edit_params" type="select" display="radio" label="Add all additional parameters to the generated tool form so they are user editable?" \n+\t\t\t help="If no (default), users will NOT be able to alter any additional parameters. If yes, these will appear on the tool form as text fields with no validation or sanitizing">\n+\t\t\t<option value="yes">Yes, allow user to edit all additional parameters on the generated tool form</option>\n+\t\t\t<option value="no" selected="true">No - use the fixed values for all additional parameters - no user editing</option>\n+\t\t</param>\n+\t\t<repeat name="additional_history_inputs" title="Additional Current history data files to pass in to the script - Use the \'+\' button to add as many as needed"\n+\t\t\t help="Each of these will become a history item selector as input for users of the new tool you are making">\n+\t\t\t<param name="input_files" type="data" format="data" label="Select an input file from your history" optional="true" size="120" multiple="false"\n+\t\t\t help=""/>\n+\t\t\t<param name="input_formats" type="select" multiple="true" label="Select the datatype(s) that your tool/script accepts as input"\n+\t\t\t help="If your datatype is not listed here, it has to be added in galaxy\'s datatypes_conf.xml" value="tabular">\n+\t\t\t <options from_parameter="tool.app.datatypes_registry.upload_file_formats">\n+\t\t\t\t<column name="value" index="0"/>\n+\t\t\t </options>\n+\t\t\t</param>\n+\t\t\t<param name="param_label" type="text" value="" label="This will become the user prompt for the form so please make it informative" size="60" \n+\t\t\t help="Note that \'~~~\' is an internal delimiter so must not appear in this text field - please work around this technical limitation" >\n+\t\t\t<sanitizer invalid_char="">\n+\t\t\t <valid initial="string.printable"> <remove value=\'~~~\'/> </valid>\n+\t\t\t <mapping initial="none"/>\n+\t\t\t</sanitizer>\n+\t\t\t</param>\n+\t\t\t<param name="param_help" type="text" value="parameter_help" label="This will become help text on the form."\n+\t\t\t help="Note that three consecutive ~ cannot be used in this text field - please work around this technical limitation" size="60">\n+\t\t\t<sanitizer invalid_char="">\n+\t\t\t <valid initial="string.printable"> <remove value=\'~~~\'/> </valid>\n+\t\t\t <mapping initial="none"/>\n+\t\t\t</sanitizer>\n+\t\t\t</param>\n+\t\t\t<param name="CL" type="text" size="60" label="Positional: ordinal integer. Argparse: argument name."\n+\t\t\t help="If you will pass positional parameters, enter the integer ordinal for this parameter. If Argparse style, \'--\' will be prepended or \'-\' if single character" value="">\t\t \n+\t\t\t</param>\n+\t\t</repeat>\n+\t\t<repeat name="additional_parameters" title="Pass additional parameters to the script"\n+\t\t\t help="See examples below to see how these can be parsed by scripts in the various languages">\n+\t\t <param name="param_name" type="text" value="parameter_name" label="Choose the name for this parameter" size="60">\n+\t\t\t<sanitizer invalid_char="">\n+\t\t\t <valid initial="string.letters,string.digits"/>\n+\t\t\t <mapping initial="none"/> \n+\t\t\t</sanitizer>\n+\t\t </param>\n+\t\t <param name="param_type" type="select" label="Select the type for this parameter">\n+\t\t\t<option value="text" selected="true">text</option>\n+\t\t\t<option value="integer">integer</option>\n+\t\t\t<option value="float">float</option>\n+\t\t </param>\n+\t\t <param name="param_value" type="text" value="parameter_value" label="Enter this parameter\'s default value" size="60"\n+\t\t\thelp="Note that \'~~~\' is an internal delimiter must not appear in this text field - please work around this technical limitation" >\n+\t\t\t<sanitizer invalid_char="">\n+\t\t\t <valid initial="string.printable"> <remove value=\'~~~\'/> </valid>\n+\t\t\t <mapping initial="none"/>\n+\t\t\t</sanitizer>\n+\t\t </param>\n+\t\t <param name="param_label" type="text" value="parameter_label'..b' nicely presented as links, with pdf images linked as thumbnails in that output.\n-This can be handy for complex scripts creating lots of output.\n+Here\'s a sample python script that can be cut and pasted into the tool form, suitable for positional parameter passing:\n \n-<![CDATA[\n-\n-**Multiple inputs**\n+::\n \n-Your script will receive up to 3 named parameters\n-INPATHS is a comma separated list of input file paths\n-INNAMES is a comma separated list of input file names in the same order\n-OUTPATH is optional if a file is being generated, your script should write there\n-Your script should open and write files in the provided working directory if you are using the Html\n-automatic presentation option.\n-\n-Python script command lines will have --INPATHS and --additional_arguments etc. to make it easy to use argparse\n-\n-Rscript will need to use commandArgs(TRUE) - see the example below - additional arguments will\n-appear as themselves - eg foo="bar" will mean that foo is defined as "bar" for the script.\n-\n-Bash and sh will see any additional parameters on their command lines and the 3 named parameters\n-in their environment magically - well, using env on the CL\n-\n-***python***::\n-\n- # argparse for 3 possible comma separated lists\n- # additional parameters need to be parsed !\n- # then echo parameters to the output file\n- import sys\n- import os\n- import argparse\n+\t# reverse order of text by row\n+\timport sys\n+\tinp = sys.argv[1]\n+\toutp = sys.argv[2]\n+\ti = open(inp,\'r\').readlines()\n+\to = open(outp,\'w\')\n+\tfor row in i:\n+\t rs = row.rstrip()\n+\t rs = list(rs)\n+\t rs.reverse()\n+\t o.write(\'\'.join(rs))\n+\t o.write(\'\\n\')\n+\to.close()\n \n- argp=argparse.ArgumentParser()\n- argp.add_argument(\'--innames\',default=None)\n- argp.add_argument(\'--inpaths\',default=None)\n- argp.add_argument(\'--outpath\',default=None)\n- argp.add_argument(\'--additional_parameters\',default=[],action="append")\n- argp.add_argument(\'otherargs\', nargs=argparse.REMAINDER)\n- args = argp.parse_args()\n- opath = args.outpath\n- odir = os.path.split(opath)[0]\n- try:\n- os.makedirs(odir)\n- print \'### made\',odir\n- except:\n- pass\n- f= open(opath,\'w\')\n- s = \'### args=%s\\n\' % str(args)\n- f.write(s)\n- s = \'sys.argv=%s\\n\' % sys.argv\n- f.write(s)\n- f.close()\n-\n-\n-***Rscript***::\n+With argparse style parameters:\n+\t\n+::\n \n- # tool factory Rscript parser suggested by Forester\n- # http://www.r-bloggers.com/including-arguments-in-r-cmd-batch-mode/\n- # additional parameters will appear in the ls() below - they are available\n- # to your script\n- # echo parameters to the output file\n- ourargs = commandArgs(TRUE)\n- if(length(ourargs)==0){\n- print("No arguments supplied.")\n- }else{\n- for(i in 1:length(ourargs)){\n- eval(parse(text=ourargs[[i]]))\n- }\n- sink(OUTPATH)\n- cat(\'INPATHS=\',INPATHS,\'\\n\')\n- cat(\'INNAMES=\',INNAMES,\'\\n\')\n- cat(\'OUTPATH=\',OUTPATH,\'\\n\')\n- x=ls()\n- cat(\'all objects=\',x,\'\\n\')\n- sink()\n- }\n- sessionInfo()\n- print.noquote(date())\n-\n-\n-***bash/sh***::\n-\n- # tool factory sets up these environmental variables\n- # this example writes those to the output file\n- # additional params appear on command line\n- if [ ! -f "$OUTPATH" ] ; then\n- touch "$OUTPATH"\n- fi\n- echo "INPATHS=$INPATHS" >> "$OUTPATH"\n- echo "INNAMES=$INNAMES" >> "$OUTPATH"\n- echo "OUTPATH=$OUTPATH" >> "$OUTPATH"\n- echo "CL=$@" >> "$OUTPATH"\n-\n-***perl***::\n-\n- (my $INPATHS,my $INNAMES,my $OUTPATH ) = @ARGV;\n- open(my $fh, \'>\', $OUTPATH) or die "Could not open file \'$OUTPATH\' $!";\n- print $fh "INPATHS=$INPATHS\\n INNAMES=$INNAMES\\n OUTPATH=$OUTPATH\\n";\n- close $fh;\n- \n-\n-]]>\n-\n-\n+\t# reverse order of text by row\n+\timport argparse\n+\tparser = argparse.ArgumentParser()\n+\ta = parser.add_argument\n+\ta(\'--infile\',default=\'\')\n+\ta(\'--outfile\',default=None)\n+\targs = parser.parse_args()\n+\tinp = args.infile\n+\toutp = args.outfile\n+\ti = open(inp,\'r\').readlines()\n+\to = open(outp,\'w\')\n+\tfor row in i:\n+\t rs = row.rstrip()\n+\t rs = list(rs)\n+\t rs.reverse()\n+\t o.write(\'\'.join(rs))\n+\t o.write(\'\\n\')\n+\to.close()\n+\t \n \n Paper_ :\n \n' |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/infile.tabular --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/infile.tabular Thu Jul 30 06:48:45 2020 -0400 |
| b |
| @@ -0,0 +1,7 @@ +0.1 +0.00001 +0.2 +0.3 +0.004 +0.5 +0.000000002 |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/reverseargp2.toolshed.gz |
| b |
| Binary file test-data/reverseargp2.toolshed.gz has changed |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/runme.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/runme.py Thu Jul 30 06:48:45 2020 -0400 |
| b |
| @@ -0,0 +1,19 @@ +# reverse order of columns in a tabular file +import argparse +parser = argparse.ArgumentParser() +a = parser.add_argument +a('--infile',default='') +a('--outfile',default=None) +a('--prefix',default=None) +args = parser.parse_args() +inp = args.infile +outp = args.outfile +i = open(inp,'r').readlines() +o = open(outp,'w') +for row in i: + rs = row.rstrip() + rs = list(rs) + rs.reverse() + o.write('%s:%s' % (args.prefix,''.join(rs))) + o.write('\n') +o.close() |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/tf2_test.html --- a/test-data/tf2_test.html Mon Mar 02 05:18:21 2015 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 |
| b |
| @@ -1,25 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> - <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> - <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> - <meta name="generator" content="Galaxy rgToolFactory2.py tool output - see http://g2.trac.bx.psu.edu/" /> - <title></title> - <link rel="stylesheet" href="/static/style/base.css" type="text/css" /> - </head> - <body> - <div class="toolFormBody"> - -<div class="infomessage">Galaxy Tool "tf2_test" run at 29/01/2015 11:24:54</div><br/> -<div class="toolFormTitle">tf2 log output</div> -tf2_test_error.log is empty<br/> -<div class="toolFormTitle">Other log output</div> -/tmp/tmpdRM3H0/job_working_directory/000/2/dataset_3_files/tf2_test_runner.log is empty<br/> -<div class="toolFormTitle">All output files available for downloading</div> - -<div><table class="colored" cellpadding="3" cellspacing="3"><tr><th>Output File Name (click to view)</th><th>Size</th></tr> - -<tr><td><a href="tf2_test.python">tf2_test.python</a></td><td>0 B</td></tr> -<tr class="odd_row"><td><a href="tf2_test_error.log">tf2_test_error.log</a></td><td>0 B</td></tr> -<tr><td><a href="tf2_test_runner.log">tf2_test_runner.log</a></td><td>226 B</td></tr> -</table></div><br/> -</div></body></html> - |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/tf2_test.toolshed.gz |
| b |
| Binary file test-data/tf2_test.toolshed.gz has changed |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/tf2_test_in.xls --- a/test-data/tf2_test_in.xls Mon Mar 02 05:18:21 2015 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 |
| b |
| @@ -1,3 +0,0 @@ -col1 col2 col3 -r11 r12 r13 -r21 r22 r23 |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 test-data/tf2_test_runme.py --- a/test-data/tf2_test_runme.py Mon Mar 02 05:18:21 2015 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 |
| [ |
| @@ -1,25 +0,0 @@ -### bog standard argparse for 3 possible comma separated lists -## followed by some silly reverse each row code provided as an example -## you're supposed to replace it with your great code.. -import sys -import argparse -import copy -argp=argparse.ArgumentParser() -argp.add_argument('--INNAMES',default=None) -argp.add_argument('--INPATHS',default=None) -argp.add_argument('--OUTPATH',default=None) -argp.add_argument('--additional_parameters',default=[],action="append") -argp.add_argument('otherargs', nargs=argparse.REMAINDER) -args = argp.parse_args() -fout = open(args.OUTPATH,'w') -sins = open(args.INPATHS.split(',')[0]).readlines() -for i,sin in enumerate(sins): - row = sin.strip().split('\t') - rrow = copy.copy(row) - lrow = len(row) - if (lrow > 1): - for j in range(lrow): - rrow[j] = row[lrow-j-1] - fout.write('\t'.join(rrow)) - fout.write('\n') -fout.close() |
| b |
| diff -r 9fe74bd23af2 -r db35d39e1de9 tool_dependencies.xml --- a/tool_dependencies.xml Mon Mar 02 05:18:21 2015 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 |
| b |
| @@ -1,12 +0,0 @@ -<?xml version="1.0"?> -<tool_dependency> - <package name="ghostscript" version="9.10"> - <repository changeset_revision="a285e78179bd" name="package_ghostscript_9_10" owner="devteam" toolshed="https://toolshed.g2.bx.psu.edu" /> - </package> - <package name="graphicsmagick" version="1.3.20"> - <repository changeset_revision="d87c457bbbc2" name="package_graphicsmagick_1_3_20" owner="iuc" toolshed="https://toolshed.g2.bx.psu.edu" /> - </package> - <readme> - Only Admins can use this tool generator but please do NOT install on a public facing Galaxy as it exposes unrestricted scripting as your Galaxy user - </readme> -</tool_dependency> |