| Previous changeset 118:e43c43396a70 (2020-12-11) Next changeset 120:0c6c3e10a8f4 (2021-01-07) |
|
Commit message:
Uploaded |
|
modified:
toolfactory/README.md toolfactory/rgToolFactory2.py toolfactory/rgToolFactory2.xml |
|
added:
toolfactory/.planemo.yml toolfactory/.shed.yml toolfactory/images/TFasIDE.png toolfactory/images/dynamicScriptTool.png toolfactory/images/hello_toolfactory_form.png toolfactory/install_tf_demos.py toolfactory/install_tf_demos_toolshed.tgz toolfactory/planemo_install_tfdemo.tar.gz |
|
removed:
toolfactory/galaxy-tool-test toolfactory/testtf.sh |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/.planemo.yml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/toolfactory/.planemo.yml Tue Jan 05 00:34:48 2021 +0000 |
| b |
| @@ -0,0 +1,23 @@ +## Planemo Global Configuration File. +## Everything in this file is completely optional - these values can all be +## configured via command line options for the corresponding commands. + +## Specify a default galaxy_root for test and server commands here. +galaxy_root: /galaxy-central +## Username used with toolshed(s). +shed_username: galaxy +sheds: + # For each tool shed you wish to target, uncomment key or both email and + # password. + toolshed: + #key: "<TODO>" + #email: "<TODO>" + #password: "<TODO>" + testtoolshed: + #key: "<TODO>" + #email: "<TODO>" + #password: "<TODO>" + local: + key: "fakekey" + email: "admin@galaxy.org" + password: "password" |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/.shed.yml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/toolfactory/.shed.yml Tue Jan 05 00:34:48 2021 +0000 |
| b |
| @@ -0,0 +1,13 @@ +name: toolfactory +owner: fubar +description: ToolFactory - tool to make Galaxy tools ready for the toolshed +homepage_url: https://github.com/fubar2/toolfactory +long_description: | + ToolFactory - turn executable packages and R/python/perl/bash scripts into ordinary Galaxy tools + + Creating re-usable tools from scripts: The Galaxy Tool Factory Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team + Bioinformatics 2012; doi: 10.1093/bioinformatics/bts573 +remote_repository_url: https://github.com/fubar2/toolfactory +type: tool_dependency_definition +categories: +- Tool Generators |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/README.md --- a/toolfactory/README.md Fri Dec 11 04:23:48 2020 +0000 +++ b/toolfactory/README.md Tue Jan 05 00:34:48 2021 +0000 |
| [ |
| b'@@ -1,55 +1,255 @@\n-\xef\xbb\xbf**Breaking news! Docker container is recommended as at August 2020**\n+\xef\xbb\xbf## Breaking news! Docker container at https://github.com/fubar2/toolfactory-galaxy-docker recommended as at December 2020\n \n-A Docker container can be built - see the docker directory.\n-It is highly recommended for isolation. It also has an integrated toolshed to allow installation of new tools back \n-into the Galaxy being used to generate them. \n+## This is the original ToolFactory suitable for non-docker situations. Please use the docker container if you can because it\'s integrated with a Toolshed...\n \n-Built from quay.io/bgruening/galaxy:20.05 but updates the\n-Galaxy code to the dev branch - it seems to work fine with updated bioblend>=0.14\n-with planemo and the right version of gxformat2 needed by the ToolFactory (TF).\n+# WARNING\n \n-The runclean.sh script run from the docker subdirectory of your local clone of this repository\n-should create a container (eventually) and serve it at localhost:8080 with a toolshed at\n-localhost:9009.\n+Install this tool to a throw-away private Galaxy or Docker container ONLY!\n \n-Once it\'s up, please restart Galaxy in the container with \n-```docker exec [container name] supervisorctl restart galaxy: ```\n-Jobs just do not seem to run properly otherwise and the next steps won\'t work!\n-\n-The generated container includes a workflow and 2 sample data sets for the workflow\n+Please NEVER on a public or production instance where a hostile user may\n+be able to gain access if they can acquire an administrative account login.\n \n-Load the workflow. Adjust the inputs for each as labelled. The perl example counts GC in phiX.fasta. \n-The python scripts use the rgToolFactory.py as their input - any text file will work but I like the\n-recursion. The BWA example has some mitochondrial reads and reference. Run the workflow and watch.\n-This should fill the history with some sample tools you can rerun and play with.\n-Note that each new tool will have been tested using Planemo. In the workflow, in Galaxy.\n-Extremely cool to watch.\n+It only runs for server administrators - the ToolFactory tool will refuse to execute for an ordinary user since\n+it can install new tools to the Galaxy server it executes on! This is not something you should allow other than\n+on a throw away instance that is protected from potentially hostile users.\n \n-*WARNING* \n-\n- Install this tool on a throw-away private Galaxy or Docker container ONLY\n- Please NEVER on a public or production instance\n-\n-*Short Story*\n+## Short Story\n \n Galaxy is easily extended to new applications by adding a new tool. Each new scientific computational package added as\n-a tool to Galaxy requires some special instructions to be written. This is sometimes termed "wrapping" the package\n-because the instructions tell Galaxy how to run the package as a new Galaxy tool. Any tool in a Galaxy is \n-readily available to all the users through a consistent and easy to use interface.\n+a tool to Galaxy requires an XML document describing how the application interacts with Galaxy.\n+This is sometimes termed "wrapping" the package because the instructions tell Galaxy how to run the package\n+as a new Galaxy tool. Any tool that has been wrapped is readily available to all the users through a consistent\n+and easy to use interface once installed in the local Galaxy server.\n+\n+Most Galaxy tool wrappers have been manually prepared by skilled programmers, many using Planemo because it\n+automates much of the boilerplate and makes the process much easier.\n+The ToolFactory (TF) now uses Planemo under the hood for testing, but hides the command\n+line complexities. The user will still need appropriate skills in terms of describing the interface between\n+Galaxy and the new application, but will be helped by a Galaxy tool form to collect all the needed\n+settings, together with automated testing and uploading to a toolshed with optional local installation.\n+\n+\n+## ToolFactory generated tools are ordinar'..b'shed fussing. Once everything quietens\n-down, find the container with\n-```docker ps```\n-and use\n-```docker exec [containername] supervisorctl restart galaxy:```\n-That colon is not a typographical mistake.\n-Not restarting after first boot seems to leave the job/worflow system confused and the workflow\n-just will not run properly until Galaxy has restarted.\n+The Docker container https://github.com/fubar2/toolfactory-galaxy-docker/blob/main/README.md\n+is the best way to use the TF because it is preconfigured\n+to automate new tool testing and has a built in local toolshed where each new tool\n+is uploaded. If you grab the docker container, it should just work after a restart and you\n+can run a workflow to generate all the sample tools. Running the samples and rerunning the ToolFactory\n+jobs that generated them allows you to add fields and experiment to see how things work.\n \n-Login as admin@galaxy.org with password "password". Feel free to change it once you are logged in. \n-There should be a companion toolshed at localhost:9090. The history should have some sample data for\n-the workflow.\n-\n-Run the workflow and make sure the right dataset is selected for each of the input files. Most of the \n-examples use text files so should run, but the bwa example needs the right ones to work properly.\n-\n-When the workflow is finished, you will have half a dozen examples to rerun and play with. They have also \n-all been tested and installed so you should find them in your tool menu under "Generated Tools"\n-\n-It is easy to install without Docker, but you will need to make some \n+It can be installed like any other tool from the Toolshed, but you will need to make some\n configuration changes (TODO write a configuration). You can install it most conveniently using the\n administrative "Search and browse tool sheds" link. Find the Galaxy Main\n toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory\n repository in the Tool Maker section. Open it and review the code and select the option to install it.\n \n-Otherwise, if not already there pending an accepted PR,\n-please add:\n-<datatype extension="tgz" type="galaxy.datatypes.binary:Binary"\n-mimetype="multipart/x-gzip" subclass="True" />\n-to your local data_types_conf.xml.\n+If not already there please add:\n+\n+```\n+<datatype extension="tgz" type="galaxy.datatypes.binary:Binary" mimetype="multipart/x-gzip" subclass="True" />\n+```\n+\n+to your local config/data_types_conf.xml.\n \n \n-*Restricted execution*\n+## Restricted execution\n+\n+The tool factory tool itself will ONLY run for admin users -\n+people with IDs in config/galaxy.yml "admin_users".\n \n-The tool factory tool itself will then be usable ONLY by admin users -\n-people with IDs in admin_users. **Yes, that\'s right. ONLY\n-admin_users can run this tool** Think about it for a moment. If allowed to\n-run any arbitrary script on your Galaxy server, the only thing that would\n-impede a miscreant bent on destroying all your Galaxy data would probably\n-be lack of appropriate technical skills.\n+*ONLY admin_users can run this tool*\n \n-**Generated tool Security**\n+That doesn\'t mean it\'s safe to install on a shared or exposed instance - please don\'t.\n+\n+## Generated tool Security\n \n Once you install a generated tool, it\'s just\n another tool - assuming the script is safe. They just run normally and their\n user cannot do anything unusually insecure but please, practice safe toolshed.\n Read the code before you install any tool. Especially this one - it is really scary.\n \n-**Send Code**\n-\n-Pull requests and suggestions welcome as git issues please?\n-\n-**Attribution**\n+## Attribution\n \n Creating re-usable tools from scripts: The Galaxy Tool Factory\n Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team\n@@ -200,12 +378,3 @@\n \n http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref\n \n-**Licensing**\n-\n-Copyright Ross Lazarus 2010\n-ross lazarus at g mail period com\n-\n-All rights reserved.\n-\n-Licensed under the LGPL\n-\n' |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/galaxy-tool-test --- a/toolfactory/galaxy-tool-test Fri Dec 11 04:23:48 2020 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 |
| [ |
| b'@@ -1,457 +0,0 @@\n-#!/usr/bin/env python\n-\n-import argparse\n-import datetime as dt\n-import json\n-import logging\n-import os\n-import sys\n-import tempfile\n-from collections import namedtuple\n-from concurrent.futures import thread, ThreadPoolExecutor\n-\n-import yaml\n-\n-from galaxy.tool_util.verify.interactor import (\n- DictClientTestConfig,\n- GalaxyInteractorApi,\n- verify_tool,\n-)\n-\n-DESCRIPTION = """Script to quickly run a tool test against a running Galaxy instance."""\n-DEFAULT_SUITE_NAME = "Galaxy Tool Tests"\n-ALL_TESTS = -1\n-ALL_TOOLS = "*"\n-ALL_VERSION = "*"\n-LATEST_VERSION = None\n-\n-\n-TestReference = namedtuple("TestReference", ["tool_id", "tool_version", "test_index"])\n-TestException = namedtuple("TestException", ["tool_id", "exception", "was_recorded"])\n-\n-\n-class Results:\n-\n- def __init__(self, default_suitename, test_json, append=False):\n- self.test_json = test_json or "-"\n- test_results = []\n- test_exceptions = []\n- suitename = default_suitename\n- if append:\n- assert test_json != "-"\n- with open(test_json) as f:\n- previous_results = json.load(f)\n- test_results = previous_results["tests"]\n- if "suitename" in previous_results:\n- suitename = previous_results["suitename"]\n- self.test_results = test_results\n- self.test_exceptions = test_exceptions\n- self.suitename = suitename\n-\n- def register_result(self, result):\n- self.test_results.append(result)\n-\n- def register_exception(self, test_exception):\n- self.test_exceptions.append(test_exception)\n-\n- def already_successful(self, test_reference):\n- test_id = _test_id_for_reference(test_reference)\n- for test_result in self.test_results:\n- if test_result.get(\'id\') != test_id:\n- continue\n-\n- has_data = test_result.get(\'has_data\', False)\n- if has_data:\n- test_data = test_result.get("data", {})\n- if \'status\' in test_data and test_data[\'status\'] == \'success\':\n- return True\n-\n- return False\n-\n- def write(self):\n- tests = sorted(self.test_results, key=lambda el: el[\'id\'])\n- n_passed, n_failures, n_skips = 0, 0, 0\n- n_errors = len([e for e in self.test_exceptions if not e.was_recorded])\n- for test in tests:\n- has_data = test.get(\'has_data\', False)\n- if has_data:\n- test_data = test.get("data", {})\n- if \'status\' not in test_data:\n- raise Exception(f"Test result data {test_data} doesn\'t contain a status key.")\n- status = test_data[\'status\']\n- if status == "success":\n- n_passed += 1\n- elif status == "error":\n- n_errors += 1\n- elif status == "skip":\n- n_skips += 1\n- elif status == "failure":\n- n_failures += 1\n- report_obj = {\n- \'version\': \'0.1\',\n- \'suitename\': self.suitename,\n- \'results\': {\n- \'total\': n_passed + n_failures + n_skips + n_errors,\n- \'errors\': n_errors,\n- \'failures\': n_failures,\n- \'skips\': n_skips,\n- },\n- \'tests\': tests,\n- }\n- if self.test_json == "-":\n- print(json.dumps(report_obj))\n- else:\n- with open(self.test_json, "w") as f:\n- json.dump(report_obj, f)\n-\n- def info_message(self):\n- messages = []\n- passed_tests = self._tests_with_status(\'success\')\n- messages.append("Passed tool tests ({}): {}".format(\n- len(passed_tests),\n- [t["id"] for t in passed_tests]\n- ))\n- failed_tests = self._tests_with_status(\'failure\')\n- messages.append("Failed tool tests ({}): {}".format(\n- len(failed_tests),\n-'..b' logger.setLevel(logging.DEBUG if verbose else logging.INFO)\n- logger.addHandler(console)\n-\n- if not log_file:\n- # delete = false is chosen here because it is always nice to have a log file\n- # ready if you need to debug. Not having the "if only I had set a log file"\n- # moment after the fact.\n- temp = tempfile.NamedTemporaryFile(prefix="ephemeris_", delete=False)\n- log_file = temp.name\n- file_handler = logging.FileHandler(log_file)\n- logger.addHandler(file_handler)\n- logger.info(f"Storing log file in: {log_file}")\n- return logger\n-\n-\n-def _arg_parser():\n- parser = argparse.ArgumentParser(description=DESCRIPTION)\n- parser.add_argument(\'-u\', \'--galaxy-url\', default="http://localhost:8080", help=\'Galaxy URL\')\n- parser.add_argument(\'-k\', \'--key\', default=None, help=\'Galaxy User API Key\')\n- parser.add_argument(\'-a\', \'--admin-key\', default=None, help=\'Galaxy Admin API Key\')\n- parser.add_argument(\'--force_path_paste\', default=False, action="store_true", help=\'This requires Galaxy-side config option "allow_path_paste" enabled. Allows for fetching test data locally. Only for admins.\')\n- parser.add_argument(\'-t\', \'--tool-id\', default=ALL_TOOLS, help=\'Tool ID\')\n- parser.add_argument(\'--tool-version\', default=None, help=\'Tool Version (if tool id supplied). Defaults to just latest version, use * to test all versions\')\n- parser.add_argument(\'-i\', \'--test-index\', default=ALL_TESTS, type=int, help=\'Tool Test Index (starting at 0) - by default all tests will run.\')\n- parser.add_argument(\'-o\', \'--output\', default=None, help=\'directory to dump outputs to\')\n- parser.add_argument(\'--append\', default=False, action="store_true", help="Extend a test record json (created with --output-json) with additional tests.")\n- parser.add_argument(\'--skip-successful\', default=False, action="store_true", help="When used with --append, skip previously run successful tests.")\n- parser.add_argument(\'-j\', \'--output-json\', default=None, help=\'output metadata json\')\n- parser.add_argument(\'--verbose\', default=False, action="store_true", help="Verbose logging.")\n- parser.add_argument(\'-c\', \'--client-test-config\', default=None, help="Test config YAML to help with client testing")\n- parser.add_argument(\'--suite-name\', default=DEFAULT_SUITE_NAME, help="Suite name for tool test output")\n- parser.add_argument(\'--with-reference-data\', dest="with_reference_data", default=False, action="store_true")\n- parser.add_argument(\'--skip-with-reference-data\', dest="with_reference_data", action="store_false", help="Skip tests the Galaxy server believes use data tables or loc files.")\n- parser.add_argument(\'--history-per-suite\', dest="history_per_test_case", default=False, action="store_false", help="Create new history per test suite (all tests in same history).")\n- parser.add_argument(\'--history-per-test-case\', dest="history_per_test_case", action="store_true", help="Create new history per test case.")\n- parser.add_argument(\'--no-history-cleanup\', default=False, action="store_true", help="Perserve histories created for testing.")\n- parser.add_argument(\'--parallel-tests\', default=1, type=int, help="Parallel tests.")\n- parser.add_argument(\'--retries\', default=0, type=int, help="Retry failed tests.")\n- parser.add_argument(\'--page-size\', default=0, type=int, help="If positive, use pagination and just run one \'page\' to tool tests.")\n- parser.add_argument(\'--page-number\', default=0, type=int, help="If page size is used, run this \'page\' of tests - starts with 0.")\n- parser.add_argument(\'--download-attempts\', default=1, type=int, help="Galaxy may return a transient 500 status code for download if test results are written but not yet accessible.")\n- parser.add_argument(\'--download-sleep\', default=1, type=int, help="If download attempts is greater than 1, the amount to sleep between download attempts.")\n- return parser\n-\n-\n-if __name__ == "__main__":\n- main()\n' |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/images/TFasIDE.png |
| b |
| Binary file toolfactory/images/TFasIDE.png has changed |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/images/dynamicScriptTool.png |
| b |
| Binary file toolfactory/images/dynamicScriptTool.png has changed |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/images/hello_toolfactory_form.png |
| b |
| Binary file toolfactory/images/hello_toolfactory_form.png has changed |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/install_tf_demos.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/toolfactory/install_tf_demos.py Tue Jan 05 00:34:48 2021 +0000 |
| b |
| @@ -0,0 +1,47 @@ +import argparse +import os +import subprocess +import sys +import urllib.request + +from bioblend import galaxy + +WF = "https://drive.google.com/uc?export=download&id=13xE8o7tucHGNA0qYkEP98FfUGl2wdOU5" +HIST = ( + "https://drive.google.com/uc?export=download&id=1V0ZN9ZBuqcGJvt2AP7s3g0q11uYEhdDB" +) +WF_FILE = "tf_workflow.ga" +HIST_FILE = "tf_history.tgz" + + +def _parser(): + parser = argparse.ArgumentParser() + parser.add_argument( + "-g", "--galaxy", help="URL of target galaxy", default="http://localhost:9090" + ) + parser.add_argument("-a", "--key", help="Galaxy admin key", default=None) + return parser + + +def main(): + """ + load the planemo tool_factory demonstration history and tool generating workflow + fails in planemo served galaxies because there seems to be no user in trans? + """ + args = _parser().parse_args() + urllib.request.urlretrieve(WF, WF_FILE) + urllib.request.urlretrieve(HIST, HIST_FILE) + assert args.key, "Need an administrative key for the target Galaxy supplied please" + wfp = os.path.abspath(WF_FILE) + hp = os.path.abspath(HIST_FILE) + gi = galaxy.GalaxyInstance( + url=args.galaxy, key=args.key, email="planemo@galaxyproject.org" + ) + x = gi.workflows.import_workflow_from_local_path(WF_FILE, publish=True) + print(f"installed {WF_FILE} Returned = {x}\n") + x = gi.histories.import_history(file_path=HIST_FILE) + print(f"installed {HIST_FILE} Returned = {x}\n") + + +if __name__ == "__main__": + main() |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/install_tf_demos_toolshed.tgz |
| b |
| Binary file toolfactory/install_tf_demos_toolshed.tgz has changed |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/planemo_install_tfdemo.tar.gz |
| b |
| Binary file toolfactory/planemo_install_tfdemo.tar.gz has changed |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/rgToolFactory2.py --- a/toolfactory/rgToolFactory2.py Fri Dec 11 04:23:48 2020 +0000 +++ b/toolfactory/rgToolFactory2.py Tue Jan 05 00:34:48 2021 +0000 |
| [ |
| b'@@ -13,18 +13,9 @@\n # 1. Fix the toolfactory so it works - done for simplest case\n # 2. Fix planemo so the toolfactory function works\n # 3. Rewrite bits using galaxyxml functions where that makes sense - done\n-#\n-# uses planemo in a biodocker sort of image as a requirement\n-# otherwise planemo seems to leak dependencies back into the\n-# calling venv. Hilarity ensues.\n-\n-\n \n import argparse\n import copy\n-import datetime\n-import grp\n-import json\n import logging\n import os\n import re\n@@ -35,12 +26,9 @@\n import tempfile\n import time\n \n-\n from bioblend import ConnectionError\n from bioblend import toolshed\n \n-import docker\n-\n import galaxyxml.tool as gxt\n import galaxyxml.tool.parameters as gxtp\n \n@@ -54,8 +42,9 @@\n toolFactoryURL = "https://github.com/fubar2/toolfactory"\n ourdelim = "~~~"\n \n-# --input_files="$intab.input_files~~~$intab.input_CL~~~$intab.input_formats\\\n-#~~~$intab.input_label~~~$intab.input_help"\n+# --input_files="$intab.input_files~~~$intab.input_CL~~~\n+# $intab.input_formats# ~~~$intab.input_label\n+# ~~~$intab.input_help"\n IPATHPOS = 0\n ICLPOS = 1\n IFMTPOS = 2\n@@ -63,7 +52,8 @@\n IHELPOS = 4\n IOCLPOS = 5\n \n-# --output_files "$otab.history_name~~~$otab.history_format~~~$otab.history_CL~~~$otab.history_test"\n+# --output_files "$otab.history_name~~~$otab.history_format~~~\n+# $otab.history_CL~~~$otab.history_test"\n ONAMEPOS = 0\n OFMTPOS = 1\n OCLPOS = 2\n@@ -72,7 +62,8 @@\n \n \n # --additional_parameters="$i.param_name~~~$i.param_value~~~\n-# $i.param_label~~~$i.param_help~~~$i.param_type~~~$i.CL~~~i$.param_CLoverride"\n+# $i.param_label~~~$i.param_help~~~$i.param_type\n+# ~~~$i.CL~~~i$.param_CLoverride"\n ANAMEPOS = 0\n AVALPOS = 1\n ALABPOS = 2\n@@ -106,13 +97,21 @@\n return \'"%s"\' % s\n \n \n-html_escape_table = {"&": "&", ">": ">", "<": "<", "$": r"\\$","#":"#", "$":"$"}\n-cheetah_escape_table = {"$": "\\$","#":"\\#"}\n+html_escape_table = {\n+ "&": "&",\n+ ">": ">",\n+ "<": "<",\n+ "#": "#",\n+ "$": "$",\n+}\n+cheetah_escape_table = {"$": "\\\\$", "#": "\\\\#"}\n+\n \n def html_escape(text):\n """Produce entities within text."""\n return "".join([html_escape_table.get(c, c) for c in text])\n \n+\n def cheetah_escape(text):\n """Produce entities within text."""\n return "".join([cheetah_escape_table.get(c, c) for c in text])\n@@ -124,8 +123,8 @@\n t = t.replace(">", ">")\n t = t.replace("<", "<")\n t = t.replace("\\\\$", "$")\n- t = t.replace("$","$")\n- t = t.replace("#","#")\n+ t = t.replace("$", "$")\n+ t = t.replace("#", "#")\n return t\n \n \n@@ -137,7 +136,9 @@\n if citation.startswith("doi"):\n citation_tuples.append(("doi", citation[len("doi") :].strip()))\n else:\n- citation_tuples.append(("bibtex", citation[len("bibtex") :].strip()))\n+ citation_tuples.append(\n+ ("bibtex", citation[len("bibtex") :].strip())\n+ )\n return citation_tuples\n \n \n@@ -168,7 +169,9 @@\n self.executeme = self.args.sysexe\n else:\n if self.args.packages:\n- self.executeme = self.args.packages.split(",")[0].split(":")[0]\n+ self.executeme = (\n+ self.args.packages.split(",")[0].split(":")[0].strip()\n+ )\n else:\n self.executeme = None\n aCL = self.cl.append\n@@ -226,8 +229,12 @@\n else:\n aCL(self.executeme)\n aXCL(self.executeme)\n- self.elog = os.path.join(self.repdir, "%s_error_log.txt" % self.tool_name)\n- self.tlog = os.path.join(self.repdir, "%s_runner_log.txt" % self.tool_name)\n+ self.elog = os.path.join(\n+ self.repdir, "%s_error_log.txt" % self.tool_name\n+ )\n+ self.tlog = os.path.join(\n+ self.repdir, "%s_runner_log.txt" % self.tool_name\n+ )\n \n if self.args.parampass == "0":\n self.clsimple()\n@@ -235,15 +242,15 @@\n clsuffix = []\n '..b'ourcwd, shell=False, stderr=tout, stdout=tout\n+ cll,\n+ env=self.ourenv,\n+ cwd=self.ourcwd,\n+ shell=False,\n+ stderr=tout,\n+ stdout=tout,\n )\n tout.write(\n- "installed %s - got retcode %d\\n" % (self.tool_name, subp.returncode)\n+ "installed %s - got retcode %d\\n"\n+ % (self.tool_name, subp.returncode)\n )\n tout.close()\n return subp.returncode\n \n-\n-\n def writeShedyml(self):\n """for planemo"""\n yuser = self.args.user_email.split("@")[0]\n@@ -950,7 +895,11 @@\n % (tdest, self.testdir)\n )\n tf = tarfile.open(self.newtarpath, "w:gz")\n- tf.add(name=self.tooloutdir, arcname=self.tool_name, filter=exclude_function)\n+ tf.add(\n+ name=self.tooloutdir,\n+ arcname=self.tool_name,\n+ filter=exclude_function,\n+ )\n tf.close()\n shutil.copyfile(self.newtarpath, self.args.new_tool)\n \n@@ -990,7 +939,8 @@\n \n def main():\n """\n- This is a Galaxy wrapper. It expects to be called by a special purpose tool.xml\n+ This is a Galaxy wrapper.\n+ It expects to be called by a special purpose tool.xml\n \n """\n parser = argparse.ArgumentParser()\n@@ -1020,35 +970,48 @@\n a("--new_tool", default="new_tool")\n a("--galaxy_url", default="http://localhost:8080")\n a("--toolshed_url", default="http://localhost:9009")\n- # make sure this is identical to tool_sheds_conf.xml localhost != 127.0.0.1 so validation fails\n+ # make sure this is identical to tool_sheds_conf.xml\n+ # localhost != 127.0.0.1 so validation fails\n a("--toolshed_api_key", default="fakekey")\n a("--galaxy_api_key", default="fakekey")\n a("--galaxy_root", default="/galaxy-central")\n a("--galaxy_venv", default="/galaxy_venv")\n args = parser.parse_args()\n assert not args.bad_user, (\n- \'UNAUTHORISED: %s is NOT authorized to use this tool until Galaxy admin adds %s to "admin_users" in the galaxy.yml Galaxy configuration file\'\n+ \'UNAUTHORISED: %s is NOT authorized to use this tool until Galaxy \\\n+admin adds %s to "admin_users" in the galaxy.yml Galaxy configuration file\'\n % (args.bad_user, args.bad_user)\n )\n- assert args.tool_name, "## Tool Factory expects a tool name - eg --tool_name=DESeq"\n+ assert (\n+ args.tool_name\n+ ), "## Tool Factory expects a tool name - eg --tool_name=DESeq"\n assert (\n args.sysexe or args.packages\n- ), "## Tool Factory wrapper expects an interpreter or an executable package"\n- args.input_files = [x.replace(\'"\', "").replace("\'", "") for x in args.input_files]\n+ ), "## Tool Factory wrapper expects an interpreter \\\n+or an executable package in --sysexe or --packages"\n+ args.input_files = [\n+ x.replace(\'"\', "").replace("\'", "") for x in args.input_files\n+ ]\n # remove quotes we need to deal with spaces in CL params\n for i, x in enumerate(args.additional_parameters):\n- args.additional_parameters[i] = args.additional_parameters[i].replace(\'"\', "")\n+ args.additional_parameters[i] = args.additional_parameters[i].replace(\n+ \'"\', ""\n+ )\n r = ScriptRunner(args)\n r.writeShedyml()\n r.makeTool()\n if args.make_Tool == "generate":\n- retcode = r.run() # for testing toolfactory itself\n+ retcode = r.run()\n r.moveRunOutputs()\n r.makeToolTar()\n else:\n- r.planemo_biodocker_test() # test to make outputs and then test\n+ retcode = r.planemo_test(genoutputs=True) # this fails :( - see PR\n r.moveRunOutputs()\n r.makeToolTar()\n+ retcode = r.planemo_test(genoutputs=False)\n+ r.moveRunOutputs()\n+ r.makeToolTar()\n+ print(f"second planemo_test returned {retcode}")\n if args.make_Tool == "gentestinstall":\n r.shedLoad()\n r.eph_galaxy_load()\n' |
| b |
| diff -r e43c43396a70 -r 8ea1133b9d9a toolfactory/rgToolFactory2.xml --- a/toolfactory/rgToolFactory2.xml Fri Dec 11 04:23:48 2020 +0000 +++ b/toolfactory/rgToolFactory2.xml Tue Jan 05 00:34:48 2021 +0000 |
| [ |
| @@ -1,4 +1,4 @@ -<tool id="rgtfd" name="toolfactory" version="2.00" profile="16.04" > +<tool id="rgtf2" name="toolfactory" version="2.00" profile="16.04" > <description>Scripts into tools v2.0</description> <macros> <xml name="tool_metadata"> @@ -73,8 +73,9 @@ <param name="history_name" type="text" label="Name for this output to appear in new history" optional="false" help="No spaces! Argparse will also use this name as --[name]"> <sanitizer invalid_char="_"> - <valid initial="string.letters,string.digits"/> - <add value="_" /> + <valid initial="string.ascii_letters,string.digits"> + <add value="_" /> + </valid> </sanitizer> </param> <param name="history_format" type="select" multiple="false" label="Select the datatype for this output" @@ -134,7 +135,7 @@ </param> <param name="param_CL" type="text" label="Positional ordinal | argparse argument name" help="Using positional parameters, enter the integer ordinal for this parameter on the command line. Using Argparse style, '--' will be prepended on the CL" value="" /> - <param name="param_CLprefixed" type="text" label="Override the generated default argparse name prefix if not empty - eg ----foo if needed" + <param name="param_CLprefixed" type="text" label="Override the generated default argparse name prefix if not empty - eg ~~--foo if needed" help="Some targets like Planemo expect an unadorned action like 'test' before --galaxy_root." value="" /> </repeat> </section> @@ -145,8 +146,7 @@ <requirement type="package" version="0.4.11">galaxyxml</requirement> <requirement type="package" version="0.14.0">bioblend</requirement> <requirement type="package" version="0.10.6">ephemeris</requirement> - <requirement type="package" version="4.4.0">docker-py</requirement> - <requirement type="package" version="0.72.0">planemo</requirement> + <requirement type="package" version="0.74.1">planemo</requirement> </requirements> <command ><
