Mercurial > repos > peterjc > blast_top_hit_species
changeset 0:68d65aeb3567 draft
Uploaded v0.0.1
| author | peterjc |
|---|---|
| date | Mon, 30 Mar 2015 11:25:10 -0400 |
| parents | |
| children | 165f0b05fa25 |
| files | N_abberans_piechart_mouseover.png README.rst blast_top_hit_species.ga blast_top_hit_species.png repository_dependencies.xml |
| diffstat | 5 files changed, 551 insertions(+), 0 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/README.rst Mon Mar 30 11:25:10 2015 -0400 @@ -0,0 +1,211 @@ +Introduction +============ + +Galaxy is a web-based platform for biological data analysis, supporting +extension with additional tools (often wrappers for existing command line +tools) and datatypes. See http://www.galaxyproject.org/ and the public +server at http://usegalaxy.org for an example. + +The NCBI BLAST suite is a widely used set of tools for biological sequence +comparison. It is available as standalone binaries for use at the command +line, and via the NCBI website for smaller searches. For more details see +http://blast.ncbi.nlm.nih.gov/Blast.cgi + +This is an example workflow using the Galaxy wrappers for NCBI BLAST+, +see https://github.com/peterjc/galaxy_blast + + +Galaxy workflow for counting species of top BLAST hits +====================================================== + +This Galaxy workflow (file ``blast_top_hit_species.ga``) is intended for an +initial assessment of a transcriptome assembly to give a crude indication of +any major contaimination present based on the species of the top BLAST hit +of 1000 representative sequences. + +.. image:: https://raw.githubusercontent.com/peterjc/galaxy_blast/master/workflows/blast_top_hit_species/blast_top_hit_species.png + +In words, the workflow proceeds as follows: + +1. Upload/import your transcriptome assembly or any nucleotide FASTA file. +2. Samples 1000 representative sequences, selected uniformly/evenly though + the file. +3. Convert the sampled FASTA file into a three column tabular file. +4. Runs NCBI BLASTX of the sampled FASTA file against the latest NCBI ``nr`` + database (assuming this is already available setup on your local Galaxy + under the alias ``nr``), requesting tabular output including the taxonomy + fields, and at most one matching target sequence. +5. Remove any duplicate alignments (multiple HSPs for the same match). +6. Combine the filtered BLAST output with the tabular version of the 1000 + sequences to give a new tabular file with exactly 1000 lines, adding + ``None`` for sequences missing a BLAST hit. +7. Count the BLAST species names in this file. +8. Sort the counts. + +Finally we would suggest visualising the sorted tally table as a Pie Chart. + + +Sample Data +=========== + +As an example, you can upload the transcriptome assembly of the nematode +*Nacobbus abberans* from Eves van den Akker *et al.* (2015), +http://dx.doi.org/10.1093/gbe/evu171 using this URL: + +http://nematode.net/Data/nacobbus_aberrans_transcript_assembly/N.abberans_reference_no_contam.zip + +Running this workflow with a copy of the NCBI non-redundant ``nr`` database +from 16 Oct 2014 (which did **not** contain this *N. abberans* dataset) gave +the following results - note 609 out of the 1000 sequences gave no BLAST hit. + +===== ================== +Count Subject Blast Name +----- ------------------ + 609 None + 244 nematodes + 30 ascomycetes + 27 eukaryotes + 8 basidiomycetes + 6 aphids + 5 eudicots + 5 flies + ... ... +===== ================== + +As you might guess from the filename ``N.abberans_reference_no_contam.fasta``, +this transcriptome assembly has already had obvious contamination removed. + +At the time of writing, Galaxy's visualizations could not be included in +a workflow. You can generate a pie chart from the final count file using +the counts (c1) and labels (c2), like this: + +.. image:: https://raw.githubusercontent.com/peterjc/galaxy_blast/master/workflows/blast_top_hit_species/N_abberans_piechart_mouseover.png + +Note the nematode count in this image was shown as a mouse-over effect. + + +Disclaimer +========== + +Species assignment by top BLAST hit is not suitable for any in depth +analysis. It is particularly prone to false positives where contaiminants +in public datasets are mislabled. See for example Ed Yong (2015), +"There's No Plague on the NYC Subway. No Platypuses Either.": + +http://phenomena.nationalgeographic.com/2015/02/10/theres-no-plague-on-the-nyc-subway-no-platypuses-either/ + + +Known Issues +============ + +This workflow uses the Galaxy "Count" tool, version 1.0.0, as shipped with +the current stable release (Galaxy v15.03, i.e. March 2015). + +The updated "Count" tool version 1.0.1 includes a fix not to remove spaces +in the fields being counted. In the example above, while the top hits are +not affected, minor entries like "cellular slime molds" are shown as +"cellularslimemolds" instead (look closely at the Pie Chart key).. + +The updated "Count" tool version 1.0.1 also adds a new option to sort the +output, which avoids the additional sorting step in the current version of +the workflow. + +A future update to this workflow will use the revised "Count" tool, once +this is included in the next stable Galaxy release - or migrated to the +Galaxy Tool Shed. + + +Availability +============ + +This workflow is available to download and/or install from the main Galaxy Tool Shed: + +http://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species + +Test releases (which should not normally be used) are on the Test Tool Shed: + +http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species + +Development is being done on github here: + +https://github.com/peterjc/galaxy_blast/tree/master/workflows/blast_top_hit_species + + +Citation +======== + +Please cite the following paper (currently available as a preprint): + +NCBI BLAST+ integrated into Galaxy. +P.J.A. Cock, J.M. Chilton, B. Gruening, J.E. Johnson, N. Soranzo +bioRxiv DOI: http://dx.doi.org/10.1101/014043 (preprint) + +You should also cite Galaxy, and the NCBI BLAST+ tools: + +BLAST+: architecture and applications. +C. Camacho et al. BMC Bioinformatics 2009, 10:421. +DOI: http://dx.doi.org/10.1186/1471-2105-10-421 + + +Automated Installation +====================== + +Installation via the Galaxy Tool Shed should take care of the dependencies +on Galaxy tools including the NCBI BLAST+ wrappers and associated binaries. + +However, this workflow requires a current version of the NCBI nr protein +BLAST database to be listed in ``blastdb_p.loc`` with the key ``nr`` (lower +case). + + +History +======= + +======= ====================================================================== +Version Changes +------- ---------------------------------------------------------------------- +v0.1.0 - Initial Tool Shed release, targetting NCBI BLAST+ 2.2.29 +======= ====================================================================== + + +Developers +========== + +This workflow is under source code control here: + +https://github.com/peterjc/galaxy_blast/tree/master/workflows/blast_top_hit_species + +To prepare the tar-ball for uploading to the Tool Shed, I use this: + + $ tar -cf blast_top_hit_species.tar.gz README.rst repository_dependencies.xml blast_top_hit_species.ga blast_top_hit_species.png N_abberans_piechart_mouseover.png + +Check this, + + $ tar -tzf blast_top_hit_species.tar.gz + README.rst + repository_dependencies.xml + blast_top_hit_species.ga + blast_top_hit_species.png + N_abberans_piechart_mouseover.png + + +Licence (MIT) +============= + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/blast_top_hit_species.ga Mon Mar 30 11:25:10 2015 -0400 @@ -0,0 +1,331 @@ +{ + "a_galaxy_workflow": "true", + "annotation": "", + "format-version": "0.1", + "name": "Species of top BLAST hits", + "steps": { + "0": { + "annotation": "", + "id": 0, + "input_connections": {}, + "inputs": [ + { + "description": "", + "name": "Transcriptome FASTA file" + } + ], + "label": null, + "name": "Input dataset", + "outputs": [], + "position": { + "left": 242, + "top": 119 + }, + "tool_errors": null, + "tool_id": null, + "tool_state": "{\"name\": \"Transcriptome FASTA file\"}", + "tool_version": null, + "type": "data_input", + "user_outputs": [], + "uuid": "e445b44b-02a7-4fd1-8944-cd680f967062" + }, + "1": { + "annotation": "This workflow is deliberately a simple/crude assessment, and there is no need to run BLASTX on all the sequences - a sample of 1000 should be enough.", + "id": 1, + "input_connections": { + "input_file": { + "id": 0, + "output_name": "output" + } + }, + "inputs": [], + "label": null, + "name": "Sub-sample sequences files", + "outputs": [ + { + "name": "output_file", + "type": "input" + } + ], + "position": { + "left": 435, + "top": 119 + }, + "post_job_actions": { + "RenameDatasetActionoutput_file": { + "action_arguments": { + "newname": "1000 sequences from #{input_file}" + }, + "action_type": "RenameDatasetAction", + "output_name": "output_file" + } + }, + "tool_errors": null, + "tool_id": "toolshed.g2.bx.psu.edu/repos/peterjc/sample_seqs/sample_seqs/0.2.1", + "tool_state": "{\"__page__\": 0, \"input_file\": \"null\", \"__rerun_remap_job_id__\": null, \"sampling\": \"{\\\"count\\\": \\\"1000\\\", \\\"type\\\": \\\"desired_count\\\", \\\"__current_case__\\\": 2}\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\", \"interleaved\": \"\\\"False\\\"\"}", + "tool_version": "0.2.1", + "type": "tool", + "user_outputs": [], + "uuid": "87ce69ef-5fb0-41b0-9575-d3b96544f8be" + }, + "2": { + "annotation": "We only want one line per query, so limit this to the best scoring target sequence. Assumes current NCBI nr database is available locally as \"nr\".", + "id": 2, + "input_connections": { + "query": { + "id": 1, + "output_name": "output_file" + } + }, + "inputs": [], + "label": null, + "name": "NCBI BLAST+ blastx", + "outputs": [ + { + "name": "output1", + "type": "tabular" + } + ], + "position": { + "left": 489, + "top": 263 + }, + "post_job_actions": { + "RenameDatasetActionoutput1": { + "action_arguments": { + "newname": "Top BLAST match" + }, + "action_type": "RenameDatasetAction", + "output_name": "output1" + } + }, + "tool_errors": null, + "tool_id": "toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastx_wrapper/0.1.01", + "tool_state": "{\"evalue_cutoff\": \"\\\"0.001\\\"\", \"__page__\": 0, \"adv_opts\": \"{\\\"adv_optional_id_files_opts\\\": {\\\"adv_optional_id_files_opts_selector\\\": \\\"none\\\", \\\"__current_case__\\\": 0}, \\\"matrix\\\": \\\"BLOSUM62\\\", \\\"adv_opts_selector\\\": \\\"advanced\\\", \\\"ungapped\\\": \\\"False\\\", \\\"filter_query\\\": \\\"True\\\", \\\"word_size\\\": \\\"0\\\", \\\"__current_case__\\\": 1, \\\"parse_deflines\\\": \\\"False\\\", \\\"strand\\\": \\\"-strand both\\\", \\\"max_hits\\\": \\\"1\\\"}\", \"__rerun_remap_job_id__\": null, \"db_opts\": \"{\\\"db_opts_selector\\\": \\\"db\\\", \\\"subject\\\": \\\"\\\", \\\"histdb\\\": \\\"\\\", \\\"__current_case__\\\": 0, \\\"database\\\": \\\"nr\\\"}\", \"query_gencode\": \"\\\"1\\\"\", \"query\": \"null\", \"output\": \"{\\\"out_format\\\": \\\"cols\\\", \\\"std_cols\\\": [\\\"qseqid\\\", \\\"sseqid\\\", \\\"pident\\\", \\\"length\\\", \\\"mismatch\\\", \\\"gapopen\\\", \\\"qstart\\\", \\\"qend\\\", \\\"sstart\\\", \\\"send\\\", \\\"evalue\\\", \\\"bitscore\\\"], \\\"ids_cols\\\": null, \\\"tax_cols\\\": [\\\"staxids\\\", \\\"sscinames\\\", \\\"scomnames\\\", \\\"sblastnames\\\", \\\"sskingdoms\\\"], \\\"__current_case__\\\": 2, \\\"misc_cols\\\": null, \\\"ext_cols\\\": null}\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\"}", + "tool_version": "0.1.01", + "type": "tool", + "user_outputs": [], + "uuid": "1559a0b0-0b66-40f9-b777-2f062fcda4cc" + }, + "3": { + "annotation": "Having a tabular file of all 1000 sequences is used in the \"join\" step to count the sequences giving no BLAST hit.", + "id": 3, + "input_connections": { + "input": { + "id": 1, + "output_name": "output_file" + } + }, + "inputs": [], + "label": null, + "name": "FASTA-to-Tabular", + "outputs": [ + { + "name": "output", + "type": "tabular" + } + ], + "position": { + "left": 696, + "top": 139 + }, + "post_job_actions": { + "HideDatasetActionoutput": { + "action_arguments": {}, + "action_type": "HideDatasetAction", + "output_name": "output" + }, + "RenameDatasetActionoutput": { + "action_arguments": { + "newname": "1000 sequences as tabular" + }, + "action_type": "RenameDatasetAction", + "output_name": "output" + } + }, + "tool_errors": null, + "tool_id": "toolshed.g2.bx.psu.edu/repos/devteam/fasta_to_tabular/fasta2tab/1.1.0", + "tool_state": "{\"__page__\": 0, \"keep_first\": \"\\\"0\\\"\", \"descr_columns\": \"\\\"2\\\"\", \"input\": \"null\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\", \"__rerun_remap_job_id__\": null}", + "tool_version": "1.1.0", + "type": "tool", + "user_outputs": [], + "uuid": "31f11208-b2bd-4d9d-9745-dc1a6ed7ccf9" + }, + "4": { + "annotation": "Some BLAST matches will give multiple HSPs, and thus multiple lines in the tabular output. We only want one line per query.", + "id": 4, + "input_connections": { + "input": { + "id": 2, + "output_name": "output1" + } + }, + "inputs": [], + "label": null, + "name": "Unique", + "outputs": [ + { + "name": "outfile", + "type": "input" + } + ], + "position": { + "left": 665, + "top": 376 + }, + "post_job_actions": { + "HideDatasetActionoutfile": { + "action_arguments": {}, + "action_type": "HideDatasetAction", + "output_name": "outfile" + }, + "RenameDatasetActionoutfile": { + "action_arguments": { + "newname": "One HSP per BLAST hit" + }, + "action_type": "RenameDatasetAction", + "output_name": "outfile" + } + }, + "tool_errors": null, + "tool_id": "toolshed.g2.bx.psu.edu/repos/bgruening/unique/bg_uniq/0.3", + "tool_state": "{\"__page__\": 0, \"ignore_case\": \"\\\"False\\\"\", \"adv_opts\": \"{\\\"column_end\\\": {\\\"__class__\\\": \\\"UnvalidatedValue\\\", \\\"value\\\": \\\"2\\\"}, \\\"column_start\\\": {\\\"__class__\\\": \\\"UnvalidatedValue\\\", \\\"value\\\": \\\"1\\\"}, \\\"adv_opts_selector\\\": \\\"advanced\\\", \\\"__current_case__\\\": 1}\", \"__rerun_remap_job_id__\": null, \"is_numeric\": \"\\\"False\\\"\", \"input\": \"null\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\"}", + "tool_version": "0.3", + "type": "tool", + "user_outputs": [], + "uuid": "acf948e3-71dc-4f35-8357-3998bd0abdd8" + }, + "5": { + "annotation": "We don't need all the columns in this join, but the key is to assign \"None\" to the sequences with no BLAST hits.", + "id": 5, + "input_connections": { + "input1": { + "id": 3, + "output_name": "output" + }, + "input2": { + "id": 4, + "output_name": "outfile" + } + }, + "inputs": [], + "label": null, + "name": "Join two Datasets", + "outputs": [ + { + "name": "out_file1", + "type": "input" + } + ], + "position": { + "left": 827, + "top": 263 + }, + "post_job_actions": { + "HideDatasetActionout_file1": { + "action_arguments": {}, + "action_type": "HideDatasetAction", + "output_name": "out_file1" + }, + "RenameDatasetActionout_file1": { + "action_arguments": { + "newname": "Top BLAST hits or None" + }, + "action_type": "RenameDatasetAction", + "output_name": "out_file1" + } + }, + "tool_errors": null, + "tool_id": "join1", + "tool_state": "{\"input2\": \"null\", \"__page__\": 0, \"field1\": \"{\\\"__class__\\\": \\\"UnvalidatedValue\\\", \\\"value\\\": \\\"1\\\"}\", \"partial\": \"\\\"\\\"\", \"field2\": \"{\\\"__class__\\\": \\\"UnvalidatedValue\\\", \\\"value\\\": \\\"1\\\"}\", \"__rerun_remap_job_id__\": null, \"fill_empty_columns\": \"{\\\"fill_empty_columns_switch\\\": \\\"fill_empty\\\", \\\"do_fill_empty_columns\\\": {\\\"column_fill_type\\\": \\\"single_fill_value\\\", \\\"fill_value\\\": \\\"None\\\", \\\"__current_case__\\\": 0}, \\\"fill_columns_by\\\": \\\"fill_unjoined_only\\\", \\\"__current_case__\\\": 1}\", \"unmatched\": \"\\\"-u\\\"\", \"input1\": \"null\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\"}", + "tool_version": "2.0.2", + "type": "tool", + "user_outputs": [], + "uuid": "4c280b0e-b4a6-4ae4-8a81-d6e93932ef71" + }, + "6": { + "annotation": "Here we make a tally table of the BLAST species name column", + "id": 6, + "input_connections": { + "input": { + "id": 5, + "output_name": "out_file1" + } + }, + "inputs": [], + "label": null, + "name": "Count", + "outputs": [ + { + "name": "out_file1", + "type": "tabular" + } + ], + "position": { + "left": 952, + "top": 398 + }, + "post_job_actions": { + "HideDatasetActionout_file1": { + "action_arguments": {}, + "action_type": "HideDatasetAction", + "output_name": "out_file1" + }, + "RenameDatasetActionout_file1": { + "action_arguments": { + "newname": "Top BLAST hit species counts (unsorted)" + }, + "action_type": "RenameDatasetAction", + "output_name": "out_file1" + } + }, + "tool_errors": null, + "tool_id": "Count1", + "tool_state": "{\"__page__\": 0, \"column\": \"{\\\"__class__\\\": \\\"UnvalidatedValue\\\", \\\"value\\\": [\\\"19\\\"]}\", \"__rerun_remap_job_id__\": null, \"delim\": \"\\\"T\\\"\", \"input\": \"null\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\"}", + "tool_version": "1.0.0", + "type": "tool", + "user_outputs": [], + "uuid": "d3322137-1911-426d-87a7-c82b5fc16825" + }, + "7": { + "annotation": "Sorting the counts makes the results easier to interpret directly.", + "id": 7, + "input_connections": { + "input": { + "id": 6, + "output_name": "out_file1" + } + }, + "inputs": [], + "label": null, + "name": "Sort", + "outputs": [ + { + "name": "out_file1", + "type": "input" + } + ], + "position": { + "left": 1056, + "top": 506 + }, + "post_job_actions": { + "RenameDatasetActionout_file1": { + "action_arguments": { + "newname": "Top BLAST hit species counts" + }, + "action_type": "RenameDatasetAction", + "output_name": "out_file1" + } + }, + "tool_errors": null, + "tool_id": "sort1", + "tool_state": "{\"__page__\": 0, \"style\": \"\\\"num\\\"\", \"column\": \"{\\\"__class__\\\": \\\"UnvalidatedValue\\\", \\\"value\\\": \\\"1\\\"}\", \"__rerun_remap_job_id__\": null, \"column_set\": \"[]\", \"input\": \"null\", \"chromInfo\": \"\\\"/mnt/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/?.len\\\"\", \"order\": \"\\\"DESC\\\"\"}", + "tool_version": "1.0.3", + "type": "tool", + "user_outputs": [], + "uuid": "c81cc61d-52a3-44ee-b646-b23e0e004c38" + } + }, + "uuid": "9fe8754a-3a87-4f6a-89a2-141b02b4793e" +} \ No newline at end of file
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/repository_dependencies.xml Mon Mar 30 11:25:10 2015 -0400 @@ -0,0 +1,9 @@ +<?xml version="1.0"?> +<repositories description="This workflow requires the NCBI BLAST+ tools etc"> + <repository changeset_revision="2fe07f50a41e" name="ncbi_blast_plus" owner="devteam" toolshed="https://toolshed.g2.bx.psu.edu" /> + <repository changeset_revision="9d189d08f2ad" name="fasta_to_tabular" owner="devteam" toolshed="https://toolshed.g2.bx.psu.edu" /> + <repository changeset_revision="02c13ef1a669" name="sample_seqs" owner="peterjc" toolshed="https://toolshed.g2.bx.psu.edu" /> + <repository changeset_revision="7ce75adb93be" name="unique" owner="bgruening" toolshed="https://toolshed.g2.bx.psu.edu" /> + <!-- Also uses tool_id join1, Count1, and sort1 which are currently + still shipped with Galaxy itself rather than via the Tool Shed --> +</repositories>