comparison tools/ncbi_blast_plus/README.rst @ 13:623f727cdff1 draft

Uploaded v0.1.00, uses BLAST+ 2.2.29, allows custom column selection for tabular output - including taxonomy fields.

12:6560192c5098

Galaxy wrappers for NCBI BLAST+ suite
=====================================

These wrappers are copyright 2010-2013 by Peter Cock, The James Hutton Institute
(formerly SCRI, Scottish Crop Research Institute), UK. All rights reserved.
See the licence text below.

Currently tested with NCBI BLAST 2.2.28+ (i.e. version 2.2.28 of BLAST+),
and does not work with the NCBI 'legacy' BLAST suite (e.g. ``blastall``).

Note that these wrappers (and the associated datatypes) were originally

Galaxy should be able to automatically install the dependencies, i.e. the
``blast_datatypes`` repository which defines the BLAST XML file format
(``blastxml``) and protein and nucleotide BLAST databases (``blastdbp`` and
``blastdbn``).

You must tell Galaxy about any system level BLAST databases using configuration
files blastdb.loc (nucleotide databases like NT) and blastdb_p.loc (protein
databases like NR), and blastdb_d.loc (protein domain databases like CDD or
SMART) which are located in the tool-data/ folder. Sample files are included
which explain the tab-based format to use.

You can download the NCBI provided databases as tar-balls from here:

* ftp://ftp.ncbi.nlm.nih.gov/blast/db/ (nucleotide and protein databases like NR)
* ftp://ftp.ncbi.nih.gov/pub/mmdb/cdd/little_endian/ (domain databases like CDD)


Manual Installation
===================

For those not using Galaxy's automated installation from the Tool Shed, put

Run the functional tests (adjusting the section identifier to match your
``tool_conf.xml.sample`` file)::

    ./run_functional_tests.sh -sid NCBI_BLAST+-ncbi_blast_plus_tools


History
=======

======= ======================================================================

          'blast_datatypes' repository from the Tool Shed.
v0.0.17 - The BLAST+ search tools now default to extended tabular output
          (all too often our users where having to re-run searches just to
          get one of the missing columns like query or subject length)
v0.0.18 - Defensive quoting of filenames in case of spaces (where possible,
          BLAST+ handling of some mult-file arguments is problematic).
v0.0.19 - Added wrappers for rpsblast and rpstblastn, and new blastdb_d.loc
          for the domain databases they use (e.g. CDD, PFAM or SMART).
        - Correct case of exception regular expression (for error handling
          fall-back in case the return code is not set properly).
        - Clearer naming of output files.

        - Dependency on new package_blast_plus_2_2_26 in Tool Shed.
        - Adopted standard MIT License.
        - Development moved to GitHub, https://github.com/peterjc/galaxy_blast
        - Updated citation information (Cock et al. 2013).
v0.0.21 - Use macros to simplify the XML wrappers.
        - Added wrapper for dustmasker
        - Enabled masking for makeblastdb
        - Requires 'maskinfo-asn1' and 'maskinfo-asn1-binary' datatypes
          defined in updated blast_datatypes on Galaxy ToolShed.
        - Tests updated for BLAST+ 2.2.27 instead of BLAST+ 2.2.26
        - Now depends on package_blast_plus_2_2_27 in ToolShed
v0.0.22 - More use macros to simplify the wrappers
        - Set number of threads via $GALAXY_SLOTS environment variable
        - More descriptive default output names
        - Tests require updated BLAST DB definitions (blast_datatypes v0.0.18)
        - Pre-check for duplicate identifiers in makeblastdb wrapper.
        - Tests updated for BLAST+ 2.2.28 instead of BLAST+ 2.2.27
        - Now depends on package_blast_plus_2_2_28 in ToolShed
        - Extended tabular output includes 'salltitles' as column 25.
======= ======================================================================


Bug Reports
===========

13:623f727cdff1

Galaxy wrappers for NCBI BLAST+ suite
=====================================

These wrappers are copyright 2010-2013 by Peter Cock (The James Hutton Institute,
UK) and additional contributors. All rights reserved. See the licence text below.

Currently tested with NCBI BLAST 2.2.28+ (i.e. version 2.2.28 of BLAST+),
and does not work with the NCBI 'legacy' BLAST suite (e.g. ``blastall``).

Note that these wrappers (and the associated datatypes) were originally

Galaxy should be able to automatically install the dependencies, i.e. the
``blast_datatypes`` repository which defines the BLAST XML file format
(``blastxml``) and protein and nucleotide BLAST databases (``blastdbp`` and
``blastdbn``).

See the configuration notes below.

Manual Installation
===================

For those not using Galaxy's automated installation from the Tool Shed, put

Run the functional tests (adjusting the section identifier to match your
``tool_conf.xml.sample`` file)::

    ./run_functional_tests.sh -sid NCBI_BLAST+-ncbi_blast_plus_tools

Configuration
=============

You must tell Galaxy about any system level BLAST databases using configuration
files blastdb.loc (nucleotide databases like NT) and blastdb_p.loc (protein
databases like NR), and blastdb_d.loc (protein domain databases like CDD or
SMART) which are located in the tool-data/ folder. Sample files are included
which explain the tab-based format to use.

You can download the NCBI provided databases as tar-balls from here:

* ftp://ftp.ncbi.nlm.nih.gov/blast/db/ (nucleotide and protein databases like NR)
* ftp://ftp.ncbi.nih.gov/pub/mmdb/cdd/little_endian/ (domain databases like CDD)

If using the optional taxonomy columns, you will also need to download the
NCBI taxonomy files (``taxdb.btd`` and ``taxdb.bti`` from ``taxdb.tar.gz`` on
the BLAST database FTP site). Currently explicit version tracking of the
taxonomy is not supported, and in order to use this you must set the
``$BLASTDB`` environment variable to include the path where you unzipped the
taxonomy files. If this is not done, the taxonomy columns like species name
will appear as ``N/A`` in the tabular output.

The BLAST+ binaries support multi-threaded operation, which is handled via the
$GALAXY_SLOTS environment variable. This should be set automatically by Galaxy
via your job runner settings, which allows you to (for example) allocate four
cores to each BLAST job.

In addition, the BLAST+ wrappers also support high level parallelism by task
splitting if ``use_tasked_jobs = True`` is enabled in your ``universe_wsgi.ini``
configuration file. Essentially, the FASTA input query files are broken up into
batches of 1000 sequences, a separate BLAST child job is run for each chunk,
and then the BLAST output files are merged (in order). This is transparent
for the end user.

History
=======

======= ======================================================================

          'blast_datatypes' repository from the Tool Shed.
v0.0.17 - The BLAST+ search tools now default to extended tabular output
          (all too often our users where having to re-run searches just to
          get one of the missing columns like query or subject length)
v0.0.18 - Defensive quoting of filenames in case of spaces (where possible,
          BLAST+ handling of some multi-file arguments is problematic).
v0.0.19 - Added wrappers for rpsblast and rpstblastn, and new blastdb_d.loc
          for the domain databases they use (e.g. CDD, PFAM or SMART).
        - Correct case of exception regular expression (for error handling
          fall-back in case the return code is not set properly).
        - Clearer naming of output files.

        - Dependency on new package_blast_plus_2_2_26 in Tool Shed.
        - Adopted standard MIT License.
        - Development moved to GitHub, https://github.com/peterjc/galaxy_blast
        - Updated citation information (Cock et al. 2013).
v0.0.21 - Use macros to simplify the XML wrappers.
        - Added wrapper for dustmasker.
        - Enabled masking for makeblastdb.
        - Requires 'maskinfo-asn1' and 'maskinfo-asn1-binary' datatypes.
          defined in updated blast_datatypes on Galaxy ToolShed.
        - Tests updated for BLAST+ 2.2.27 instead of BLAST+ 2.2.26.
        - Now depends on package_blast_plus_2_2_27 in ToolShed.
v0.0.22 - More use macros to simplify the wrappers.
        - Set number of threads via $GALAXY_SLOTS environment variable.
        - More descriptive default output names.
        - Tests require updated BLAST DB definitions (blast_datatypes v0.0.18).
        - Pre-check for duplicate identifiers in makeblastdb wrapper.
        - Tests updated for BLAST+ 2.2.28 instead of BLAST+ 2.2.27.
        - Now depends on package_blast_plus_2_2_28 in ToolShed.
        - Extended tabular output includes 'salltitles' as column 25.
v0.1.00 - Now depends on package_blast_plus_2_2_29 in ToolShed.
        - Tabular output now includes option to pick specific columns,
          including previously unavailable taxonomy columns.
        - BLAST XML to tabular tool supports multiple input files.
        - More detailed descriptions for BLASTN and BLASTP task option.
        - Wrappers for segmasker, dustmasker and convert2blastmask.
        - Supports using maskinfo with makeblastdb wrapper.
        - Supports setting a taxonomy ID in makeblastdb wrapper.
        - Subtle changes like new conditional settings will require some old
          workflows be updated to cope. 
======= ======================================================================


Bug Reports
===========