Mercurial > repos > weilong-guo > bs_seeker2
view BSseeker2/README.md @ 0:e6df770c0e58 draft
Initial upload
| author | weilong-guo |
|---|---|
| date | Fri, 12 Jul 2013 18:47:28 -0400 |
| parents | |
| children | 8b26adf64adc |
line wrap: on
line source
BS-Seeker2 ========= BS-Seeker2 (BS Seeker 2) performs accurate and fast mapping of bisulfite-treated short reads. BS-Seeker2 is an updated version on BS-Seeker. 0. Availability ============ Homepage of [BS-Seeker2](http://pellegrini.mcdb.ucla.edu/BS_Seeker2/). The source code for this package is available from [https://github.com/BSSeeker/BSseeker2](https://github.com/BSSeeker/BSseeker2). Also, you can use an instance of BS-Seeker 2 in Galaxy from [http://galaxy.hoffman2.idre.ucla.edu](http://galaxy.hoffman2.idre.ucla.edu). (Label: "NGS: Methylation Mapping"/"Methylation Map with BS Seeker2") 1. Remarkable new features ============ * Reduced index for RRBS, accelerating the mapping speed and increasing mappability * Allowing local alignment with Bowtie 2, increased the mappability 2. Other features ============ * Supported library types - whole genome-wide bisulfite sequencing (WGBS) - reduced representative bisulfite sequencing (RRBS) * Supported formats for input file - [fasta](http://en.wikipedia.org/wiki/FASTA_format) - [fastq](http://en.wikipedia.org/wiki/FASTQ_format) - [qseq](http://jumpgate.caltech.edu/wiki/QSeq) - pure sequence (one-line one-sequence) * Supported alignment tools - [bowtie](http://bowtie-bio.sourceforge.net/index.shtml) : Single-seed - [bowtie2](http://bowtie-bio.sourceforge.net/bowtie2/index.shtml) : Multiple-seed, gapped-alignment - [local alignment](http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#local-alignment-example) - [end-to-end alignment](http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#end-to-end-alignment-example) - [SOAP](http://soap.genomics.org.cn/) * Supported formats for mapping results - [BAM](http://genome.ucsc.edu/FAQ/FAQformat.html#format5.1) - [SAM](http://samtools.sourceforge.net/) - [BS-seeker 1](http://pellegrini.mcdb.ucla.edu/BS_Seeker/USAGE.html) 3. System requirements ============ * Linux or Mac OS platform * One of the following Aligner - [bowtie](http://bowtie-bio.sourceforge.net/) - [bowtie2](http://bowtie-bio.sourceforge.net/bowtie2/) (Recommend) - [soap](http://soap.genomics.org.cn/) * [Python](http://www.python.org/download/) (Version 2.6 +) (It is normally pre-installed in Linux. Type " python -V" to see the installed version.) * [pysam](http://code.google.com/p/pysam/) package is needed. (Read "Questions & Answers" if you have problem when installing this package.) 4. Modules' descriptions ============ (0) FilterReads.py ------------ Optional and independent module. Some reads would be extremely amplified during the PCR. This script helps you get unique reads before doing the mapping. You can decide whether or not to filter reads before doing the mapping. ##Usage : $ python FilterReads.py Usage: FilterReads.py -i <input> -o <output> [-k] Author : Guo, Weilong; guoweilong@gmail.com; 2012-11-10 Unique reads for qseq/fastq/fasta/sequencce, and filter low quality file in qseq file. Options: -h, --help show this help message and exit -i FILE Name of the input qseq/fastq/fasta/sequence file -o FILE Name of the output file -k Would not filter low quality reads if specified ##Tip : - This step is not suggested for RRBS library, as reads from RRBS library would more likely from the same location. (1) bs_seeker2-build.py ------------ Module to build the index for BS-Seeker2. ##Usage : $ python bs_seeker2-build.py -h Usage: bs_seeker2-build.py [options] Options: -h, --help show this help message and exit -f FILE, --file=FILE Input your reference genome file (fasta) --aligner=ALIGNER Aligner program to perform the analysis: bowtie, bowtie2, soap [Default: bowtie2] -p PATH, --path=PATH Path to the aligner program. Defaults: bowtie: /u/home/mcdb/weilong/install/bowtie-0.12.8 bowtie2: /u/home/mcdb/weilong/install/bowtie2-2.0.0-beta7 soap: /u/home/mcdb/weilong/install/soap2.21release/ -d DBPATH, --db=DBPATH Path to the reference genome library (generated in preprocessing genome) [Default: /u/home/mcdb/weilong/i nstall/BSseeker2/bs_utils/reference_genomes] -v, --version show version of BS-Seeker2 Reduced Representation Bisulfite Sequencing Options: Use this options with conjuction of -r [--rrbs] -r, --rrbs Build index specially for Reduced Representation Bisulfite Sequencing experiments. Genome other than certain fragments will be masked. [Default: False] -l LOW_BOUND, --low=LOW_BOUND lower bound of fragment length (excluding recognition sequence such as C-CGG) [Default: 40] -u UP_BOUND, --up=UP_BOUND upper bound of fragment length (excluding recognition sequence such as C-CGG ends) [Default: 500] -c CUT_FORMAT, --cut-site=CUT_FORMAT Cut sites of restriction enzyme. Ex: MspI(C-CGG), Mael:(C-TAG), double-enzyme MspI&Mael:(C-CGG,C-TAG). [Default: C-CGG] ##Example * Build genome index for WGBS using bowtie, path of bowtie should be included in $PATH python bs_seeker2-build.py -f genome.fa --aligner=bowtie * Build genome index for RRBS with default parameters specifying the path for bowtie2 python bs_seeker2-build.py -f genome.fa --aligner=bowtie2 -p ~/install/bowtie2-2.0.0-beta7/ -r * Build genome index for RRBS library using bowite2, with fragment lengths ranging [40bp, 400bp] python bs_seeker2-build.py -f genome.fa -r -l 40 -u 400 --aligner=bowtie2 * Build genome index for RRBS library for double-enzyme : MspI (C-CGG) & ApeKI (G-CWGC, where W=A|T, see [IUPAC code](http://www.bioinformatics.org/sms/iupac.html)) python bs_seeker2-build.py -f genome.fa -r -c C-CGG,G-CWGC ##Tips: - Index built for BS-Seeker2 is different from the index for BS-Seeker 1. For RRBS, you need to specify "-r" in the parameters. Also, you need to specify LOW_BOUND and UP_BOUND for the range of fragment lengths according your protocol. - The fragment length is different from read length. Fragments refers to the DNA fragments which you get by size-selection step (i.e. gel-cut oor AMPure beads). Lengths of fragments are supposed to be in a range, such as [50bp,250bp]. - The indexes for RRBS and WGBS are different. Also, indexes for RRBS are specific for fragment length parameters (LOW_BOUND and UP_BOUND). (2) bs_seeker2-align.py ------------ Module to map reads on 3-letter converted genome. ##Usage : $ python ~/install/BSseeker2/bs_seeker2-align.py -h Usage: bs_seeker2-align.py [options] Options: -h, --help show this help message and exit For single end reads: -i INFILE, --input=INFILE Input your read file name (FORMAT: sequences, fastq, qseq,fasta) For pair end reads: -1 FILE, --input_1=FILE Input your read file end 1 (FORMAT: sequences, qseq, fasta, fastq) -2 FILE, --input_2=FILE Input your read file end 2 (FORMAT: sequences, qseq, fasta, fastq) --minins=MIN_INSERT_SIZE The minimum insert size for valid paired-end alignments [Default: -1] --maxins=MAX_INSERT_SIZE The maximum insert size for valid paired-end alignments [Default: 400] Reduced Representation Bisulfite Sequencing Options: -r, --rrbs Process reads from Reduced Representation Bisulfite Sequencing experiments -c pattern, --cut-site=pattern Cutting sites of restriction enzyme. Ex: MspI(C-CGG), Mael:(C-TAG), double-enzyme MspI&Mael:(C-CGG,C-TAG). -L RRBS_LOW_BOUND, --low=RRBS_LOW_BOUND lower bound of fragment length (excluding C-CGG ends) [Default: 40] -U RRBS_UP_BOUND, --up=RRBS_UP_BOUND upper bound of fragment length (excluding C-CGG ends) [Default: 500] General options: -t TAG, --tag=TAG [Y]es for undirectional lib, [N]o for directional [Default: N] -s CUTNUMBER1, --start_base=CUTNUMBER1 The first base of your read to be mapped [Default: 1] -e CUTNUMBER2, --end_base=CUTNUMBER2 The last cycle number of your read to be mapped [Default: 200] -a FILE, --adapter=FILE Input text file of your adaptor sequences (to be trimed from the 3'end of the reads). Input 1 seq for dir. lib., 2 seqs for undir. lib. One line per sequence --am=ADAPTER_MISMATCH Number of mismatches allowed in adaptor [Default: 1] -g GENOME, --genome=GENOME Name of the reference genome (the same as the reference genome file in the preprocessing step) [ex. chr21_hg18.fa] -m INT_NO_MISMATCHES, --mismatches=INT_NO_MISMATCHES Number of mismatches in one read [Default: 4] --aligner=ALIGNER Aligner program to perform the analisys: bowtie, bowtie2, soap [Default: bowtie2] -p PATH, --path=PATH Path to the aligner program. Defaults: bowtie: /u/home/mcdb/weilong/install/bowtie-0.12.8 bowtie2: /u/home/mcdb/weilong/install/bowtie2-2.0.0-beta7 soap: /u/home/mcdb/weilong/soap2.21release/ -d DBPATH, --db=DBPATH Path to the reference genome library (generated in preprocessing genome) [Default: /u/home/mcdb/weilong/i nstall/BSseeker2/bs_utils/reference_genomes] -l NO_SPLIT, --split_line=NO_SPLIT Number of lines per split (the read file will be split into small files for mapping. The result will be merged. [Default: 4000000] -o OUTFILE, --output=OUTFILE The name of output file [INFILE.bs(se|pe|rrbs)] -f FORMAT, --output-format=FORMAT Output format: bam, sam, bs_seeker1 [Default: bam] --no-header Suppress SAM header lines [Default: False] --temp_dir=PATH The path to your temporary directory [Default: /tmp] --XS=XS_FILTER Filter definition for tag XS, format X,Y. X=0.8 and y=5 indicate that for one read, if #(mCH sites)/#(all CH sites)>0.8 and #(mCH sites)>5, then tag XS=1; or else tag XS=0. [Default: 0.5,5] --multiple-hit Output reads with multiple hits to file"Multiple_hit.fa" -v, --version show version of BS-Seeker2 Aligner Options: You may specify any additional options for the aligner. You just have to prefix them with --bt- for bowtie, --bt2- for bowtie2, --soap- for soap, and BS Seeker will pass them on. For example: --bt-p 4 will increase the number of threads for bowtie to 4, --bt--tryhard will instruct bowtie to try as hard as possible to find valid alignments when they exist, and so on. Be sure that you know what you are doing when using these options! Also, we don't do any validation on the values. ##Examples : * Align from fasta format with bowtie2 (local alignment) for whole genome, allowing 3 mismatches python bs_seeker2-align.py -i WGBS.fa -m 3 --aligner=bowtie2 -o WGBS.bam -f bam -g genome.fa * Align from qseq format for RRBS with bowtie, default parameters for RRBS fragments python bs_seeker2-align.py -i RRBS.fa --aligner=bowtie -o RRBS.sam -f sam -g genome.fa -r -a adapter.txt * Align from qseq format for RRBS with bowtie (end-to-end), specifying lengths of fragments ranging [40bp, 400bp] python bs_seeker2-align.py -i RRBS.qseq --aligner=bowtie2 --bt2--end-to-end -o RRBS.bam -f bam -g genome.fa -r --low=40 --up=400 -a adapter.txt The parameters '--low' and '--up' should be the same with corresponding parameters when building the genome index ##Input file: - Adapter.txt (example) : AGATCGGAAGAGCACACGTC ##Output files: - SAM file Sample: 10918 0 chr1 133859922 255 100M * 0 0 TGGTTGTTTTTGTTATAGTTTTTTGTTGTAGAGTTTTTTTTGGAAAGTTGTGTTTATTTTTTTTTTTGTTTGGGTTTTGTTTGAAAGGGGTGGATGAGTT * XO:Z:+FW XS:i:0 NM:i:3 XM:Z:x--yx-zzzy--y--y--zz-zyx-yx-y--------z------------x--------z--zzz----y----y--x-zyx--------y--------z XG:Z:-C_CGGCCGCCCCTGCTGCAGCCTCCCGCCGCAGAGTTTTCTTTGGAAAGTTGCGTTTATTTCTTCCCTTGTCTGGGCTGCGCCCGAAAGGGGCAGATGAGTC_AC Format descriptions: BS-Seeker2 specific tags: XO : orientation, from forward/reverted XS : 1 when read is recognized as not fully converted by bisulfite treatment, or else 0 XM : number of sites for mismatch X: methylated CG x: un-methylated CG Y: methylated CHG y: un-methylated CHG Z: methylated CHH z: un-methylated CHH XG : genome sequences, with 2bp extended on both ends, from 5' to 3' YR : tag only for RRBS, serial id of mapped fragment YS : tag only for RRBS, start position of mapped fragment YE : tag only for RRBS, end position of mapped fragment Note: For reads mapped on Watson(minus) strand, the 10th colum in SAM file is not the original reads but the revered sequences. ##Tips: - Removing adapter is recommended. If you don't know what's your parameter, please ask the person who generate the library for you. If you are too shy to ask for it, you can try to de novo motif finding tools (such as [DME](http://cb1.utdallas.edu/dme/index.htm) and [MEME](http://meme.nbcr.net/meme/cgi-bin/meme.cgi)) find the enriched pattern in 1000 reads. Of course, you can also use other tools (such as [cutadapt](http://code.google.com/p/cutadapt/) ) to remove adaptor first. - It's always better to use a wider range for fragment length. For example, if 95% of reads come from fragments with length range [50bp, 250bp], you'd better choose [40bp, 300bp]. (3) bs_seeker2-call_methylation.py ------------ This module calls methylation levels from the mapping result. ##Usage: $ python bs_seeker2-call_methylation.py -h Usage: bs_seeker2-call_methylation.py [options] Options: -h, --help show this help message and exit -i INFILE, --input=INFILE BAM output from bs_seeker2-align.py -d DBPATH, --db=DBPATH Path to the reference genome library (generated in preprocessing genome) [Default: /u/home/mcdb/weilong/i nstall/BSseeker2/bs_utils/reference_genomes] -o OUTFILE, --output-prefix=OUTFILE The output prefix to create ATCGmap and wiggle files [INFILE] --wig=OUTFILE The output .wig file [INFILE.wig] --CGmap=OUTFILE The output .CGmap file [INFILE.CGmap] --ATCGmap=OUTFILE The output .ATCGmap file [INFILE.ATCGmap] -x, --rm-SX Removed reads with tag 'XS:i:1', which would be considered as not fully converted by bisulfite treatment [Default: False] -r READ_NO, --read-no=READ_NO The least number of reads covering one site to be shown in wig file [Default: 1] -v, --version show version of BS-Seeker2 ##Example : -For WGBS (whole genome bisulfite sequencing): python bs_seeker2-call_methylation.py -i WGBS.bam -o output --db /path/to/BSseeker2/bs_utils/reference_genomes/genome.fa_bowtie/ -For RRBS: python bs_seeker2-call_methylation.py -i RRBS.bam -o output --db /path/to/BSseeker2/bs_utils/reference_genomes/genome.fa_rrbs_40_400_bowtie2/ -For RRBS and removed un-converted reads (with tag XS=1): python bs_seeker2-call_methylation.py -x -i RRBS.bam -o output --db /path/to/BSseeker2/bs_utils/reference_genomes/genome.fa_rrbs_75_280_bowtie2/ -For RRBS and only show sites covered by at least 10 reads in WIG file: python bs_seeker2-call_methylation.py -r 10 -i RRBS.bam -o output --db /path/to/BSseeker2/bs_utils/reference_genomes/genome.fa_rrbs_75_280_bowtie2/ The folder “genome.fa\_rrbs\_40\_500\_bowtie2” is built in the first step ##Output files: - wig file Sample: variableStep chrom=chr1 3000419 0.000000 3000423 -0.2 3000440 0.000000 3000588 0.5 3000593 -0.000000 Format descriptions: WIG file format. Negative value for 2nd column indicate a Cytosine on minus strand. - CGmap file Sample: chr1 G 3000851 CHH CC 0.1 1 10 chr1 C 3001624 CHG CA 0.0 0 9 chr1 C 3001631 CG CG 1.0 5 5 Format descriptions: (1) chromosome (2) nucleotide on Watson (+) strand (3) position (4) context (CG/CHG/CHH) (5) dinucleotide-context (CA/CC/CG/CT) (6) methyltion-level = #-of-C / (#-of-C + #-of-T) (7) #-of-C (methylated) (8) (#-ofC + #-of-T) (all cytosines) - ATCGmap file Sample: chr1 T 3009410 -- -- 0 10 0 0 0 0 0 0 0 0 na chr1 C 3009411 CHH CC 0 10 0 0 0 0 0 0 0 0 0.0 chr1 C 3009412 CHG CC 0 10 0 0 0 0 0 0 0 0 0.0 chr1 C 3009413 CG CG 0 10 50 0 0 0 0 0 0 0 0.833333333333 Format descriptions: (1) chromosome (2) nucleotide on Watson (+) strand (3) position (4) context (CG/CHG/CHH) (5) dinucleotide-context (CA/CC/CG/CT) (6) - (10) plus strand (6) # of reads from Watson strand mapped here, support A on Watson strand (7) # of reads from Watson strand mapped here, support T on Watson strand (8) # of reads from Watson strand mapped here, support C on Watson strand (9) # of reads from Watson strand mapped here, support G on Watson strand (10) # of reads from Watson strand mapped here, support N (11) - (15) minus strand (11) # of reads from Crick strand mapped here, support A on Watson strand and T on Crick strand (12) # of reads from Crick strand mapped here, support T on Watson strand and A on Crick strand (13) # of reads from Crick strand mapped here, support C on Watson strand and G on Crick strand (14) # of reads from Crick strand mapped here, support G on Watson strand and C on Crick strand (15) # of reads from Crick strand mapped here, support N (16) methylation_level = #C/(#C+#T) = (C8+C14)/(C7+C8+C11+C14); "nan" means none reads support C/T at this position. Contact Information ============ If you still have questions on BS-Seeker 2, or you find bugs when using BS-Seeker 2, or you have suggestions, please write email to guoweilong@gmail.com (Weilong Guo). Questions & Answers ============ (1) Speed-up your alignment Q: "It takes me days to do the alignment for one lane" ... A: Yes, alignment is a time-consuming work, especially because the sequencing depth is increasing. An efficient way to align is : i. cut the original sequence file into multiple small pieces; Ex: split -l 4000000 input.fq ii. align them in parallel; iii. merge all the BAM files into a single one before running "bs-seeker2_call-methylation.py" (user "samtools merge" command). Ex: samtools merge out.bam in1.bam in2.bam in3.bam (2) read in BAM/SAM Q: Is the read sequence in BAM/SAM file is the same as my original one? A: NO. They are different for several reasons. i. For RRBS, some reads are short because of trimming of the adapters ii. For read mapping on Crick (-) strand, the reads are in fact the antisense version of the original sequence, opposite both in nucleotides and direction (3) "Pysam" package related problem Q: I'm normal account user for Linux(Cluster). I can't install "pysam". I get following error massages: $ python setup.py install running install error: can't create or remove files in install directory The following error occurred while trying to add or remove files in the installation directory: [Errno 13] Permission denied: '/usr/lib64/python2.6/site-packages/test-easy-install-26802.write-test' ... A: You can ask the administrator of your cluster to install pysam. If you don't want to bother him/her, you might need to build your own python, and then install the "pysam" package. The following script could be helpful for you. mkdir ~/install cd ~/install/ # install python wget http://www.python.org/ftp/python/2.7.4/Python-2.7.4.tgz # download the python from websites tar zxvf Python-2.7.4.tgz # decompress cd Python-2.7.4 ./configure --prefix=`pwd` make make install # Add the path of Python to $PATH # Please add the following line to file ~/.bashrc export PATH=~/install/Python-2.7.4:$PATH # save the ~/.bashrc file source ~/.bashrc # install pysam package wget https://pysam.googlecode.com/files/pysam-0.7.4.tar.gz tar zxvf pysam-0.7.4.tar.gz cd pysam-0.7.4 python setup.py build python setup.py install # re-login the shell after finish installing pysam # install BS-Seeker2 wget https://github.com/BSSeeker/BSseeker2/archive/master.zip mv master BSSeeker2.zip unzip BSSeeker2.zip cd BSseeker2-master/ （4）Run BS-Seeker2 Q: Can I add the path of BS-Seeker2's *.py to the $PATH, so I can call BS-Seeker2 from anywhere? A: If you're using the "python" from path "/usr/bin/python", you can directly add the path of BS-Seeker2 in file "~/.bash_profile" (bash) or "~/.profile" (other shell) or "~/.bashrc" (per-interactive-shell startup). But if you are using python under other directories, you might need to modify BS-Seeker2's script first. For example, if your python path is "/my_python/python", please change the first line in "bs_seeker-build.py", "bs_seeker-align.py" and "bs_seeker-call_methylation.py" to #!/my_python/python Then add export PATH=/path/to/BS-Seeker2/:$PATH to file "~/.bash_profile" (e.g.), and source the file: source ~/.bash_profile Then you can use BS-Seeker2 globally by typing: bs_seeker_build.py -h bs_seeker-align.py -h bs_seeker-call_methylation.py -h