annotate README.md @ 10:d103c41b6931 draft

Uploaded
author dereeper
date Thu, 30 May 2024 16:35:22 +0000
parents e42d30da7a74
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
1 # PanExplorer_workflow
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
2
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
3 # About
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
4
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
5 This workflow is a snakemake worklow that can be run in the backend of the PanExplorer web application.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
6
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
7 **Homepage:** [https://panexplorer.southgreen.fr/](https://panexplorer.southgreen.fr/)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
8
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
9 It allows to perform a pan-genome analysis using published and annotated bacteria genomes, using different tools that can be invoked: Roary, PGAP, PanACoTA.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
10
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
11 It provides a presence/absence matrix of genes, an UpsetR Diagram for synthetizing the matrix information and a COG assignation summary for each strain.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
12
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
13
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
14 ## Citation
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
15
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
16 [https://doi.org/10.1093/bioinformatics/btac504](https://doi.org/10.1093/bioinformatics/btac504)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
17
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
18 ## Authors
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
19
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
20 * Alexis Dereeper (IRD)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
21
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
22 ## Prerequisites - Tool dependencies
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
23
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
24 Using a singularity container, the only dependency you will need is **singularity**.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
25
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
26 This singularity image (panexplorer.sif) already contains all dependencies required for running the workflow:
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
27
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
28 - Snakemake
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
29 - Roary
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
30 - PGAP
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
31 - Panaroo
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
32 - Panacota
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
33 - Minigraph/cactus
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
34 - PanGenome Graph Builder (PGGB)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
35 - ncbi-blast+ (version BLAST 2.4.0+)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
36 - R (version 4.2.0) and following packages:
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
37 - optparse : ``install.packages("optparse")``
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
38 - dendextend : ``install.packages("dendextend")``
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
39 - svglite : ``install.packages("svglite")``
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
40 - heatmaply : ``install.packages("heatmaply")``
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
41 - gplots : ``install.packages("gplots")``
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
42 - UpSetR : ``install.packages("UpSetR")``
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
43
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
44 ## Install
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
45
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
46 1- Git clone
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
47
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
48 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
49 git clone https://github.com/SouthGreenPlatform/PanExplorer_workflow.git
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
50 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
51
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
52 2- Define the PANEX_PATH environnement variable
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
53
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
54 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
55 cd PanExplorer_workflow
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
56 export PANEX_PATH=$PWD
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
57 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
58
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
59 3- Get preformatted RPS-BLAST+ database of the CDD COG distribution
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
60
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
61 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
62 wget https://ftp.ncbi.nlm.nih.gov/pub/mmdb/cdd/little_endian/Cog_LE.tar.gz
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
63 tar -xzvf Cog_LE.tar.gz -C $PANEX_PATH/COG
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
64 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
65
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
66 4- Get the singularity container
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
67
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
68 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
69 wget -P $PANEX_PATH/singularity https://panexplorer.southgreen.fr/singularity/panexplorer.sif
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
70 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
71
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
72
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
73 ## Prepare your list of genomes to be analyzed
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
74
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
75 Edit the configuration file config.yaml to list the Genbank identifiers of complete assembled and annotated genomes.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
76 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
77 #########################################################
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
78 # Complete one of the following input data
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
79 # Remove the other lines if not needed
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
80 #########################################################
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
81
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
82 # Genbank accessions of assembly accession (GCA, GCF)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
83 ids:
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
84 - GCA_001042775.1
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
85 - GCA_001021915.1
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
86 - GCA_022406815.1
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
87
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
88 # Path of genbank files
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
89 input_genbanks:
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
90 - data/GCA_001518895.1.gb
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
91 - data/GCA_001746615.1.gb
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
92 - data/GCA_003382895.1.gb
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
93
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
94 # Input genomes as fasta and annotation files in GFF format
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
95 # Only applied when using Orthofinder or PGGB workflows, starting from fasta and GFF
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
96 # To be used preferentially for eukaryotes
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
97 input_genomes:
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
98 "MSU7":
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
99 "fasta": "/share/banks/Oryza/sativa/japonica/MSU7/all.con"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
100 "gff3": "/share/banks/Oryza/sativa/japonica/MSU7/all.gff3"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
101 "name": "MSU7"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
102 "kitaake":
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
103 "fasta": "/share/banks/Oryza/sativa/japonica/kitaake/Oryza_sativa_japonica_Kitaake.assembly.fna"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
104 "gff3": "/share/banks/Oryza/sativa/japonica/kitaake/Oryza_sativa_japonica_Kitaake.gff3"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
105 "name": "kitaake"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
106 "nivara":
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
107 "fasta": "/share/banks/Oryza/nivara/Oryza_nivara.assembly.fna"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
108 "gff3": "/share/banks/Oryza/nivara/Oryza_nivara.gff3"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
109 "name": "nivara"
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
110 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
111
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
112 It's best not to mix NCBI genomes with your own annotated genomes, to avoid biaises due to annotation method/software. Keep an homogeneous annotation procedure to feed the workflow.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
113
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
114 ## Run the workflow
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
115
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
116 **For prokaryotes**
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
117
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
118 Creating a pangenome using Roary
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
119
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
120 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
121 singularity exec $PANEX_PATH/singularity/panexplorer.sif snakemake --cores 1 -s $PANEX_PATH/Snakemake_files/Snakefile_wget_roary_heatmap_upset_COG
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
122 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
123
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
124 Creating a pangenome using PanACoTA
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
125
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
126 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
127 singularity exec $PANEX_PATH/singularity/panexplorer.sif snakemake --cores 1 -s $PANEX_PATH/Snakemake_files/Snakefile_wget_panacota_heatmap_upset_COG
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
128 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
129
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
130 Creating a pangenome graph using Minigraph/Cactus and derived pangenes matrix
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
131
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
132 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
133 singularity exec $PANEX_PATH/singularity/panexplorer.sif snakemake --cores 1 -s $PANEX_PATH/Snakemake_files/Snakefile_wget_cactus_heatmap_upset_COG
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
134 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
135
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
136 Creating a pangenome graph using PanGenomeGraph Builder (PGGB) and derived pangenes matrix
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
137
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
138 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
139 singularity exec $PANEX_PATH/singularity/panexplorer.sif snakemake --cores 1 -s $PANEX_PATH/Snakemake_files/Snakefile_wget_pggb_heatmap_upset_COG
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
140 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
141
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
142 **For eukaryotes**
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
143
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
144 Creating a pangenome using Orthofinder
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
145
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
146 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
147 singularity exec $PANEX_PATH/singularity/panexplorer.sif snakemake --cores 1 -s $PANEX_PATH/Snakemake_files/Snakefile_orthofinder_heatmap_upset
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
148 ```
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
149
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
150 ## Graphical outputs
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
151
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
152 In all cases, you should a new directory named "outputs" containing all output files.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
153
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
154 In case of a pangenome graph analysis with PGGB, you will obtain vizualizations of the graph (using ODGI)
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
155
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
156 * **2D graph visualization** : outputs/pggb_out/all_genomes.fa.lay.draw.png
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
157
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
158 <img src="images/all_genomes.fa.lay.draw.png" align="center" width="40%" style="display: block; margin: auto;"/>
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
159
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
160 * **1D graph visualization** : outputs/pggb_out/all_genomes.fa.og.viz_multiqc.png
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
161
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
162 <img src="images/all_genomes.fa.og.viz_multiqc.png" align="center" width="90%" style="display: block; margin: auto;"/>
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
163
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
164
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
165 In all cases, it also includes:
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
166
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
167 * **ANI (Average Nucleotide Identity)** : outputs/fastani.out.svg
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
168
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
169 The heatmap chart generated from distances calculated based on the ANI values.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
170 ANI values are calcultaed using FastANI software.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
171
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
172 <img src="images/fastani.out.svg" align="center" width="90%" style="display: block; margin: auto;"/>
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
173
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
174 * **Presence/absence matrix of accessory genes**: outputs/heatmap.svg.complete.new.svg
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
175
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
176 Both gene clusters and samples have been ordered using a Hierarchical Clustering.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
177
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
178 <img src="images/heatmap.svg.complete.new.svg" align="center" width="90%" style="display: block; margin: auto;"/>
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
179
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
180 * **Upset plot**: outputs/upsetr.svg
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
181
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
182 An Upset plot is an alternative to the Venn Diagram used to deal with more than 3 sets.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
183 The total size of each set is represented on the left barplot.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
184 Every possible intersection is represented by the bottom plot, and their occurence is shown on the top barplot.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
185 Each row corresponds to a possible intersection: the filled-in cells show which set is part of an intersection.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
186
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
187 <img src="images/upsetr.svg" align="center" width="90%" style="display: block; margin: auto;"/>
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
188
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
189 * **Rarefaction curve**: outputs/rarefaction_curves.svg
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
190
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
191 The rarefaction curve (computed by micropan R package) is the cumulative number of gene clusters we can observe as more and more genomes are being considered.
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
192
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
193 <img src="images/rarefaction_curves.svg" align="center" width="70%" style="display: block; margin: auto;"/>
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
194
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
195
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
196 ## License
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
197
e42d30da7a74 Uploaded
dereeper
parents:
diff changeset
198 GNU General Public GPLv3 License