Repository 'scanpy_normalize'
hg clone https://toolshed.g2.bx.psu.edu/repos/iuc/scanpy_normalize

Changeset 17:5dada6f76047 (2024-09-14)
Previous changeset 16:9f354c7c8c92 (2024-08-20) Next changeset 18:56c0d4acef02 (2024-09-19)
Commit message:
planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/scanpy/ commit 91121b1e72696f17478dae383badaa71e9f96dbb
modified:
README.md
macros.xml
normalize.xml
test-data/cosg.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_1.h5ad
test-data/pbmc68k_reduced.h5ad
test-data/pl.clustermap.krumsiek11.png
test-data/pl.diffmap.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.png
test-data/pl.dotplot.krumsiek11.png
test-data/pl.dpt_timeseries.dpt.diffmap.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.png
test-data/pl.draw_graph.png
test-data/pl.embedding_density.pbmc68k_reduced.png
test-data/pl.heatmap.krumsiek11.png
test-data/pl.highest_expr_genes.filter_genes_dispersion.krumsiek11-seurat.png
test-data/pl.highly_variable_genes.seurat.blobs.png
test-data/pl.matrixplot.krumsiek11.png
test-data/pl.paga.paul15_gauss_braycurtis.png
test-data/pl.paga_compare.paul15_gauss_braycurtis.png
test-data/pl.pca.pbmc68k_reduced.CD3D_CD79A_components_2d.pdf
test-data/pl.pca_loadings.pp.pca.krumsiek11.png
test-data/pl.pca_overview.pp.pca.krumsiek11.png
test-data/pl.pca_variance_ratio.pp.pca.krumsiek11.png
test-data/pl.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_marker_1.png
test-data/pl.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_marker_filtered_1.png
test-data/pl.rank_genes_groups.rank_genes_groups.krumsiek11.png
test-data/pl.rank_genes_groups_dotplot.rank_genes_groups.krumsiek11.png
test-data/pl.rank_genes_groups_heatmap.rank_genes_groups.krumsiek11.png
test-data/pl.rank_genes_groups_matrixplot.rank_genes_groups.krumsiek11.png
test-data/pl.rank_genes_groups_stacked_violin.rank_genes_groups.krumsiek11.png
test-data/pl.rank_genes_groups_violin.Ery.png
test-data/pl.rank_genes_groups_violin.Mk.png
test-data/pl.rank_genes_groups_violin.Mo.png
test-data/pl.rank_genes_groups_violin.Neu.png
test-data/pl.rank_genes_groups_violin.progenitor.png
test-data/pl.scatter.krumsiek11.png
test-data/pl.scatter.pbmc68k_reduced.png
test-data/pl.scatter.umap.pbmc68k_reduced.png
test-data/pl.stacked_violin.krumsiek11.png
test-data/pl.tsne.krumsiek11.png
test-data/pl.umap.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.png
test-data/pl.violin.pbmc68k_reduced_custom.png
test-data/pp.recipe_zheng17.random-randint.h5ad
test-data/random-randint.h5ad
test-data/sparce_csr_matrix.h5ad
test-data/tl.embedding_density.umap.pbmc68k_reduced.h5ad
test-data/tl.rank_genes_groups.newton-cg.pbmc68k_reduced.h5ad
added:
test-data/external.pp.bbknn.krumsiek11.h5ad
test-data/pl.rank_genes_groups_tracksplot.newton-cg.pbmc68k_highly_reduced_marker_filtered_1.png
test-data/pl.scrublet_score_distribution.png
test-data/pl.stacked_violin_pp.filter_genes_dispersion.krumsiek11-seurat.png
test-data/pp.pca.krumsiek11.batch.h5ad
test-data/pp.scrublet.krumsiek11.h5ad
removed:
test-data/external.pp.magic.all_genes.krumsiek11.h5ad
test-data/external.pp.magic.pca_only.krumsiek11.h5ad
test-data/pp.calculate_qc_metrics.sparce_csr_matrix.h5ad
test-data/pp.downsample_counts.random-randint.h5ad
test-data/pp.filter_cells.krumsiek11-max_genes.h5ad
test-data/pp.filter_cells.krumsiek11-min_counts.h5ad
test-data/pp.filter_genes.krumsiek11-min_counts.h5ad
test-data/pp.filter_rank_genes_groups.h5ad
test-data/pp.highly_variable_genes.krumsiek11-cell_ranger.h5ad
test-data/pp.log1p.krumsiek11.h5ad
test-data/pp.normalize_total.krumsiek11.h5ad
test-data/pp.pca.krumsiek11_chunk.h5ad
test-data/pp.recipe_seurat.recipe_zheng17.h5ad
test-data/pp.recipe_weinreb17.paul15_subsample.updated.h5ad
test-data/pp.scale.krumsiek11.h5ad
test-data/pp.scale_max_value.krumsiek11.h5ad
test-data/pp.sqrt.krumsiek11.h5ad
test-data/pp.subsample.krumsiek11_fraction.h5ad
test-data/pp.subsample.krumsiek11_n_obs.h5ad
test-data/tl.embedding_density.umap.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.h5ad
test-data/tl.leiden.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad
test-data/tl.louvain.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad
test-data/tl.paga.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad
test-data/tl.rank_genes_groups.liblinear.krumsiek11.h5ad
test-data/tl.score_genes.krumsiek11.h5ad
test-data/tl.score_genes_cell_cycle.krumsiek11.h5ad
b
diff -r 9f354c7c8c92 -r 5dada6f76047 README.md
--- a/README.md Tue Aug 20 09:52:24 2024 +0000
+++ b/README.md Sat Sep 14 12:42:55 2024 +0000
[
@@ -25,6 +25,7 @@
     `pp.highly_variable_genes` | Extract highly variable genes
     `pp.subsample` | Subsample to a fraction of the number of observations
     `pp.downsample_counts` | Downsample counts so that each cell has no more than target_counts
+    `pp.scrublet` | Predict doublets
 
 3. Normalize (`normalize.xml`)
 
@@ -34,14 +35,18 @@
     `pp.recipe_zheng17` | Normalization and filtering as of [Zheng17]
     `pp.recipe_weinreb17` | Normalization and filtering as of [Weinreb17]
     `pp.recipe_seurat` | Normalization and filtering as of Seurat [Satija15]
+    `external.pp.magic` | Denoising using Markov Affinity-based Graph Imputation of Cells (MAGIC) API
 
 4. Remove confounders (`remove_confounder.xml`)
 
     Methods | Description
     --- | ---
    `pp.regress_out` | Regress out unwanted sources of variation
-   `pp.mnn_correct` | Correct batch effects by matching mutual nearest neighbors
+   <!-- `pp.mnn_correct` | Correct batch effects by matching mutual nearest neighbors -->
    `pp.combat` | ComBat function for batch effect correction
+    `external.pp.bbknn` | Batch effect removal with Batch balanced KNN (BBKNN)
+    `external.pp.harmony_integrate` | Integrate multiple single-cell experiments with Harmony
+    `external.pp.scanorama_integrate` | Integrate multiple single-cell experiments with Scanorama
 
 5. Clustering, embedding and trajectory inference (`cluster_reduce_dimension.xml`)
 
@@ -49,14 +54,14 @@
     --- | ---
     `tl.louvain` | Cluster cells into subgroups
     `tl.leiden` | Cluster cells into subgroups
-    `tl.pca` | Principal component analysis
-    `pp.pca` | Principal component analysis (appears to be the same func...)
+    `pp.pca` | Principal component analysis
     `tl.diffmap` | Diffusion Maps
     `tl.tsne` | t-SNE
     `tl.umap` | Embed the neighborhood graph using UMAP
     `tl.draw_graph` | Force-directed graph drawing
     `tl.dpt` | Infer progression of cells through geodesic distance along the graph
     `tl.paga` | Mapping out the coarse-grained connectivity structures of complex manifolds
+    `tl.embedding_density` | Calculate the density of cells in an embedding (per condition)
 
 6. Plot (`plot.xml`)
 
@@ -66,18 +71,20 @@
         --- | ---
         `pl.scatter` | Scatter plot along observations or variables axes
         `pl.heatmap` | Heatmap of the expression values of set of genes
+        `pl.tracksplot` | Tracks plot of the expression values per cell
         `pl.dotplot` | Makes a dot plot of the expression values
         `pl.violin` | Violin plot
         `pl.stacked_violin` | Stacked violin plots
         `pl.matrixplot` | Heatmap of the mean expression values per cluster
         `pl.clustermap` | Hierarchically-clustered heatmap
-    
+
     2. Preprocessing
 
         Methods | Description
         --- | ---
         `pl.highest_expr_genes` | Plot the fraction of counts assigned to each gene over all cells
         `pl.highly_variable_genes` | Plot dispersions versus means for genes
+        `pl.scrublet_score_distribution` | Histogram of doublet scores
 
     3. PCA
 
@@ -96,12 +103,13 @@
         `pl.umap` | Scatter plot in UMAP basis
         `pl.diffmap` | Scatter plot in Diffusion Map basis
         `pl.draw_graph` | Scatter plot in graph-drawing basis
+        `pl.embedding_density` | Density of cells in an embedding (per condition)
 
     5. Branching trajectories and pseudotime, clustering
 
         Methods | Description
         --- | ---
-        `pl.dpt_groups_pseudotime` | Plot groups and pseudotime
+        <!-- `pl.dpt_groups_pseudotime` | Plot groups and pseudotime -->
         `pl.dpt_timeseries` | Heatmap of pseudotime series
         `pl.paga` | Plot the abstracted graph through thresholding low-connectivity edges
         `pl.paga_compare` | Scatter and PAGA graph side-by-side
@@ -113,3 +121,8 @@
         --- | ---
         `pl.rank_genes_groups` | Plot ranking of genes using dotplot plot
         `pl.rank_genes_groups_violin` | Plot ranking of genes for all tested comparisons
+        `pl.rank_genes_groups_stacked_violin` | Plot ranking of genes as stacked violin plot
+        `pl.rank_genes_groups_heatmap` | Plot ranking of genes as heatmap plot
+        `pl.rank_genes_groups_dotplot` | Plot ranking of genes as dotplot plot
+        `pl.rank_genes_groups_matrixplot` | Plot ranking of genes as matrixplot plot
+        `pl.rank_genes_groups_tracksplot` | Plot ranking of genes as tracksplot plot
b
diff -r 9f354c7c8c92 -r 5dada6f76047 macros.xml
--- a/macros.xml Tue Aug 20 09:52:24 2024 +0000
+++ b/macros.xml Sat Sep 14 12:42:55 2024 +0000
[
b'@@ -1,17 +1,15 @@\n <macros>\n-    <token name="@TOOL_VERSION@">1.9.6</token>\n-    <token name="@VERSION_SUFFIX@">4</token>\n-    <token name="@profile@">21.09</token>\n+    <token name="@TOOL_VERSION@">1.10.2</token>\n+    <token name="@VERSION_SUFFIX@">0</token>\n+    <token name="@PROFILE@">21.09</token>\n     <xml name="requirements">\n         <requirements>\n             <requirement type="package" version="@TOOL_VERSION@">scanpy</requirement>\n-            <requirement type="package" version="3.0.6">loompy</requirement>\n-            <requirement type="package" version="0.10.1">leidenalg</requirement>\n-            <requirement type="package" version="0.8.1">louvain</requirement>\n-            <requirement type="package" version="1.5.3">pandas</requirement>\n-            <requirement type="package" version="3.7">matplotlib</requirement>\n-            <requirement type="package" version="0.12.2">seaborn</requirement>\n-            <requirement type="package" version="3.0.0">magic-impute</requirement>\n+            <requirement type="package" version="0.10.3">anndata</requirement>\n+            <requirement type="package" version="1.26.4">numpy</requirement>\n+            <requirement type="package" version="2.2.2">pandas</requirement>\n+            <requirement type="package" version="1.14.1">scipy</requirement>\n+            <requirement type="package" version="0.14.2">statsmodels</requirement>           \n             <yield />\n         </requirements>\n     </xml>\n@@ -22,7 +20,7 @@\n     </xml>\n     <xml name="creators">\n         <creator>\n-            <organization name="European Galaxy Team" url="https://galaxyproject.org/eu/" />\n+            <organization name="European Galaxy Team" url="https://galaxyproject.org/eu/"/>\n         </creator>\n     </xml>\n     <xml name="citations">\n@@ -31,28 +29,13 @@\n             <citation type="doi">10.1093/gigascience/giaa102</citation>\n         </citations>\n     </xml>\n-    <xml name="version_command">\n-        <version_command><![CDATA[python -c "import scanpy as sc;print(\'scanpy version: %s\' % sc.__version__)"]]></version_command>\n-    </xml>\n-    <token name="@CMD@"><![CDATA[\n-cp \'$adata\' \'anndata.h5ad\' &&\n-cat \'$script_file\' > \'$hidden_output\' &&\n-python \'$script_file\' >> \'$hidden_output\' &&\n-ls . >> \'$hidden_output\' &&\n-touch \'anndata_info.txt\' &&\n-cat \'anndata_info.txt\' @CMD_prettify_stdout@\n-    ]]>\n-    </token>\n-    <token name="@CMD_imports@"><![CDATA[\n-import scanpy as sc\n-import pandas as pd\n-import numpy as np\n-    ]]>\n-    </token>\n+    \n+    \n+    <!-- param macros -->\n     <xml name="sanitize_query" token_validinitial="string.printable">\n         <sanitizer>\n             <valid initial="@VALIDINITIAL@">\n-                <remove value="&apos;" />\n+                <remove value="&apos;"/>\n             </valid>\n        </sanitizer>\n     </xml>\n@@ -62,58 +45,55 @@\n                 <add value=","/>\n             </valid>\n         </sanitizer>\n-    </xml>\n+    </xml>      \n     <xml name="inputs_anndata">\n         <param name="adata" type="data" format="h5ad" label="Annotated data matrix"/>\n     </xml>\n-    <token name="@CMD_read_inputs@"><![CDATA[\n-adata = sc.read_h5ad(\'anndata.h5ad\')\n-]]>\n-    </token>\n     <xml name="inputs_common_advanced">\n         <section name="advanced_common" title="Advanced Options" expanded="false">\n-            <param name="show_log" type="boolean" checked="false" label="Output Log?" />\n+            <param name="show_log" type="boolean" checked="false" label="Output Log?"/>\n         </section>\n     </xml>\n     <xml name="anndata_outputs">\n-        <data name="anndata_out" format="h5ad" from_work_dir="anndata.h5ad" label="${tool.name} (${method.method}) on ${on_string}: Annotated data matrix"/>\n-            <data name="hidden_output" format="txt" label="Log file" >\n+        <data name="anndata_out" format="h5ad" from_work_dir="anndata.h5ad" label="${tool.name} (${method.method}) on ${on_string}: Annotated data matrix">\n+            <yield />\n+        </data>\n+       '..b'row_palette" type="select" optional="true" label="Colors to use in each of the stacked violin plots">\n             <option value="muted">muted</option>\n             <expand macro="seaborn_color_palette_options"/>\n         </param>\n-        <param argument="standard_scale" type="select" label="Standardize a dimension between 0 and 1" help="Each variable or observation is subtracted by the minimum and divided each by its maximum.">\n-            <option value="None">No standardization</option>\n-            <option value="var">Standardization on variable</option>\n-            <option value="obs">Standardization on observation</option>\n-        </param>\n-        <expand macro="seaborn_violinplot"/>\n+        <expand macro="param_standard_scale"/>\n+        <expand macro="params_seaborn_violinplot"/>\n+        <param argument="yticklabels" type="boolean" truevalue="True" falsevalue="False" checked="false" label="Set to view the y tick labels"/>\n+        <expand macro="param_vmin" label="The value representing the lower limit of the color scale" help="Values smaller than vmin are plotted with the same color as vmin."/>\n+        <expand macro="param_vmax" label="The value representing the upper limit of the color scale" help="Values larger than vmax are plotted with the same color as vmax."/>\n+        <expand macro="param_vcenter"/>\n     </xml>\n-    <token name="@CMD_pl_stacked_violin@"><![CDATA[\n+    <token name="@CMD_PARAMS_PL_STACKED_VIOLIN@"><![CDATA[\n     swap_axes=$method.swap_axes,\n-    @CMD_conditional_stripplot@\n-    scale=\'$method.violin_plot.scale\',\n-    #if $method.row_palette\n+    @CMD_CONDITIONAL_STRIPPLOT@\n+    density_norm=\'$method.violin_plot.density_norm\',\n+    #if $method.row_palette:\n     row_palette=\'$method.row_palette\',\n     #end if\n-    #if str($method.standard_scale) != \'None\'\n-    standard_scale=\'$method.standard_scale\',\n+    @CMD_STANDARD_SCALE@\n+    @CMD_PARAMS_SEABORN_VIOLINPLOT@\n+    yticklabels=$method.yticklabels,\n+    #if str($method.vmin) != \'\':\n+    vmin=$method.vmin,\n     #end if\n-    @CMD_params_seaborn_violinplot@\n+    #if str($method.vmax) != \'\':\n+    vmax=$method.vmax,\n+    #end if\n+    #if str($method.vcenter) != \'\':\n+    vcenter=$method.vcenter,\n+    #end if\n     ]]>\n     </token>\n+\n+    <xml name="params_scatter_outine">\n+        <param argument="add_outline" type="boolean" truevalue="True" falsevalue="False" checked="false" label="Add a thin border around groups of dots" help="In some situations this can enhance the aesthetics of the resulting image"/>\n+        <param name="outline_color_border" type="select" optional="true" label="Border color around the scatter dot" help="Default: black">\n+            <expand macro="matplotlib_color"/>\n+        </param>\n+        <param name="outline_color_gap" type="select" optional="true" label="Gap color between the border color and the scatter dot" help="Default: white">\n+            <expand macro="matplotlib_color"/>\n+        </param>\n+        <param argument="outline_width_border" type="float" value="0.3" optional="true" label=" width of the border color as a fraction of the scatter dot size "/>\n+        <param argument="outline_width_gap" type="float" value="0.05" optional="true" label="The width of the gap color"/>\n+    </xml>\n+    <token name="@CMD_SCATTER_OUTINE@"><![CDATA[\n+    add_outline=$method.add_outline,\n+    #if $method.outline_color_border and $method.outline_color_gap:\n+    outline_color=($method.outline_color_border, $method.outline_color_gap),\n+    #end if\n+    outline_width=($method.outline_width_border, $method.outline_width_gap),\n+    ]]>\n+    </token>\n+\n+\n+    <!-- unused macros -->\n+    <!-- <xml name="param_right_margin">\n+        <param argument="right_margin" type="float" value="1" label="Width of the space right of each plotting panel"/>\n+    </xml>\n+    <xml name="param_left_margin">\n+        <param argument="left_margin" type="float" value="1" label="Width of the space left of each plotting panel"/>\n+    </xml> -->\n </macros>\n'
b
diff -r 9f354c7c8c92 -r 5dada6f76047 normalize.xml
--- a/normalize.xml Tue Aug 20 09:52:24 2024 +0000
+++ b/normalize.xml Sat Sep 14 12:42:55 2024 +0000
[
b'@@ -1,45 +1,40 @@\n-<tool id="scanpy_normalize" name="Normalize" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@profile@">\n-    <description>and impute with scanpy</description>\n+<tool id="scanpy_normalize" name="Scanpy normalize" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@PROFILE@">\n+    <description>and impute</description>\n     <macros>\n         <import>macros.xml</import>\n     </macros>\n     <expand macro="bio_tools"/>\n-    <expand macro="requirements"/>\n+    <expand macro="requirements">\n+        <requirement type="package" version="3.0.0">magic-impute</requirement>\n+    </expand>\n     <expand macro="version_command"/>\n     <command detect_errors="exit_code"><![CDATA[\n @CMD@\n       ]]></command>\n     <configfiles>\n         <configfile name="script_file"><![CDATA[\n-@CMD_imports@\n-@CMD_read_inputs@\n+@CMD_IMPORTS@\n+@CMD_READ_INPUTS@\n \n-#if $method.method == "pp.normalize_total"\n+#if str($method.method) == \'pp.normalize_total\':\n sc.pp.normalize_total(\n     adata,\n-    #if str($method.target_sum) != \'\'\n+    #if str($method.target_sum) != \'\':\n     target_sum=$method.target_sum,\n     #end if\n     exclude_highly_expressed=$method.exclude_highly_expressed.exclude_highly_expressed,\n-    #if $method.exclude_highly_expressed.exclude_highly_expressed == "True"\n+    #if str($method.exclude_highly_expressed.exclude_highly_expressed) == \'True\':\n     max_fraction=$method.exclude_highly_expressed.max_fraction,\n     #end if\n-    #if $method.key_added\n+    #if str($method.key_added) != \'\':\n     key_added=\'$method.key_added\',\n     #end if\n-    #if $method.layers\n-        #if str($method.layers) != \'all\'\n-    layers[str(x.strip()) for x in str($method.layers).split(\',\')],\n-        #else\n-    layers=\'$method.layers\',\n-        #end if\n-    #end if\n-    #if str($method.layer_norm) != "None"\n-        layer_norm=\'$method.layer_norm\',\n+    #if str($method.layer) != \'\':\n+        layer=\'$method.layer\',\n     #end if\n     inplace=True)\n \n-#else if $method.method == "pp.recipe_zheng17"\n+#else if str($method.method) == \'pp.recipe_zheng17\':\n sc.pp.recipe_zheng17(\n     adata=adata,\n     n_top_genes=$method.n_top_genes,\n@@ -47,52 +42,62 @@\n     plot=False,\n     copy=False)\n \n-#else if $method.method == "pp.recipe_weinreb17"\n+#else if str($method.method) == \'pp.recipe_weinreb17\':\n sc.pp.recipe_weinreb17(\n     adata=adata,\n     log=$method.log,\n-    mean_threshold=$method.mean_threshold,\n-    cv_threshold=$method.cv_threshold,\n-    n_pcs=$method.n_pcs,\n-    svd_solver=\'$method.svd_solver\',\n-    random_state=$method.random_state,\n+    mean_threshold=0.01,\n+    cv_threshold=2,\n+    n_pcs=50,\n+    svd_solver=\'randomized\',\n+    random_state=0,\n     copy=False)\n \n-#else if $method.method == "pp.recipe_seurat"\n+#else if str($method.method) == \'pp.recipe_seurat\':\n sc.pp.recipe_seurat(\n     adata=adata,\n     log=$method.log,\n     plot=False,\n     copy=False)\n \n-#else if $method.method == "external.pp.magic"\n+#else if str($method.method) == \'external.pp.magic\':\n+print("stats before magic:", "min=", f"{adata.X.min():.5f}", "max=", f"{adata.X.max():.5f}", "mean=", f"{adata.X.mean():.5f}")\n+\n sc.external.pp.magic(\n     adata=adata,\n     name_list=\'$method.name_list\',\n     knn=$method.knn,\n-    #if str($method.decay) != \'\'\n+    #if str($method.decay) != \'\':\n     decay=$method.decay,\n     #end if\n-    #if str($method.knn_max) != \'\'\n+    #if str($method.knn_max) != \'\':\n     knn_max=$method.knn_max,\n     #end if\n-    #if $method.t == -1\n+    #if $method.t == -1:\n     t=\'auto\',\n     #else\n     t=$method.t,\n     #end if\n-    #if str($method.n_pca) != \'\'\n+    #if str($method.n_pca) != \'\':\n     n_pca=$method.n_pca,\n     #end if\n     solver=\'$method.solver\',\n     knn_dist=\'$method.knn_dist\',\n+    #if str($method.random_state) != \'\':\n     random_state=$method.random_state,\n+    #else\n+    random_state=None,\n+    #end if\n     copy=False)\n+\n+    #if str($method.name_list) == \'all_genes\':\n+print("stats after magic:", "min=", f"{adata.X.min():.5f}", "max=", f"'..b'lue="true"/>\n             </section>\n             <output name="hidden_output">\n                 <assert_contents>\n@@ -282,13 +283,20 @@\n                     <has_text_matching expression="name_list=\'all_genes\'"/>\n                     <has_text_matching expression="t=\'auto\'"/>\n                     <has_text_matching expression="n_pca=5"/>\n+                    <has_text_matching expression="stats before magic: min= -0.01630 max= 1.01060 mean= 0.28644"/>\n+                    <has_text_matching expression="stats after magic: min= -0.00857 max= 1.00546 mean= 0.28645"/>\n                 </assert_contents>\n             </output>\n-            <output name="anndata_out" file="external.pp.magic.all_genes.krumsiek11.h5ad" ftype="h5ad" compare="sim_size"/>\n+            <output name="anndata_out" ftype="h5ad">\n+                <assert_contents>\n+                    <has_h5_keys keys="obs/cell_type"/>\n+                </assert_contents>\n+            </output>\n         </test>\n+\n+        <!-- test 6 -->\n         <test expect_num_outputs="2">\n-            <!-- test 6 -->\n-            <param name="adata" value="krumsiek11.h5ad" />\n+            <param name="adata" value="krumsiek11.h5ad"/>\n             <conditional name="method">\n                 <param name="method" value="external.pp.magic"/>\n                 <param name="name_list" value="pca_only"/>\n@@ -296,7 +304,7 @@\n                 <param name="n_pca" value="5"/>\n             </conditional>\n             <section name="advanced_common">\n-                <param name="show_log" value="true" />\n+                <param name="show_log" value="true"/>\n             </section>\n             <output name="hidden_output">\n                 <assert_contents>\n@@ -306,23 +314,26 @@\n                     <has_text_matching expression="n_pca=5"/>\n                 </assert_contents>\n             </output>\n-            <output name="anndata_out" file="external.pp.magic.pca_only.krumsiek11.h5ad" ftype="h5ad" compare="sim_size"/>\n+            <output name="anndata_out" ftype="h5ad">\n+                <assert_contents>\n+                    <has_h5_keys keys="obsm/X_magic"/>\n+                </assert_contents>\n+            </output>\n             <assert_stdout>\n                 <has_text text="X_magic"/>\n             </assert_stdout>\n         </test>\n     </tests>\n     <help><![CDATA[\n-Normalize total counts per cell (`pp.normalize_per_cell`)\n-=========================================================\n+Normalize total counts per cell (`pp.normalize_total`)\n+======================================================\n \n-Normalize each cell by total counts over all genes, so that every cell has\n-the same total count after normalization.\n+Normalize each cell by total counts over all genes, so that every cell has the same total count after normalization. If choosing target_sum=1e6, this is CPM normalization.\n \n Similar functions are used, for example, by Seurat, Cell Ranger or SPRING.\n \n More details on the `scanpy documentation\n-<https://scanpy.readthedocs.io/en/stable/api/scanpy.pp.normalize_per_cell.html>`__\n+<https://scanpy.readthedocs.io/en/stable/api/scanpy.pp.normalize_total.html>`__\n \n \n Normalization and filtering as of Zheng et al. (2017), the Cell Ranger R Kit of 10x Genomics (`pp.recipe_zheng17`)\n@@ -369,7 +380,7 @@\n \n MAGIC is an algorithm for denoising and transcript recover of single cells applied to single-cell sequencing data. MAGIC builds a graph from the data and uses diffusion to smooth out noise and recover the data manifold.\n \n-The algorithm implemented here has changed primarily in two ways compared to the algorithm described in Van Dijk D et al. (2018). \n+The algorithm implemented here has changed primarily in two ways compared to the algorithm described in Van Dijk D et al. (2018).\n \n - Firstly, we use the adaptive kernel described in Moon et al, (2019) for improved stability.\n - Secondly, data diffusion is applied in the PCA space, rather than the data space, for speed and memory improvements.\n'
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/cosg.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_1.h5ad
b
Binary file test-data/cosg.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_1.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/external.pp.bbknn.krumsiek11.h5ad
b
Binary file test-data/external.pp.bbknn.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/external.pp.magic.all_genes.krumsiek11.h5ad
b
Binary file test-data/external.pp.magic.all_genes.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/external.pp.magic.pca_only.krumsiek11.h5ad
b
Binary file test-data/external.pp.magic.pca_only.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pbmc68k_reduced.h5ad
b
Binary file test-data/pbmc68k_reduced.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.clustermap.krumsiek11.png
b
Binary file test-data/pl.clustermap.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.diffmap.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.png
b
Binary file test-data/pl.diffmap.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.dotplot.krumsiek11.png
b
Binary file test-data/pl.dotplot.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.dpt_timeseries.dpt.diffmap.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.png
b
Binary file test-data/pl.dpt_timeseries.dpt.diffmap.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.draw_graph.png
b
Binary file test-data/pl.draw_graph.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.embedding_density.pbmc68k_reduced.png
b
Binary file test-data/pl.embedding_density.pbmc68k_reduced.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.heatmap.krumsiek11.png
b
Binary file test-data/pl.heatmap.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.highest_expr_genes.filter_genes_dispersion.krumsiek11-seurat.png
b
Binary file test-data/pl.highest_expr_genes.filter_genes_dispersion.krumsiek11-seurat.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.highly_variable_genes.seurat.blobs.png
b
Binary file test-data/pl.highly_variable_genes.seurat.blobs.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.matrixplot.krumsiek11.png
b
Binary file test-data/pl.matrixplot.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.paga.paul15_gauss_braycurtis.png
b
Binary file test-data/pl.paga.paul15_gauss_braycurtis.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.paga_compare.paul15_gauss_braycurtis.png
b
Binary file test-data/pl.paga_compare.paul15_gauss_braycurtis.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.pca.pbmc68k_reduced.CD3D_CD79A_components_2d.pdf
b
Binary file test-data/pl.pca.pbmc68k_reduced.CD3D_CD79A_components_2d.pdf has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.pca_loadings.pp.pca.krumsiek11.png
b
Binary file test-data/pl.pca_loadings.pp.pca.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.pca_overview.pp.pca.krumsiek11.png
b
Binary file test-data/pl.pca_overview.pp.pca.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.pca_variance_ratio.pp.pca.krumsiek11.png
b
Binary file test-data/pl.pca_variance_ratio.pp.pca.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_marker_1.png
b
Binary file test-data/pl.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_marker_1.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_marker_filtered_1.png
b
Binary file test-data/pl.rank_genes_groups.newton-cg.pbmc68k_highly_reduced_marker_filtered_1.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups.rank_genes_groups.krumsiek11.png
b
Binary file test-data/pl.rank_genes_groups.rank_genes_groups.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_dotplot.rank_genes_groups.krumsiek11.png
b
Binary file test-data/pl.rank_genes_groups_dotplot.rank_genes_groups.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_heatmap.rank_genes_groups.krumsiek11.png
b
Binary file test-data/pl.rank_genes_groups_heatmap.rank_genes_groups.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_matrixplot.rank_genes_groups.krumsiek11.png
b
Binary file test-data/pl.rank_genes_groups_matrixplot.rank_genes_groups.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_stacked_violin.rank_genes_groups.krumsiek11.png
b
Binary file test-data/pl.rank_genes_groups_stacked_violin.rank_genes_groups.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_tracksplot.newton-cg.pbmc68k_highly_reduced_marker_filtered_1.png
b
Binary file test-data/pl.rank_genes_groups_tracksplot.newton-cg.pbmc68k_highly_reduced_marker_filtered_1.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_violin.Ery.png
b
Binary file test-data/pl.rank_genes_groups_violin.Ery.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_violin.Mk.png
b
Binary file test-data/pl.rank_genes_groups_violin.Mk.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_violin.Mo.png
b
Binary file test-data/pl.rank_genes_groups_violin.Mo.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_violin.Neu.png
b
Binary file test-data/pl.rank_genes_groups_violin.Neu.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.rank_genes_groups_violin.progenitor.png
b
Binary file test-data/pl.rank_genes_groups_violin.progenitor.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.scatter.krumsiek11.png
b
Binary file test-data/pl.scatter.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.scatter.pbmc68k_reduced.png
b
Binary file test-data/pl.scatter.pbmc68k_reduced.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.scatter.umap.pbmc68k_reduced.png
b
Binary file test-data/pl.scatter.umap.pbmc68k_reduced.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.scrublet_score_distribution.png
b
Binary file test-data/pl.scrublet_score_distribution.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.stacked_violin.krumsiek11.png
b
Binary file test-data/pl.stacked_violin.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.stacked_violin_pp.filter_genes_dispersion.krumsiek11-seurat.png
b
Binary file test-data/pl.stacked_violin_pp.filter_genes_dispersion.krumsiek11-seurat.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.tsne.krumsiek11.png
b
Binary file test-data/pl.tsne.krumsiek11.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.umap.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.png
b
Binary file test-data/pl.umap.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pl.violin.pbmc68k_reduced_custom.png
b
Binary file test-data/pl.violin.pbmc68k_reduced_custom.png has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.calculate_qc_metrics.sparce_csr_matrix.h5ad
b
Binary file test-data/pp.calculate_qc_metrics.sparce_csr_matrix.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.downsample_counts.random-randint.h5ad
b
Binary file test-data/pp.downsample_counts.random-randint.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.filter_cells.krumsiek11-max_genes.h5ad
b
Binary file test-data/pp.filter_cells.krumsiek11-max_genes.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.filter_cells.krumsiek11-min_counts.h5ad
b
Binary file test-data/pp.filter_cells.krumsiek11-min_counts.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.filter_genes.krumsiek11-min_counts.h5ad
b
Binary file test-data/pp.filter_genes.krumsiek11-min_counts.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.filter_rank_genes_groups.h5ad
b
Binary file test-data/pp.filter_rank_genes_groups.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.highly_variable_genes.krumsiek11-cell_ranger.h5ad
b
Binary file test-data/pp.highly_variable_genes.krumsiek11-cell_ranger.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.log1p.krumsiek11.h5ad
b
Binary file test-data/pp.log1p.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.normalize_total.krumsiek11.h5ad
b
Binary file test-data/pp.normalize_total.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.pca.krumsiek11.batch.h5ad
b
Binary file test-data/pp.pca.krumsiek11.batch.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.pca.krumsiek11_chunk.h5ad
b
Binary file test-data/pp.pca.krumsiek11_chunk.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.recipe_seurat.recipe_zheng17.h5ad
b
Binary file test-data/pp.recipe_seurat.recipe_zheng17.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.recipe_weinreb17.paul15_subsample.updated.h5ad
b
Binary file test-data/pp.recipe_weinreb17.paul15_subsample.updated.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.recipe_zheng17.random-randint.h5ad
b
Binary file test-data/pp.recipe_zheng17.random-randint.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.scale.krumsiek11.h5ad
b
Binary file test-data/pp.scale.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.scale_max_value.krumsiek11.h5ad
b
Binary file test-data/pp.scale_max_value.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.scrublet.krumsiek11.h5ad
b
Binary file test-data/pp.scrublet.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.sqrt.krumsiek11.h5ad
b
Binary file test-data/pp.sqrt.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.subsample.krumsiek11_fraction.h5ad
b
Binary file test-data/pp.subsample.krumsiek11_fraction.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/pp.subsample.krumsiek11_n_obs.h5ad
b
Binary file test-data/pp.subsample.krumsiek11_n_obs.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/random-randint.h5ad
b
Binary file test-data/random-randint.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/sparce_csr_matrix.h5ad
b
Binary file test-data/sparce_csr_matrix.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.embedding_density.umap.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.h5ad
b
Binary file test-data/tl.embedding_density.umap.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.embedding_density.umap.pbmc68k_reduced.h5ad
b
Binary file test-data/tl.embedding_density.umap.pbmc68k_reduced.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.leiden.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad
b
Binary file test-data/tl.leiden.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.louvain.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad
b
Binary file test-data/tl.louvain.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.paga.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad
b
Binary file test-data/tl.paga.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.rank_genes_groups.liblinear.krumsiek11.h5ad
b
Binary file test-data/tl.rank_genes_groups.liblinear.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.rank_genes_groups.newton-cg.pbmc68k_reduced.h5ad
b
Binary file test-data/tl.rank_genes_groups.newton-cg.pbmc68k_reduced.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.score_genes.krumsiek11.h5ad
b
Binary file test-data/tl.score_genes.krumsiek11.h5ad has changed
b
diff -r 9f354c7c8c92 -r 5dada6f76047 test-data/tl.score_genes_cell_cycle.krumsiek11.h5ad
b
Binary file test-data/tl.score_genes_cell_cycle.krumsiek11.h5ad has changed