FilippoGas/GeneProgramsProfiler
Characterize cell type specific gene program deregulations from scRNAseq datasets.
Overview
Latest release: None, Last update: 2026-08-28
Share link: https://snakemake.github.io/snakemake-workflow-catalog?wf=FilippoGas/GeneProgramsProfiler
Quality control: linting: passed formatting: passed
Deployment
Step 1: Install Snakemake and Snakedeploy
Snakemake and Snakedeploy are best installed via the Conda package manager. It is recommended to install conda via Miniforge. Run
conda create -c conda-forge -c bioconda -c nodefaults --name snakemake snakemake snakedeploy
to install both Snakemake and Snakedeploy in an isolated environment. For all following commands ensure that this environment is activated via
conda activate snakemake
For other installation methods, refer to the Snakemake and Snakedeploy documentation.
Step 2: Deploy workflow
With Snakemake and Snakedeploy installed, the workflow can be deployed as follows. First, create an appropriate project working directory on your system and enter it:
mkdir -p path/to/project-workdir
cd path/to/project-workdir
In all following steps, we will assume that you are inside of that directory. Then run
snakedeploy deploy-workflow https://github.com/FilippoGas/GeneProgramsProfiler . --tag None
Snakedeploy will create two folders, workflow and config. The former contains the deployment of the chosen workflow as a Snakemake module, the latter contains configuration files which will be modified in the next step in order to configure the workflow to your needs.
Step 3: Configure workflow
To configure the workflow, adapt config/config.yml to your needs following the instructions below.
Step 4: Run workflow
The deployment method is controlled using the --software-deployment-method (short --sdm) argument.
To run the workflow with automatic deployment of all required software via conda/mamba, use
snakemake --cores all --sdm conda
Snakemake will automatically detect the main Snakefile in the workflow subfolder and execute the workflow module that has been defined by the deployment in step 2.
For further options such as cluster and cloud execution, see the docs.
Step 5: Generate report
After finalizing your data analysis, you can automatically generate an interactive visual HTML report for inspection of results together with parameters and code inside of the browser using
snakemake --report report.zip
Configuration
The following section is imported from the workflow’s config/README.md.
Configuration
This workflow is configured via config/config.yaml. All parameters are validated at startup against the schema defined in workflow/schemas/config.schema.yaml.
The test dataset (Natri et al., IPF vs Control) is configured in .test/config/config.yaml and used automatically when running CI or dry-runs from the repository root.
Input data
Seurat object (.rds)
A Seurat object with the following columns in its metadata:
Metadata column |
Description |
|---|---|
Sample name column |
One column identifying the biological sample each cell belongs to (set via |
Condition column |
One column with the case/control condition per sample (set via |
Cell-cycle phase column |
One column with the cell-cycle phase per cell (set via |
Cell-type annotation column |
One column with the cell-type label per cell (set via |
Cytopus cell-type dictionary (.json)
A JSON file mapping the cell-type labels present in your Seurat object to cytopus cell-type identifiers. Example:
{
"Macrophages": "mac",
"Fibroblasts": "fib",
"AT1": "at1",
"AT2": "at2"
}
Global settings
Key |
Type |
Description |
|---|---|---|
|
string |
Path to the input Seurat |
|
string |
Path to the cytopus cell-type conversion JSON |
|
string |
Name of this analysis. All outputs are written to |
|
string |
Label for the case/disease condition in the condition column |
|
string |
Label for the control condition in the condition column |
|
string |
HPC queue name for CPU jobs (used by the cluster launcher) |
Module 1: Preprocessing (preprocess)
preprocess.annotate_and_save
Annotates the Seurat object with cytopus cell types, performs cell-cycle scoring, and saves the dataset as .rds, 10X Genomics .mtx format, and AnnData .h5ad.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) for loading the Seurat object in R |
|
string |
yes |
Name of the cell-type annotation column in the Seurat metadata |
|
string |
yes |
Column of sample name in the Seurat metadata |
|
string |
yes |
Column of condition name in the Seurat metadata |
|
string |
yes |
Column of cell-cycle phase in the Seurat metadata |
|
string |
no |
Job walltime ( |
Module 2: Spectra gene program discovery (spectra)
Implements Spectra for identifying cell-type-specific gene programs using expression data and cytopus gene sets.
spectra.prepare_cytopus_list
Downloads cytopus gene sets for the cell types present in the dataset.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
string |
yes |
Cell type to use as global cell type in the cytopus list |
|
string |
no |
Job walltime |
spectra.run_spectra
Runs Spectra to quantify gene program activation in single cells.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
float |
yes |
Weighs the relative contribution of cytopus list vs expression loss functions (range: 0.0001–0.5) |
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
string |
no |
Job walltime |
spectra.rename_programs
Labels unlabeled factors via ORA enrichment of marker genes against cytopus gene sets.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
string |
no |
Job walltime |
spectra.spectra_WMW / spectra.spectra_LMM
Differential activation testing of spectra gene programs between conditions. WMW uses Wilcoxon-Mann-Whitney U-test; LMM uses Linear Mixed Models to correct for cell-cycle phase.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes |
Activation threshold to consider a program active in a cell |
|
string |
no |
Job walltime |
spectra.spectra_WMW_plots / spectra.spectra_LMM_plots
Plots from the differential activation analysis (volcano plots, heatmaps).
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes (WMW) |
Effect size threshold (rank-biserial correlation) |
|
float |
yes (LMM) |
Log2 fold-change threshold |
|
float |
yes |
False Discovery Rate threshold |
|
string |
no |
Job walltime |
Module 3: Differential expression analysis (DE_analysis)
DE_analysis.run_DE_analysis
Runs differential expression analysis between the case and control conditions.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes |
Log fold-change threshold to consider a gene differentially expressed |
|
float |
yes |
FDR threshold to consider a gene differentially expressed |
|
string |
no |
Job walltime |
DE_analysis.DEA_plots
Generates diagnostic plots (p-value overlap, correlation plots).
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
string |
no |
Job walltime |
Module 4: Functional enrichment (functional_enrichment)
functional_enrichment.run_gsea
Runs Gene Set Enrichment Analysis using fgsea on the differential expression results.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes |
Adjusted p-value threshold for significance |
|
string |
no |
Job walltime |
functional_enrichment.run_ora
Runs Over-Representation Analysis on the differential expression results.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes |
Adjusted p-value threshold for significance |
|
string |
no |
Job walltime |
Module 5: cNMF gene program discovery (cNMF)
Implements consensus NMF as an alternative gene program discovery method, with automatic k-selection and consensus clustering.
cNMF.cNMF_prepare
Normalizes the count matrix and prepares the factorization step. Defines the range of k values to evaluate.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
integer |
yes |
Maximum NMF optimization iterations per replicate |
|
integer |
yes |
Minimum value of k to try |
|
integer |
yes |
Maximum value of k to try |
|
integer |
yes |
Step size for k sweep |
|
integer |
yes |
Number of factorization iterations for each k |
|
string |
no |
Job walltime |
cNMF.cNMF_factorize_worker
Runs a single factorization worker. The cores parameter here sets the number of parallel workers (each worker runs with threads: 1), not CPUs per job.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) per worker |
|
integer |
yes |
Number of parallel worker jobs to spawn |
|
string |
no |
Job walltime |
cNMF.cNMF_combine
Combines factorization results across all k values.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
string |
no |
Job walltime |
cNMF.cNMF_k_selection_plot
Generates a plot estimating the trade-off between higher k, stability, and error. Used for diagnostics; the actual k is selected automatically.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
string |
no |
Job walltime |
cNMF.extract_best_k
Selects the k value with the best stability-error tradeoff from the k-selection statistics.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
string |
no |
Job walltime |
cNMF.cNMF_consensus
Generates program usage tables for the selected k. Filters out unstable outlier programs before consensus clustering.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
float |
yes |
Maximum distance threshold to nearest neighbors for filtering unstable programs |
|
string |
no |
Job walltime |
cNMF.cNMF_rename_programs
Runs ORA on cNMF program markers to assign biological labels.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
string |
no |
Job walltime |
cNMF.cNMF_WMW / cNMF.cNMF_LMM
Differential activation testing of cNMF gene programs. Same statistical approaches as the spectra equivalents.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes |
Activation threshold to consider a program active in a cell |
|
string |
no |
Job walltime |
cNMF.cNMF_WMW_plots / cNMF.cNMF_LMM_plots
Plots from cNMF differential activation analysis.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Number of threads |
|
integer |
yes |
Memory (MB) |
|
float |
yes (WMW) |
Effect size threshold (rank-biserial correlation) |
|
float |
yes (LMM) |
Log2 fold-change threshold |
|
float |
yes |
False Discovery Rate threshold |
|
string |
no |
Job walltime |
Module 6: Collect results (collect_results)
collect_results.make_comp_table
Combines results from spectra, cNMF, and functional enrichment into a single comparative table.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
float |
yes |
Adjusted p-value threshold for enrichments to include in the table |
|
string |
no |
Job walltime |
collect_results.comp_table_plots
Generates UpSet plots showing concordance of detected deregulations across methods.
Key |
Type |
Required |
Description |
|---|---|---|---|
|
integer |
yes |
Memory (MB) |
|
integer |
yes |
Number of threads |
|
float |
yes |
FDR threshold for WMW and LMM results in the plots |
|
float |
yes |
Effect size threshold for WMW results in the plots |
|
float |
yes |
Log2FC threshold for LMM results in the plots |
|
string |
no |
Job walltime |
Resource notes
timefields follow the patternHH:MM:SSand are only needed when running on HPC clusters with job scheduling (PBS/SLURM). They can be left empty or omitted for local execution.mem_mbvalues in.test/config/config.yamlare tuned for the test dataset; production values may be significantly higher, especially forcNMF.cNMF_factorize_workerandspectra.run_spectra.
Workflow parameters
The following table is automatically parsed from the workflow’s config.schema.y(a)ml file.
Parameter |
Type |
Description |
Required |
Default |
|---|---|---|---|---|
scRNAseq |
string |
yes |
||
celltype_conversion_dictionary |
string |
Path to json dictionary to match celltype in the dataset to celltypes in cytopus |
yes |
|
analysis_name |
string |
yes |
||
queues |
||||
. cpu |
string |
|||
. gpu |
string |
|||
case_condition |
string |
yes |
||
control_condition |
string |
yes |
||
preprocess |
||||
. annotate_and_save |
||||
. . celltype_annotation_colname |
string |
Name of celltype annotation column in the seurat object’s metadata |
yes |
|
. . cores |
integer |
yes |
||
. . rstudio_memory |
integer |
Memory occupied by loading the scRNAseq dataset on R |
yes |
|
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . sample_column |
string |
Column of sample name in the Seurat metadata |
yes |
|
. . condition_column |
string |
Column of condition name in the Seurat metadata |
yes |
|
. . cell_cycle_phase_column |
string |
Column of condition cell cycle phase in the Seurat metadata |
yes |
|
spectra |
||||
. prepare_cytopus_list |
||||
. . mem_mb |
integer |
Memory required to donwload and save the required cytopus gene sets |
yes |
|
. . cores |
integer |
yes |
||
. . global_celltype |
string |
cell type to use as global celltype in the cytopus list. For more details visit https://github.com/wallet-maker/cytopus |
yes |
|
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. run_spectra |
||||
. . lambda |
number |
weighs relative contribution of cytopus list and expression loss functions. For more details visit https://github.com/dpeerlab/spectra |
yes |
|
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. rename_programs |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. spectra_WMW |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . active_cell_thresh |
number |
Activation threshold to consider a program active in a cell |
yes |
|
. spectra_WMW_plots |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . effect_size_thresh |
number |
Effect size threshold for Wilcoxon-Mann-Whitney U-test’s rank-biserial correlation |
yes |
|
. . FDR_thresh |
number |
False Discovery Rate threshold for Wilcoxon-Mann-Whitney U-test |
yes |
|
. spectra_LMM |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . active_cell_thresh |
number |
Activation threshold to consider a program active in a cell |
yes |
|
. spectra_LMM_plots |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . log2FC_thresh |
number |
Effect size threshold for Wilcoxon-Mann-Whitney U-test’s rank-biserial correlation |
yes |
|
. . FDR_thresh |
number |
False Discovery Rate threshold for Wilcoxon-Mann-Whitney U-test |
yes |
|
DE_analysis |
||||
. run_DE_analysis |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . logFC |
number |
log(Fold Change) threshold to consider a gene to be differentially expressed |
yes |
|
. . FDR |
number |
False Discovery Rate threshold to consider a gene to be differentially expressed |
yes |
|
. DEA_plots |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
functional_enrichment |
||||
. run_gsea |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . padj_thresh |
number |
Value to use as threshold for adjusted pvalue to consider fgsea results significant. |
yes |
|
. run_ora |
||||
. . cores |
integer |
|||
. . mem_mb |
integer |
|||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . padj_thresh |
number |
|||
cNMF |
||||
. cNMF_prepare |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. . max_nmf_iter |
integer |
maximum number of optimization iterations that the underlying Non-negative Matrix Factorization (NMF) solver is allowed to perform in order to reach convergence during a single factorization replicate. |
yes |
|
. . k_min |
integer |
Minimum value of k to try |
yes |
|
. . k_max |
integer |
Maximum value of k to try |
yes |
|
. . k_step |
integer |
yes |
||
. . n_iter |
integer |
Number of iterations for each k |
yes |
|
. cNMF_factorize_worker |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. cNMF_combine |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. cNMF_k_selection_plot |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. extract_best_k |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. cNMF_consensus |
||||
. . mem_mb |
integer |
yes |
||
. . cores |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . local_density_threshold |
number |
It sets a maximum distance threshold to nearest neighbors, filtering out unstable outlier programs before the final consensus clustering. for more details visit https://github.com/dylkot/cNMF |
yes |
|
. cNMF_rename_programs |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. cNMF_WMW |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . active_cell_thresh |
number |
Activation threshold to consider a program active in a cell |
yes |
|
. cNMF_WMW_plots |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . effect_size_thresh |
number |
Effect size threshold for Wilcoxon-Mann-Whitney U-test’s rank-biserial correlation |
yes |
|
. . FDR_thresh |
number |
False Discovery Rate threshold for Wilcoxon-Mann-Whitney U-test |
yes |
|
. cNMF_LMM |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . active_cell_thresh |
number |
Activation threshold to consider a program active in a cell |
yes |
|
. cNMF_LMM_plots |
||||
. . cores |
integer |
yes |
||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . log2FC_thresh |
number |
Effect size threshold for Wilcoxon-Mann-Whitney U-test’s rank-biserial correlation |
yes |
|
. . FDR_thresh |
number |
False Discovery Rate threshold for Wilcoxon-Mann-Whitney U-test |
yes |
|
collect_results.smk |
||||
. make_comp_table |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. . padj_thresh |
number |
Significance threshold for enrichments results to be included in the table |
yes |
|
. comp_table_plots |
||||
. . mem_mb |
integer |
yes |
||
. . time |
string |
Job walltime, if required by computing infrastructure. |
||
. . cores |
integer |
yes |
||
. . FDR_thresh |
number |
Significance threshold for Wilcoxon-Mann-Whitney and Linear Mixed Models results to be included in the plots |
yes |
|
. . effect_size_thresh |
number |
Effect size threshold for Wilcoxon-Mann-Whitney results to be included in the plots |
yes |
|
. . log2FC_thresh |
number |
log2FC threshold for Linear Mixed Models results to be included in the plots |
yes |
Linting and formatting
Linting results
All tests passed!
Formatting results
All tests passed!