tavareshugo/smk-indexer

Snakemake workflow for indexing reference genomes

Overview

Latest release: None, Last update: 2026-08-11

Share link: https://snakemake.github.io/snakemake-workflow-catalog?wf=tavareshugo/smk-indexer

Quality control: linting: failed formatting: failed

Wrappers: bio/bismark/bismark_genome_preparation bio/bowtie2/build bio/bwa-mem2/index bio/bwa/index bio/hisat2/index bio/salmon/decoys bio/salmon/index bio/samtools/faidx bio/star/index

Deployment

Step 1: Install Snakemake and Snakedeploy

Snakemake and Snakedeploy are best installed via the Conda package manager. It is recommended to install conda via Miniforge. Run

conda create -c conda-forge -c bioconda -c nodefaults --name snakemake snakemake snakedeploy

to install both Snakemake and Snakedeploy in an isolated environment. For all following commands ensure that this environment is activated via

conda activate snakemake

For other installation methods, refer to the Snakemake and Snakedeploy documentation.

Step 2: Deploy workflow

With Snakemake and Snakedeploy installed, the workflow can be deployed as follows. First, create an appropriate project working directory on your system and enter it:

mkdir -p path/to/project-workdir
cd path/to/project-workdir

In all following steps, we will assume that you are inside of that directory. Then run

snakedeploy deploy-workflow https://github.com/tavareshugo/smk-indexer . --tag None

Snakedeploy will create two folders, workflow and config. The former contains the deployment of the chosen workflow as a Snakemake module, the latter contains configuration files which will be modified in the next step in order to configure the workflow to your needs.

Step 3: Configure workflow

To configure the workflow, adapt config/config.yml to your needs following the instructions below.

Step 4: Run workflow

The deployment method is controlled using the --software-deployment-method (short --sdm) argument.

To run the workflow with automatic deployment of all required software via conda/mamba, use

snakemake --cores all --sdm conda

Snakemake will automatically detect the main Snakefile in the workflow subfolder and execute the workflow module that has been defined by the deployment in step 2.

For further options such as cluster and cloud execution, see the docs.

Step 5: Generate report

After finalizing your data analysis, you can automatically generate an interactive visual HTML report for inspection of results together with parameters and code inside of the browser using

snakemake --report report.zip

Configuration

The following section is imported from the workflow’s config/README.md.

Workflow configuration: smk-indexer

Genome sheet

The main input is a CSV file specifying the source of the genome files, with the following columns:

  • name: will be used as the output directory name; we use <species>-<genome>-<source>-<release/version> as a convention (see naming details below).

  • genome: url to genome file.

  • gtf: url to gtf annotation file.

  • gff: url to gff annotation file.

For consistency, we name our genome directories as <species>-<genome>-<source>-<release/version>, where:

  • organism is the first letter of the genus followed by species scientific name.

  • genome is the name of the genome in the respective database.

  • source is the source of the genome. For example ensembl, gencode, ncbi, flybase, etc.

  • release/version is the release or version number of the genome release on the respective database. If no version number is available use YYYYMMDD of the genome file being downloaded.

Here’s some examples:

  • hsapiens-GRCh38-ensembl-112 human genome using ENSEMBL release 112 (source)

  • hsapiens-GRCh38-gencode-45 effectively the same assembly as above, but using GENCODE release 45 (source)

  • hsapiens-GRCh37-gencode-45 the same GENCODE release, but for an earlier version of the human genome (source)

  • mmusculus-A_J_v1-ensembl-112 mouse reference genome for strain A/J on ENSEMBL release 112 (source)

  • dmelanogaster-BDGP6-ensembl-112 Drosophila melanogaster genome, using the genome name given in ENSEMBL release 112 (source)

  • dsimulans-dsim-flybase-r2.02 Drosophila simulans genome from flybase, release r2.02 (source)

Options

The config/options.yml can be used to configure the workflow options:

  • input: is the path to the genome CSV sheet, as detailed above.

  • outdir: is the path to the output directory. Sub-directories will be created within this, as detailed above.

Running the workflow

On CSD3, the workflow can be run with the following command:

snakemake --workflow-profile csd3 --singularity-args "--bind $PWD"

  • The --workflow-profile csd3 option makes snakemake look at the csd3/config.yaml file and takes resource specification from there. This is set to use SLURM and sets default resources for each rule of the workflow.

    • Make sure to edit this config file if using a different CSD3 account.

  • On Cambridge CSD3 we recommend running this from a tmux/screen session on a login node. Snakemake should then submit jobs to SLURM automatically.

For a dry-run (i.e. indicate which steps would be run without actually doing anything) do:

snakemake -np

Adding a new genome

To add a new genome, edit the genomes.csv file, adding a new row with the name and links for the new genome. Re-run the workflow as above, which should run only the steps necessary to index the new genome.

Always do a dry-run first, to check everything looks right (snakemake -np)

Using the genomes in your analysis

Within each genome directory, the following files can be found:

  • genome.fa.gz: compressed reference genome in FASTA format.

  • genome.fai: samtools index file for the FASTA file.

  • annotation.gtf.gz: compressed reference annotation in GTF format.

  • annotation.gff3.gz: compressed reference annotation in GFF3 format.

And the following directories, containing index files for the respective software tools:

  • bowtie2

  • bwa

  • bwa-mem2

  • hisat2

  • salmon

  • star

Linting and formatting

Linting results
  1Lints for snakefile /tmp/tmp8rg5jra5/workflow/Snakefile:
  2    * Deprecated singularity directive used for container definition in line 12.:
  3      Use the container directive instead (it is agnostic of the underlying
  4      container runtime).
  5      Also see:
  6      https://snakemake.readthedocs.io/en/latest/snakefiles/deployment.html#running-jobs-in-containers
  7    * Deprecated singularity directive used for container definition in line 13.:
  8      Use the container directive instead (it is agnostic of the underlying
  9      container runtime).
 10      Also see:
 11      https://snakemake.readthedocs.io/en/latest/snakefiles/deployment.html#running-jobs-in-containers
 12
 13Lints for snakefile /tmp/tmp8rg5jra5/workflow/rules/common.smk:
 14    * Absolute path "/nfcore_genomes.config" in line 9:
 15      Do not define absolute paths inside of the workflow, since this renders
 16      your workflow irreproducible on other machines. Use path relative to the
 17      working directory instead, or make the path configurable via a config
 18      file.
 19      Also see:
 20      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 21
 22Lints for snakefile /tmp/tmp8rg5jra5/workflow/rules/download.smk:
 23    * Absolute path "/{genome}/genome.fa" in line 3:
 24      Do not define absolute paths inside of the workflow, since this renders
 25      your workflow irreproducible on other machines. Use path relative to the
 26      working directory instead, or make the path configurable via a config
 27      file.
 28      Also see:
 29      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 30    * Absolute path "/{genome}/annotation.gff3" in line 4:
 31      Do not define absolute paths inside of the workflow, since this renders
 32      your workflow irreproducible on other machines. Use path relative to the
 33      working directory instead, or make the path configurable via a config
 34      file.
 35      Also see:
 36      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 37    * Absolute path "/{genome}/annotation.gtf" in line 5:
 38      Do not define absolute paths inside of the workflow, since this renders
 39      your workflow irreproducible on other machines. Use path relative to the
 40      working directory instead, or make the path configurable via a config
 41      file.
 42      Also see:
 43      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 44    * Absolute path "/{genome}/logs/download.log" in line 7:
 45      Do not define absolute paths inside of the workflow, since this renders
 46      your workflow irreproducible on other machines. Use path relative to the
 47      working directory instead, or make the path configurable via a config
 48      file.
 49      Also see:
 50      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 51    * Path composition with '+' in line 3:
 52      This becomes quickly unreadable. Usually, it is better to endure some
 53      redundancy against having a more readable workflow. Hence, just repeat
 54      common prefixes. If path composition is unavoidable, use pathlib or
 55      (python >= 3.6) string formatting with f"...".
 56    * Path composition with '+' in line 4:
 57      This becomes quickly unreadable. Usually, it is better to endure some
 58      redundancy against having a more readable workflow. Hence, just repeat
 59      common prefixes. If path composition is unavoidable, use pathlib or
 60      (python >= 3.6) string formatting with f"...".
 61    * Path composition with '+' in line 5:
 62      This becomes quickly unreadable. Usually, it is better to endure some
 63      redundancy against having a more readable workflow. Hence, just repeat
 64      common prefixes. If path composition is unavoidable, use pathlib or
 65      (python >= 3.6) string formatting with f"...".
 66    * Path composition with '+' in line 7:
 67      This becomes quickly unreadable. Usually, it is better to endure some
 68      redundancy against having a more readable workflow. Hence, just repeat
 69      common prefixes. If path composition is unavoidable, use pathlib or
 70      (python >= 3.6) string formatting with f"...".
 71
 72Lints for snakefile /tmp/tmp8rg5jra5/workflow/rules/cellranger.smk:
 73    * Deprecated singularity directive used for container definition in line 9.:
 74      Use the container directive instead (it is agnostic of the underlying
 75      container runtime).
 76      Also see:
 77      https://snakemake.readthedocs.io/en/latest/snakefiles/deployment.html#running-jobs-in-containers
 78
 79Lints for snakefile /tmp/tmp8rg5jra5/workflow/rules/nfcore_config.smk:
 80    * Absolute path "/{genome}/genome.fa" in line 4:
 81      Do not define absolute paths inside of the workflow, since this renders
 82      your workflow irreproducible on other machines. Use path relative to the
 83      working directory instead, or make the path configurable via a config
 84      file.
 85      Also see:
 86      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 87    * Absolute path "/{genome}/genome.fa.fai" in line 5:
 88      Do not define absolute paths inside of the workflow, since this renders
 89      your workflow irreproducible on other machines. Use path relative to the
 90      working directory instead, or make the path configurable via a config
 91      file.
 92      Also see:
 93      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
 94    * Absolute path "/{genome}/genome.dict" in line 6:
 95      Do not define absolute paths inside of the workflow, since this renders
 96      your workflow irreproducible on other machines. Use path relative to the
 97      working directory instead, or make the path configurable via a config
 98      file.
 99      Also see:
100      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
101    * Absolute path "/{genome}/annotation.gtf" in line 7:
102      Do not define absolute paths inside of the workflow, since this renders
103      your workflow irreproducible on other machines. Use path relative to the
104      working directory instead, or make the path configurable via a config
105      file.
106      Also see:
107      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
108    * Absolute path "/{genome}/annotation.gff3" in line 8:
109      Do not define absolute paths inside of the workflow, since this renders
110      your workflow irreproducible on other machines. Use path relative to the
111      working directory instead, or make the path configurable via a config
112      file.
113      Also see:
114      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
115    * Absolute path "/{genome}/transcripts.fa" in line 9:
116      Do not define absolute paths inside of the workflow, since this renders
117      your workflow irreproducible on other machines. Use path relative to the
118      working directory instead, or make the path configurable via a config
119      file.
120      Also see:
121      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
122    * Absolute path "/{genome}/tx2gene.csv" in line 10:
123      Do not define absolute paths inside of the workflow, since this renders
124      your workflow irreproducible on other machines. Use path relative to the
125      working directory instead, or make the path configurable via a config
126      file.
127      Also see:
128      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
129    * Absolute path "/{genome}/star" in line 11:
130      Do not define absolute paths inside of the workflow, since this renders
131      your workflow irreproducible on other machines. Use path relative to the
132      working directory instead, or make the path configurable via a config
133      file.
134      Also see:
135      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
136    * Absolute path "/{genome}/minimap2/genome.mmi" in line 12:
137      Do not define absolute paths inside of the workflow, since this renders
138      your workflow irreproducible on other machines. Use path relative to the
139      working directory instead, or make the path configurable via a config
140      file.
141      Also see:
142      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
143    * Absolute path "/{genome}/hisat2/genome.1.ht2l" in line 14:
144      Do not define absolute paths inside of the workflow, since this renders
145      your workflow irreproducible on other machines. Use path relative to the
146      working directory instead, or make the path configurable via a config
147      file.
148      Also see:
149      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
150    * Absolute path "/{genome}/bowtie2/genome.1.bt2" in line 15:
151      Do not define absolute paths inside of the workflow, since this renders
152      your workflow irreproducible on other machines. Use path relative to the
153      working directory instead, or make the path configurable via a config
154      file.
155      Also see:
156      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
157    * Absolute path "/{genome}/bwa-mem/genome.amb" in line 16:
158      Do not define absolute paths inside of the workflow, since this renders
159      your workflow irreproducible on other machines. Use path relative to the
160      working directory instead, or make the path configurable via a config
161      file.
162      Also see:
163      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
164    * Absolute path "/{genome}/bwa-mem2/genome.0123" in line 17:
165      Do not define absolute paths inside of the workflow, since this renders
166      your workflow irreproducible on other machines. Use path relative to the
167      working directory instead, or make the path configurable via a config
168      file.
169      Also see:
170      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
171    * Absolute path "/{genome}/salmon/info.json" in line 18:
172      Do not define absolute paths inside of the workflow, since this renders
173      your workflow irreproducible on other machines. Use path relative to the
174      working directory instead, or make the path configurable via a config
175      file.
176      Also see:
177      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
178    * Absolute path "/{genome}/cellranger" in line 19:
179      Do not define absolute paths inside of the workflow, since this renders
180      your workflow irreproducible on other machines. Use path relative to the
181      working directory instead, or make the path configurable via a config
182      file.
183      Also see:
184      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
185    * Absolute path "/nfcore_genomes.config" in line 21:
186      Do not define absolute paths inside of the workflow, since this renders
187      your workflow irreproducible on other machines. Use path relative to the
188      working directory instead, or make the path configurable via a config
189      file.
190      Also see:
191      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
192    * Absolute path "/*\n" in line 30:
193      Do not define absolute paths inside of the workflow, since this renders
194      your workflow irreproducible on other machines. Use path relative to the
195      working directory instead, or make the path configurable via a config
196      file.
197      Also see:
198      https://snakemake.readthedocs.io/en/latest/snakefiles/configuration.html#configuration
199    * Path composition with '+' in line 4:
200      This becomes quickly unreadable. Usually, it is better to endure some
201
202... (truncated)
Formatting results
 1[DEBUG] 
 2[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/Snakefile":  Formatted content is different from original
 3[DEBUG] 
 4[DEBUG] 
 5[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/gffread.smk":  Formatted content is different from original
 6[DEBUG] 
 7[DEBUG] 
 8[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/star.smk":  Formatted content is different from original
 9[DEBUG] 
10[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/uncompress.smk":  Formatted content is different from original
11[DEBUG] 
12[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/samtools.smk":  Formatted content is different from original
13[DEBUG] 
14[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/hisat2.smk":  Formatted content is different from original
15[DEBUG] 
16[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/salmon.smk":  Formatted content is different from original
17[DEBUG] 
18[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/nfcore_config.smk":  Formatted content is different from original
19[DEBUG] 
20[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/bwa.smk":  Formatted content is different from original
21[DEBUG] 
22[DEBUG] 
23[DEBUG] 
24[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/minimap2.smk":  Formatted content is different from original
25[DEBUG] 
26[DEBUG] In file "/tmp/tmp8rg5jra5/workflow/rules/common.smk":  Formatted content is different from original
27[INFO] 11 file(s) would be changed 😬
28[INFO] 4 file(s) would be left unchanged 🎉
29
30snakefmt version: 0.11.5