IMS-Bio2Core-Facility/GTExSnake

A fully concurrent pipeline for querying transcript-level GTEx data in specific tissues

Overview

Topics: bioinformatics snakemake reproducible-science conda singularity multithreading pipeline

Latest release: v1.2.0, Last update: 2021-10-28

Linting: linting: passed, Formatting:formatting: failed

Deployment

Step 1: Install Snakemake and Snakedeploy

Snakemake and Snakedeploy are best installed via the Mamba package manager (a drop-in replacement for conda). If you have neither Conda nor Mamba, it is recommended to install Miniforge. More details regarding Mamba can be found here.

When using Mamba, run

mamba create -c conda-forge -c bioconda --name snakemake snakemake snakedeploy

to install both Snakemake and Snakedeploy in an isolated environment. For all following commands ensure that this environment is activated via

conda activate snakemake

Step 2: Deploy workflow

With Snakemake and Snakedeploy installed, the workflow can be deployed as follows. First, create an appropriate project working directory on your system and enter it:

mkdir -p path/to/project-workdir
cd path/to/project-workdir

In all following steps, we will assume that you are inside of that directory. Then run

snakedeploy deploy-workflow https://github.com/IMS-Bio2Core-Facility/GTExSnake . --tag v1.2.0

Snakedeploy will create two folders, workflow and config. The former contains the deployment of the chosen workflow as a Snakemake module, the latter contains configuration files which will be modified in the next step in order to configure the workflow to your needs.

Step 3: Configure workflow

To configure the workflow, adapt config/config.yml to your needs following the instructions below.

Step 4: Run workflow

The deployment method is controlled using the --software-deployment-method (short --sdm) argument.

To run the workflow with automatic deployment of all required software via conda/mamba, use

snakemake --cores all --sdm conda

To run the workflow using apptainer/singularity, use

snakemake --cores all --sdm apptainer

To run the workflow using a combination of conda and apptainer/singularity for software deployment, use

snakemake --cores all --sdm conda apptainer

Snakemake will automatically detect the main Snakefile in the workflow subfolder and execute the workflow module that has been defined by the deployment in step 2.

For further options such as cluster and cloud execution, see the docs.

Step 5: Generate report

After finalizing your data analysis, you can automatically generate an interactive visual HTML report for inspection of results together with parameters and code inside of the browser using

snakemake --report report.zip

Configuration

The following section is imported from the workflow’s config/README.md.

Configuration

⚠️ Re-running a snakemake workflow If you want to run this pipeline multpile times, then be sure to move or delete the final results/process/sorted_isoforms.xlsx file. Snakemake will not re-run a pipeline if it detects the output file.

There are currently five (5) points where the user can customise the pipeline:

Number of Threads

Unless you are querying a huge number of genes, I find 6 cores to be sufficient to keep things moving. If you want to use more or less, just change the value passed to --cores. Each rule that implements concurrency will then use this many. Remember, more isn't always faster. On my laptop, setting cores greater than the number of physical cores (not threads!) in my machine gets me no improvement. YMMV

Genes of Interest

You will almost certainly have a different gene list than me! If its short, you can specify it as a parameter like so:

snakemake --use-conda --use-singularity --cores 6 --config gene_ids=\["GENE1","GENE2"\]

Alternatively, you can specify it as a list under the gene_ids parameter in the configuration file located at configuration/snakemake.yaml.

Region of Interest

The same goes for your region of interest. It can be specified as the region parameter in configuration/snakemake.yaml, or passed as below:

snakemake --use-conda --use-singularity --cores 6 --config region="Brain_Hypothalamus"

⚠️ Accepted Regions GTEx is picky about formatting, so be sure your region is from their approved list. You can find that list here under tissueSiteDetailId.

Gencode URL

This is the URL to query for the Gencode reference. We recommend not changing this unless you have a very strong reason. Things start breaking rapidly if you use different refernces than GTEx.

MANE URL

This is the URL to query for the MANE. It should be up-to-date. If we are late on an update, or you would like to use either an older or new version, change this.

Linting and formatting

Linting results

None

Formatting results

None
snakefmt version: 0.4.4