IMS-Bio2Core-Facility/GTExSnake
A fully concurrent pipeline for querying transcript-level GTEx data in specific tissues
Overview
Topics: bioinformatics snakemake reproducible-science conda singularity multithreading pipeline
Latest release: v1.2.0, Last update: 2021-10-28
Linting: linting: passed, Formatting:formatting: failed
Deployment
Step 1: Install Snakemake and Snakedeploy
Snakemake and Snakedeploy are best installed via the Mamba package manager (a drop-in replacement for conda). If you have neither Conda nor Mamba, it is recommended to install Miniforge. More details regarding Mamba can be found here.
When using Mamba, run
mamba create -c conda-forge -c bioconda --name snakemake snakemake snakedeploy
to install both Snakemake and Snakedeploy in an isolated environment. For all following commands ensure that this environment is activated via
conda activate snakemake
Step 2: Deploy workflow
With Snakemake and Snakedeploy installed, the workflow can be deployed as follows. First, create an appropriate project working directory on your system and enter it:
mkdir -p path/to/project-workdir
cd path/to/project-workdir
In all following steps, we will assume that you are inside of that directory. Then run
snakedeploy deploy-workflow https://github.com/IMS-Bio2Core-Facility/GTExSnake . --tag v1.2.0
Snakedeploy will create two folders, workflow
and config
. The former contains the deployment of the chosen workflow as a Snakemake module, the latter contains configuration files which will be modified in the next step in order to configure the workflow to your needs.
Step 3: Configure workflow
To configure the workflow, adapt config/config.yml
to your needs following the instructions below.
Step 4: Run workflow
The deployment method is controlled using the --software-deployment-method
(short --sdm
) argument.
To run the workflow with automatic deployment of all required software via conda
/mamba
, use
snakemake --cores all --sdm conda
To run the workflow using apptainer
/singularity
, use
snakemake --cores all --sdm apptainer
To run the workflow using a combination of conda
and apptainer
/singularity
for software deployment, use
snakemake --cores all --sdm conda apptainer
Snakemake will automatically detect the main Snakefile
in the workflow
subfolder and execute the workflow module that has been defined by the deployment in step 2.
For further options such as cluster and cloud execution, see the docs.
Step 5: Generate report
After finalizing your data analysis, you can automatically generate an interactive visual HTML report for inspection of results together with parameters and code inside of the browser using
snakemake --report report.zip
Configuration
The following section is imported from the workflow’s config/README.md
.
Configuration
⚠️ Re-running a snakemake workflow If you want to run this pipeline multpile times, then be sure to move or delete the finalresults/process/sorted_isoforms.xlsx
file. Snakemake will not re-run a pipeline if it detects the output file.
There are currently five (5) points where the user can customise the pipeline:
Number of Threads
Unless you are querying a huge number of genes,
I find 6 cores to be sufficient to keep things moving.
If you want to use more or less,
just change the value passed to --cores
.
Each rule that implements concurrency will then use this many.
Remember, more isn't always faster.
On my laptop,
setting cores greater than the number of physical cores (not threads!)
in my machine gets me no improvement.
YMMV
Genes of Interest
You will almost certainly have a different gene list than me! If its short, you can specify it as a parameter like so:
snakemake --use-conda --use-singularity --cores 6 --config gene_ids=\["GENE1","GENE2"\]
Alternatively, you can specify it as a list under the gene_ids
parameter in the
configuration file located at configuration/snakemake.yaml
.
Region of Interest
The same goes for your region of interest.
It can be specified as the region
parameter in configuration/snakemake.yaml
,
or passed as below:
snakemake --use-conda --use-singularity --cores 6 --config region="Brain_Hypothalamus"
⚠️ Accepted Regions GTEx is picky about formatting, so be sure your region is from their approved list. You can find that list here undertissueSiteDetailId
.
Gencode URL
This is the URL to query for the Gencode reference. We recommend not changing this unless you have a very strong reason. Things start breaking rapidly if you use different refernces than GTEx.
MANE URL
This is the URL to query for the MANE. It should be up-to-date. If we are late on an update, or you would like to use either an older or new version, change this.
Linting and formatting
Linting results
None
Formatting results
None
snakefmt version: 0.4.4