Snakemake report plugin: rocrate

This report plugin packages a finished Snakemake workflow run as a Provenance Run Crate, the most detailed profile of the Workflow Run RO-Crate family. The crate records which rules ran, when, with which input and output files and with which software, in a machine-readable RO-Crate that can be archived, shared or deposited alongside the results.

The plugin writes either a ZIP archive containing the workflow files, the data and ro-crate-metadata.json, or only ro-crate-metadata.json next to the workflow files. The crate is validated with roc-validator before the report completes.

Installation

Install this plugin by installing it with pip or mamba directly, e.g.:

pip install snakemake-report-plugin-rocrate

Or, if you are using pixi, add the plugin to your pixi.toml. Be careful to put it under the right dependency type based on the plugin’s availability, e.g.:

snakemake-report-plugin-rocrate = "*"

Usage

In order to use the plugin, run Snakemake (>=8.5) with the corresponding value for the reporter flag:

snakemake --reporter rocrate ...

with ... being any additional arguments you want to use.

Settings

The report plugin has the following settings (which can be passed via command line, the workflow or environment variables, if provided in the respective columns):

Settings

CLI argument	Description	Default	Choices	Required	Type
--report-rocrate-filename VALUE	Output filename stem; the .zip suffix is added automatically. Ignored in metadata-only mode.	''		✗
--report-rocrate-metadata-only VALUE	Write only the ro-crate-metadata.json descriptor next to the workflow outputs (the current working directory) instead of packaging the crate as a ZIP archive. No file is copied, so the recorded relative paths resolve against the real workflow files.	False		✗
--report-rocrate-run-name VALUE	Descriptive name for the workflow run.	'Snakemake Provenance Run'		✗
--report-rocrate-run-description VALUE	Description of the workflow run and its context.	'RO-Crate describing a Snakemake workflow run.'		✗
--report-rocrate-run-license VALUE	SPDX license identifier or license URL for the crate.	'CC-BY-4.0'		✗
--report-rocrate-main-tool VALUE	Optional preferred package for rule tool wrappers. When omitted, discovered environment tools are direct CreateAction instruments.	''		✗
--report-rocrate-validation-severity VALUE	Lowest RO-Crate validation level to inspect; only REQUIRED issues fail the report.	'REQUIRED'	REQUIRED, RECOMMENDED, OPTIONAL	✗
--report-rocrate-researcher-orcid VALUE	ORCID iD of the person executing the workflow.	''		✗
--report-rocrate-researcher-name VALUE	Full name of the person executing the workflow.	''		✗
--report-rocrate-agent-orcid VALUE	ORCID URL of the person responsible for the workflow run.	''		✗
--report-rocrate-agent-name VALUE	Full name of the person responsible for the workflow run.	''		✗
--report-rocrate-organization-ror VALUE	ROR identifier for the researcher’s organization.	''		✗
--report-rocrate-organization-name VALUE	Full name of the researcher’s organization.	''		✗
--report-rocrate-organization-url VALUE	Website URL of the researcher’s organization.	''		✗

Further details

What is recorded

The crate uses RO-Crate 1.1 and conforms to Provenance Run Crate 0.5, Workflow Run Crate 0.5, Process Run Crate 0.5 and Workflow RO-Crate 1.0. It contains:

• the main Snakefile as the crate’s main workflow, with one HowToStep per rule;

• one CreateAction per executed job, with its start and end time, its input files (object), output files (result) and shell command;

• a CreateAction for the workflow run as a whole, and an OrganizeAction for Snakemake orchestrating it, linked to the jobs through ControlActions;

• the input and output files, including intermediate files. Files outside the working directory are copied under _EXTERNAL/ in the ZIP;

• the Conda environment files of the jobs, and the packages installed in each job’s environment with their versions.

External inputs (files consumed by a job but not produced by another job) are recorded as the workflow run’s inputs; final outputs as its outputs.

Run the report after the workflow

Generate the report from the workflow directory after the run, with the same targets and software deployment method:

snakemake --cores 1 --software-deployment-method conda
snakemake --cores 1 --software-deployment-method conda --reporter rocrate

Use --software-deployment-method conda for the report as well, so that the plugin can find each job’s Conda environment and read the installed package versions from its conda-meta records. Without it, no package versions are recorded. Packages installed with pip only are not listed in conda-meta and are therefore not recorded.

The Snakemake version recorded is the one used to generate the report, which may differ from the version that ran the jobs.

Setting options in a profile

The options are easiest to keep in a Snakemake profile, for example profiles/rocrate/config.yaml:

report-rocrate-filename: "workflow-run"
report-rocrate-run-name: "Linear elastic plate with a hole"
report-rocrate-run-license: "CC-BY-4.0"
report-rocrate-main-tool: "fenics-dolfinx"
report-rocrate-validation-severity: "REQUIRED"
report-rocrate-agent-orcid: "https://orcid.org/0009-0008-6162-8404"
report-rocrate-agent-name: "Mahdi Jafarkhani"
report-rocrate-organization-ror: "https://ror.org/04vnq7t77"
report-rocrate-organization-name: "University of Stuttgart"
report-rocrate-organization-url: "https://www.uni-stuttgart.de/en/"

and then run:

snakemake --software-deployment-method conda --reporter rocrate --profile profiles/rocrate --cores 1

People and organizations

Two kinds of person can be recorded, and both are optional:

• the researcher (researcher-name, researcher-orcid) is recorded as the crate’s author. With organization-* options, the researcher is affiliated with that organization, and the organization is the crate’s publisher;

• the agent (agent-name together with agent-orcid, which must be an HTTP(S) ORCID URL) is the person who ran the workflow. It is recorded as the agent of the workflow run, of each job and of the orchestration action.

The plugin does not record any person or organization unless these options are given.

Main tool

Each executed rule is represented by a tool entity that is the instrument of its jobs. With main-tool, the named package (if it is installed in a job’s environment) is listed first among that tool’s softwareRequirements. Without main-tool, the packages discovered in each job’s environment are used directly as the job’s instruments. If packages were discovered but none has the name given in main-tool, the report fails and lists the available packages.

Metadata-only mode

With metadata-only, the plugin writes only ro-crate-metadata.json into the working directory, next to the Snakefile and the workflow outputs, instead of writing a ZIP. No files are copied: the identifiers in the metadata are paths relative to the working directory. filename is ignored in this mode.

Validation

The crate is validated against the Provenance Run Crate 0.5 rules of roc-validator. validation-severity sets the lowest level to inspect:

• a failed REQUIRED check always makes the report fail;

• failed RECOMMENDED or OPTIONAL checks are logged as warnings;

• checks that could not be evaluated are logged separately, and an unevaluated REQUIRED check also makes the report fail.

Files with several parts

The plugin packages the input and output files that Snakemake declares, and does not inspect file contents. If a rule produces a dataset made of several files (for example a mesh with sidecar files), declare each file as an output so that all of them are included.

Examples

The repository contains two example workflows, a linear-elastic plate with a hole (FEniCS/DOLFINx) and a Poisson equation solved and written up as a PDF. Each comes with profiles/rocrate and profiles/rocrate-metadata-only configurations.