Snakemake report plugin: fairscape

Warning

This plugin is not maintained and reviewed by the official Snakemake organization.

This plugin turns a completed Snakemake run into a FAIRSCAPE EVI RO-Crate: an ro-crate-metadata.json describing the run as a provenance graph — one Computation per executed job (with its resolved shell command, parameters, inputs and outputs), one Software per rule plus the Snakefile and the Snakemake engine itself, and one Dataset per file the workflow read or wrote. Alongside the crate it generates the FAIRSCAPE artifact set: a human-readable datasheet, an AI-Ready score, an interactive evidence graph, a LinkML/D4D export, and — on request — an inferred schema for every tabular or array data file.

Nothing in your workflow has to change, and no file needs to be flagged with report(...): the plugin reads Snakemake’s own per-job persistence metadata plus the report interface’s job records, so every declared output of every executed job is described, intermediates included.

The report step is post-hoc — it runs as a separate snakemake --reporter fairscape invocation against .snakemake/metadata, so it adds no runtime overhead, cannot fail a run, and works on runs that finished before the plugin was installed.

Identifiers are deterministic ARKs hashed from run-independent strings (Snakefile path, rule name, rule plus sorted outputs, file path), so re-generating the report — or re-executing the whole workflow with --forceall — reproduces byte-identical identifiers.

Install the plugin together with its artifact generator:

pip install 'snakemake-report-plugin-fairscape[artifacts]'

The [artifacts] extra pulls in fairscape-cli, which produces everything downstream of the crate; those steps need fairscape-cli 1.2.10 or newer. Without it the crate itself is still written and they are skipped with a note.

Installation

Install this plugin by installing it with pip or mamba directly, e.g.:

pip install snakemake-report-plugin-fairscape

Or, if you are using pixi, add the plugin to your pixi.toml. Be careful to put it under the right dependency type based on the plugin’s availability, e.g.:

snakemake-report-plugin-fairscape = "*"

Usage

In order to use the plugin, run Snakemake (>=8.5) with the corresponding value for the reporter flag:

snakemake --reporter fairscape ...

with ... being any additional arguments you want to use.

Settings

The report plugin has the following settings (which can be passed via command line, the workflow or environment variables, if provided in the respective columns):

Settings

CLI argument	Description	Default	Choices	Required	Type
--report-fairscape-path VALUE	Path of the RO-Crate metadata file to write (default: ro-crate-metadata.json in the working directory).	None		✗
--report-fairscape-naan VALUE	ARK Name Assigning Authority Number used when minting identifiers.	'59853'		✗
--report-fairscape-name VALUE	Name of the crate (default: derived from the Snakefile).	None		✗
--report-fairscape-description VALUE	Description of the crate (min 10 characters).	None		✗
--report-fairscape-author VALUE	Author recorded on the crate and its entities (default: the current user name).	None		✗
--report-fairscape-keywords VALUE	Comma-separated keywords (default: snakemake,workflow).	None		✗
--report-fairscape-license VALUE	License URL recorded on the crate root.	'https://spdx.org/licenses/CC-BY-4.0'		✗
--report-fairscape-version VALUE	Version recorded on the crate root.	'1.0'		✗
--report-fairscape-expand-directories VALUE	Walk each directory output and register one Dataset per file inside it (isPartOf the directory, generatedBy its producer). Off by default: costs one walk per directory output.	False		✗
--report-fairscape-expand-max-files VALUE	Cap on files registered per expanded directory output (the walk is sorted, so the cap is deterministic).	1000		✗
--report-fairscape-schemas VALUE	Infer an EVI Schema for every described data file with a supported extension (csv/tsv/parquet/h5/hdf5/hea/dcm) and link it from its Dataset. Off by default: reads the data files.	False		✗
--report-fairscape-no-datasheet VALUE	Skip generating ro-crate-datasheet.html and ai_ready_score.json (on by default; reads the crate JSON only).	False		✗
--report-fairscape-no-evidence-graph VALUE	Skip generating ro-crate-prov-graph.json/.html (on by default; reads the crate JSON only).	False		✗
--report-fairscape-no-linkml VALUE	Skip generating the D4D/LinkML export ro-crate-linkml.yaml (on by default; reads the crate JSON only).	False		✗
--report-fairscape-no-link-inverses VALUE	Skip completing the crate against EVI’s owl:inverseOf pairs (on by default; rewrites the crate JSON in place).	False		✗
--report-fairscape-preview VALUE	Also generate ro-crate-preview.html.	False		✗
--report-fairscape-croissant VALUE	Also generate a Croissant JSON-LD export.	False		✗
--report-fairscape-merkle VALUE	Also generate a SHA-256 Merkle tree over the crate’s files (reads every file).	False		✗

Further details

Usage

Run the workflow as usual, then ask for the report:

snakemake --cores 4
snakemake --reporter fairscape

That writes ro-crate-metadata.json into the working directory, followed by the derived artifacts:

• ro-crate-datasheet.html and ai_ready_score.json — human-readable datasheet and AI-Ready score

• ro-crate-prov-graph.json / .html — the evidence graph and an interactive visualization of it

• ro-crate-linkml.yaml — the LinkML/D4D (Datasheets for Datasets) export

Those three are on by default and read only the crate JSON. They are produced by fairscape-cli 1.2.10 or newer; with an older fairscape-cli — or none at all — the crate is still written and the steps after it are skipped with a note.

Everything that touches the data files themselves is opt-in:

# infer an EVI Schema per data file (csv/tsv/parquet/h5/hdf5/hea/dcm)
snakemake --reporter fairscape --report-fairscape-schemas

# describe every file inside a directory() output, not just the directory
snakemake --reporter fairscape --report-fairscape-expand-directories

# SHA-256 Merkle tree over the crate's files
snakemake --reporter fairscape --report-fairscape-merkle

Crate-level metadata is worth setting once you publish a crate:

snakemake --reporter fairscape \
  --report-fairscape-name "RNA-seq differential expression" \
  --report-fairscape-description "Salmon quantification and DESeq2 analysis of the pilot cohort" \
  --report-fairscape-author "Jane Doe" \
  --report-fairscape-keywords "rna-seq,deseq2,pilot" \
  --report-fairscape-license "https://spdx.org/licenses/CC-BY-4.0"

What ends up in the crate

• A run Computation for the workflow as a whole: usedDataset is the set of root inputs no job produced, generated the set of terminal outputs no job consumed, and its start/end times are the earliest and latest job times.

• One Computation per executed job, carrying the resolved shell command, the rule’s params, its inputs (from the persistence metadata) and its declared outputs, with wildcards resolved into the job’s name.

• Software entities for the Snakefile, the Snakemake engine (versioned), and every rule. A rule’s Software points at the file that actually defines its action: the .py/.R file for a script: rule, the notebook for notebook:, the resolved wrapper URL for wrapper:, and the Snakefile for shell:/run: rules. Container images and conda environments are recorded when declared.

• One Dataset per unique file — inputs, outputs, intermediates and configfiles. Files inside the crate directory get a crate-relative contentUrl; files outside it, or already deleted (temp() intermediates), get a localPath instead, since a relative contentUrl is a promise the bytes are there.

Both directions of every provenance edge are written (generated and generatedBy), so the graph can be walked from either end.

Which files are captured

Every declared output of every executed job, plus every input recorded in the persistence metadata, plus configfiles. Nothing needs to be marked with report(...) — that channel only feeds Snakemake’s own HTML reporter, and this plugin ignores it in favour of the full job list.

Not captured: files a rule writes without declaring them in output: (Snakemake has no record of them either), benchmark: files, and jobs that failed or never ran — a failed job leaves no persistence metadata, so only successful work is described.

Caveats of the post-hoc design

The reporter reconstructs the run from .snakemake/metadata after the fact. That is what makes it free of runtime risk, but it also means:

• the run Computation’s command is literally snakemake — the original command line is not part of the metadata the report step can see;

• re-running the same workflow overwrites the persistence metadata, so the crate always describes the most recent run;

• temp() intermediates are described with full provenance edges but without contentSize, since the bytes are gone by report time;

• the Snakefile must still parse at report time.

Relationship to the rest of FAIRSCAPE

This package owns the Snakemake side only: it reduces the report interface, the persistence metadata and the filesystem state to a plain-data records document. The Snakefile-to-EVI mapping and the ARK minting live in fairscape_conversion (plugins/snakemake), shared with the Nextflow, Cromwell, MLflow, WRROC and C2M2 converters; the datasheet, evidence graph, LinkML export and schema inference are fairscape-cli‘s own functions, called directly rather than reimplemented.

Setting SNAKEMAKE_FAIRSCAPE_DUMP_RECORDS=<path> on a report run dumps the records document that crosses that boundary, which is the fastest way to see what the plugin extracted.

Full documentation, including the exact provenance semantics and the list of undeclared Snakemake interfaces the plugin depends on, is in the repository README.