Snakemake executor plugin: vastai

Warning

This plugin is not maintained and reviewed by the official Snakemake organization.

Warning

No documentation found in repository https://github.com/bards-ai/snakemake-executor-plugin-vastai. The plugin should provide a docs/intro.md with some introductory sentences and optionally a docs/further.md file with details beyond the auto-generated usage instructions presented in this catalog.

Installation

Install this plugin by installing it with pip or mamba directly, e.g.:

pip install snakemake-executor-plugin-vastai

Or, if you are using pixi, add the plugin to your pixi.toml. Be careful to put it under the right dependency type based on the plugin’s availability, e.g.:

snakemake-executor-plugin-vastai = "*"

Usage

In order to use the plugin, run Snakemake (>=8.6) in the folder where your workflow code and config resides (containing either workflow/Snakefile or Snakefile) with the corresponding value for the executor flag:

snakemake --executor vastai --default-resources --jobs N ...

with N being the number of jobs you want to run in parallel and ... being any additional arguments you want to use (see below). The machine on which you run Snakemake must have the executor plugin installed, and, depending on the type of the executor plugin, have access to the target service of the executor plugin (e.g. an HPC middleware like slurm with the sbatch command, or internet access to submit jobs to some cloud provider, e.g. azure).

The flag --default-resources ensures that Snakemake auto-calculates the mem and disk resources for each job, based on the input file size. The values assumed there are conservative and should usually suffice. However, you can always override those defaults by specifying the resources in your Snakemake rules or via the --set-resources flag.

Depending on the executor plugin, you might either rely on a shared local filesystem or use a remote filesystem or storage. For the latter, you have to additionally use a suitable storage plugin (see section storage plugins in the sidebar of this catalog) and eventually check for further recommendations in the sections below.

All arguments can also be persisted via a profile, such that they don’t have to be specified on each invocation. Here, this would mean the following entries inside of the profile

executor: vastai
default_resources: []

For specifying other default resources than the built-in ones, see the docs.

Settings

The executor plugin has the following settings (which can be passed via command line, the workflow or environment variables, if provided in the respective columns):

Settings

CLI argument	Description	Default	Choices	Required	Type
--vastai-api-key VALUE	Vast.ai API key. If not set, it is resolved like the vastai CLI does: from the VAST_API_KEY environment variable or from ~/.config/vastai/vast_api_key.	None		✗
--vastai-gpu-name VALUE	Default GPU model to request, in Vast.ai naming with underscores for spaces (e.g. RTX_4090, H100_SXM, A100_SXM4). Can be overridden per job with the gpu_model resource. If unset, any GPU matching the other constraints is used.	None		✗
--vastai-image VALUE	Docker image to run jobs in. Defaults to vastai/pytorch:@vastai-automatic-tag (CUDA+PyTorch matched to the machine, OpenSSH included); an explicitly set –container-image takes precedence over that default. Snakemake is pip-installed into the container automatically if the image does not provide it; bake pip install snakemake into your own image to skip that startup cost.	None		✗
--vastai-disk VALUE	Disk space to allocate per instance in GB. Can be raised per job with the disk_mb resource.	40.0		✗
--vastai-max-price VALUE	Maximum on-demand price per instance in $/h (dph_total, including storage). Offers above this price are never rented.	None		✗
--vastai-reliability VALUE	Minimum host reliability score (0-1) required for offers. Set to 0 to disable the filter.	0.98		✗
--vastai-order VALUE	Sort order for choosing among matching offers (vastai search order syntax; append ‘-’ for descending). The default rents the cheapest matching offer. Use e.g. ‘dlperf_usd-’ for best DL performance per dollar.	'dph_total'		✗
--vastai-geolocation VALUE	Restrict offers to a geographic area: a region shortcut (EU, NA, AS, AF, LC, OC) or comma-separated ISO country codes (e.g. ‘PL,DE,CZ’).	None		✗
--vastai-no-datacenter VALUE	Also rent from non-datacenter (hobbyist) hosts. By default only verified datacenter hosts are used — they are slightly pricier but avoid the most common marketplace flakiness (Docker Hub pull rate limits, slow residential uplinks). Disabling the filter gives access to the cheapest offers.	False		✗
--vastai-search-query VALUE	Additional filters appended to every offer search, in vastai query syntax (e.g. ‘cuda_vers>=12.4 geolocation=EU inet_down>=500’). Can be extended per job with the vastai_query resource.	None		✗
--vastai-boot-timeout VALUE	Maximum seconds to wait for an instance to reach the running state (includes docker image pull) before the job is failed and the instance destroyed.	1800		✗
--vastai-no-forward-credentials VALUE	Do not forward cloud storage credentials (AWS_*, AZURE_STORAGE_*, GOOGLE_* including the application credentials file content) from the local environment into the job containers. By default they are forwarded so that the default storage provider works remotely without declaring –envvars. Credentials configured via the storage plugin’s own settings (e.g. –storage-s3-access-key or SNAKEMAKE_STORAGE_S3_ACCESS_KEY) are always forwarded by Snakemake itself, independent of this option.	False		✗
--vastai-keep-instances VALUE	Do not destroy instances after job completion or failure. Useful for debugging, but instances keep accruing charges until destroyed manually!	False		✗