spjuhel/BoARIO-Sensitivity
Sensitivity Analysis of BoARIO
Overview
Topics:
Latest release: None, Last update: 2024-08-08
Linting: linting: failed, Formatting:formatting: failed
Deployment
Step 1: Install Snakemake and Snakedeploy
Snakemake and Snakedeploy are best installed via the Mamba package manager (a drop-in replacement for conda). If you have neither Conda nor Mamba, it is recommended to install Miniforge. More details regarding Mamba can be found here.
When using Mamba, run
mamba create -c conda-forge -c bioconda --name snakemake snakemake snakedeploy
to install both Snakemake and Snakedeploy in an isolated environment. For all following commands ensure that this environment is activated via
conda activate snakemake
Step 2: Deploy workflow
With Snakemake and Snakedeploy installed, the workflow can be deployed as follows. First, create an appropriate project working directory on your system and enter it:
mkdir -p path/to/project-workdir
cd path/to/project-workdir
In all following steps, we will assume that you are inside of that directory. Then run
snakedeploy deploy-workflow https://github.com/spjuhel/BoARIO-Sensitivity . --tag None
Snakedeploy will create two folders, workflow
and config
. The former contains the deployment of the chosen workflow as a Snakemake module, the latter contains configuration files which will be modified in the next step in order to configure the workflow to your needs.
Step 3: Configure workflow
To configure the workflow, adapt config/config.yml
to your needs following the instructions below.
Step 4: Run workflow
The deployment method is controlled using the --software-deployment-method
(short --sdm
) argument.
To run the workflow with automatic deployment of all required software via conda
/mamba
, use
snakemake --cores all --sdm conda
Snakemake will automatically detect the main Snakefile
in the workflow
subfolder and execute the workflow module that has been defined by the deployment in step 2.
For further options such as cluster and cloud execution, see the docs.
Step 5: Generate report
After finalizing your data analysis, you can automatically generate an interactive visual HTML report for inspection of results together with parameters and code inside of the browser using
snakemake --report report.zip
Configuration
The following section is imported from the workflow’s config/README.md
.
Configuration is done via the config
folder content.
It requires a config.yaml
file, and at least one flood_scenarios/<flood-scenario>.yaml
file and one simulations/<simulation-scenario>.yaml
, which are described in following sections. The GitHub repository contains examples of such files that should be used as templates.
It also requires a mriot_params
directory for the MRIOT handling sub-module for which you can find documentation here.
This file governs the high-level configuration of the pipeline (i.e. paths, notably to other configurations files, experiences to run, variables to store, etc.)
To enforce testing before using the pipeline the file contains a testing: True
statement, which we suggest you keep and override when running the pipeline by using the --config testing=False
flag when invoking snakemake.
create_flood_scenarios_csv.py create_simspace_csv.py generate_local_results_rst.py generate_mrio_compare.py generate_results_index_rst.py generate_results_rst.py plot_results.py prep_plot_df.py regroup_aggreg.py simulate.py
This part of the configuration give the required details for the MRIOTs-Tools sub-module and essentially governs where MRIOTs data should be looked for and stored. We suggest to keep it as is in most cases.
-
prefix
is the sub-directory of the pipeline global output which contains everything related to the sub-module. This is where the sub-module looks for configuration files and where it store the MRIOTs data. -
XXX_dir
are the sub-sub-folders where corresponding data is to be found/stored -
common_aggreg
is the name of the aggregation common to all MRIOTs -
mriot_base_aggreg
is the name of the initial, not aggregated MRIOT for each table type.
data-mriot:
prefix: "mriot_data/"
downloaded_mriot_dir: "downloaded"
parsed_mriot_dir: "parsed"
aggregated_mriot_dir: "parsed"
mriot_params_dir: "config/mriot_params"
common_aggreg: “common_sectors_common_regions”
mriot_base_aggreg:
exiobase3_ixi : “full_sectors_full_regions”
exiobase3_pxp : “full_sectors_full_regions”
eora26: “full_no_reexport_sectors_full_regions”
euregio: “full_sectors_full_regions”
Next the file should define the scenarios to run, for instance:
flood scenario: flood_scenarios/germany21.yaml
simulation space test: simulations/test.yaml
simulation space: simulations/germany21.yaml
Flood scenarios define the direct impact for each MRIOTs. Format is the following:
name: <name of the scenario>
year: <year of the impact>
unit: <monetary unit> (“euro”, “dollar”, …)
prod_cap_impact_unit: <unit value of impact on productive capital>
house_impact_unit: <unit value of impact on households capital> (Can be 0)
estimated_gdp_unit: <unit value of GDP of the whole region impacted> Used to translate the impact in relative terms
duration: <duration of the event>
name of the country affected in each of the MRIOTs
countries_affected: exiobase3_ixi: “DE” eora26: “DEU” euregio: “DE”
this is mainly for euregio where there subregions are impacted
should also have a “common” element if you wish to run simulation on
the aggregated version of the MRIOTs
regions_affected: exiobase3_ixi: “DE”: 1. eora26: “DEU” : 1. euregio: ‘DE21’ : 0.00137977016177574 ‘DE22’ : 0.00137977016177574 ‘DE23’ : 0.00137977016177574 […] common: “DEU” : 1.
How the impact is distributed towards the impacted sectors
productive_capital_impact_sectoral_distrib_type: gdp
Currently only one sector scenario can be run per pipeline run, in the future, multiple scenario will be possible.
The scenario is defined based on the exiobase3_ixi_full_sectors.csv
file, located in the
config/mriot_base_scenarios/
folder by default (which can be changed with the mriot_base_config
attribute in the config.yaml
file). Note that for user-friendliness this file is automatically copied
to MRIOT-Tools sub-module data folder.
We use EXIOBASE 3 as a baseline
, has it has the most sectors, then values for other tables are automatically computed
by following a consistent sector mapping. The aggregation mapping used can be found
here.
This "main" csv file should contain:
- a first columns with all 163 sectors of EXIOBASE 3
- an
affected
column with 0 or 1 depending if the sector is impacted or not. Note that sectors in other MRIOTs that correspond to multiple sectors and where least one is affected, will be considered affected, which can produce inconsistencies. Refer to the aggregation file used. - a
rebuilding_factor
column which states the contribution of the sector to the reconstruction effort (0 meaning it does not contribute). The column has to sum up to 1.0 - an
inventory_size
column with either a number of day or "Infinity". A number states the number of days worth, relative to production requirements, of input produced by the sector, that are aimed to be in the stocks of any industry. That is,90
forCultivation of wheat
means all industries "store" the equivalent of 90 times what they requirefrom this sector to produce for one day. "Infinity" means that input from this sectors are not considered as a constraint to production. - a
productive_capital_to_va_ratio
column which states how much productive capital to estimate from the Value Added of the sector. - an
inventory_tau
column which state the characteristic time to resupply inputs from this sector.
Defines the simulation space, which will be a Cartesian product of each possibility.
flood_scenario: # Flood scenarios to run (needs to match a yaml file in flood_scenarios/)
- germany21
mriot: # The MRIOTs to run
exiobase3_ixi
eora26
euregio
mriot_year: # The MRIOTs years to run
2000
2010
The type of aggregation to use (mostly full_sectors_full_regions is relevant,
the two other possibilities are to evaluate the effect of differences in sector/region typologies)
mriot_aggreg:
full_sectors_full_regions
common_sectors_common_regions
common_sectors_full_regions
Required but currently unused, placeholder for future use.
sectors_scenario:
default
Recovery scenarios to run
recovery_scenario:
reclin
reb
recovery_tau:
90
180
365
545
730
Parameters used for the BoARIO model (see its documentation)
order:
alt
psi:
0.5
0.8
0.85
0.90
0.95
0.97
1.0
base_alpha:
1
max_alpha:
1.25
tau_alpha:
1
90
180
365
730
Linting and formatting
Linting results
WorkflowError:
Failed to open source file /home/sjuhel/Repos/BoARIO-MRIOT-Tools/workflow/Snakefile
FileNotFoundError: [Errno 2] No such file or directory: '/home/sjuhel/Repos/BoARIO-MRIOT-Tools/workflow/Snakefile'
Formatting results
[DEBUG]
[ERROR] In file "/tmp/tmp1f12dqr5/workflow/Snakefile": InvalidPython: Black error:
Cannot parse: 5:6: conda:
[DEBUG] In file "/tmp/tmp1f12dqr5/workflow/Snakefile":
[DEBUG] In file "/tmp/tmp1f12dqr5/workflow/rules/simulations.smk": Formatted content is different from original
[DEBUG]
[DEBUG] In file "/tmp/tmp1f12dqr5/workflow/rules/treat_results.smk": Formatted content is different from original
[DEBUG]
[DEBUG] In file "/tmp/tmp1f12dqr5/workflow/rules/prepare_simspace.smk": Formatted content is different from original
[DEBUG]
[DEBUG] In file "/tmp/tmp1f12dqr5/workflow/rules/report.smk": Formatted content is different from original
[INFO] 1 file(s) raised parsing errors 🤕
[INFO] 4 file(s) would be changed 😬
snakefmt version: 0.10.2