MISO2 Multiprocessing

Multiprocessing functions for running MISO2 models.

Created on Mon Sep 26 10:45:08 2022

@author: bgrammer

model.core.MISO2_multiprocessing.miso_run_pool(config, run_model_arguments, save_to_file=None, sf=None, logging_level=30)

Function to be passed to a Multiprocessing.Pool for parallel processing of single regions.

Config and survival functions are meant to be of a single region. If logging or saving outputs is enabled, a new folder named after the output region is created in the output path.

Parameters:

  • config (MISO2Config/str) – Config for a single region. If a string is passed instead of a MISO2Config object, it is interpreted as a path to a valid config.

  • run_model_arguments (dict) – Wraps “output_path”, “estimate_aggregates” options for model run.

  • save_to_file (list) – List of outputs to be saved. May include “sf” and “result”.

  • sf (MISO2SurvivalFunctions) – Survival functions for a single region. If None is passed to this argument, the survival functions will be created.

  • logging_level (logging.LEVEL) – Level of console logging. None will disable logging altogether. Defaults to Warning.

Returns:

Result of the model run.

Return type:

miso_output(MISO2Output)

Raises:

  • ValueError – If mismatch in MC configuration is detected.

  • TypeError – When wrong object types are passed to function.

model.core.MISO2_multiprocessing.miso_run_pool_mc(config, run_model_arguments, save_to_file=None, sf=None, repetitions=1000, logging_level=30, sf_randomization=True)

Function to be passed to a Multiprocessing.Pool for Monte Carlo parallel processing of single regions.

Config and survival functions are meant to be of a single region. The config needs to be set up for MC randomization, with uncertainty values present. The save_to_file list specifies which outputs are to be saved. A new folder named after the output region is created in the output path and will hold the base file log and the MISO2Stats object (as JSON/parquet files) that aggregates mean and variances over all MC runs. The logfiles of the individual MC runs and the outputs (if they are saved) are located in subfolders created automatically within that directory, with each holding up to 500 runs.

Be aware that if you run this function with a high number of repetitions, and you are saving all outputs, it will have a considerable runtime and consume significant amounts of disk space.

Parameters:

  • config (MISO2Config/str) – Config for a single region. If a string is passed instead of a MISO2Config object, it is interpreted as a path to a valid config.

  • run_model_arguments (dict) – Wraps “estimate_aggregates”, “mc_batch_size”.

  • save_to_file (list) – List of outputs to be saved. May include “all_outputs”, “sf” and “result”. Defaults to None.

  • sf (MISO2SurvivalFunctions) – Survival functions for a single region. If None is passed to this argument, the survival functions will be created.

  • repetitions (int) – Number of Monte Carlo repetitions. Defaults to 1000, hardcapped limit of 15.000.

  • logging_level (logging.LEVEL) – Level of console logging. Defaults to Warning.

  • sf_randomization (bool) – Randomize survival functions per run. True by default.

Returns:

Aggregated stats of all output runs.

Return type:

miso_stats(MISO2Stats)

Raises:

  • ValueError – When illegal number of repetitions are to be executed or mismatch in MC configuration

  • TypeError – When wrong object types are passed to function.

model.core.MISO2_multiprocessing.miso_run_pool_mc_batch(config, run_model_arguments, sf=None, repetitions=10000, logging_level=30)

Simplified batch version that skips construction of MISO stats objects and only saves output files.

Function to be passed to a Multiprocessing.Pool for Monte Carlo parallel processing of single regions.

Config and survival functions are meant to be of a single region. The config needs to be set up for MC randomization. The save_to_file list specifies which outputs are to be saved. A new folder named after the output region is created in the output path and will hold the base file log and the MISO2Stats object (as JSON/parquet files). The logfiles of the individual MC runs and the outputs (if they are saved) are located in subfolders created automatically within that directory, with each holding up to 500 runs.

Be aware that if you run this function with a high number of repetitions, it will have a considerable runtime and consume significant amounts of disk space.

Parameters:

  • config (MISO2Config/str) – Config for a single region. If a string is passed instead of a MISO2Config object, it is interpreted as a path to a valid config.

  • run_model_arguments (dict) – Wraps “estimate_aggregates”, “mc_batch_size”.

  • sf (MISO2SurvivalFunctions) – Survival functions for a single region. If None is passed to this argument, the survival functions will be created.

  • repetitions (int) – Number of Monte Carlo repetitions. Defaults to 1000, hardcapped limit of 15.000.

  • logging_level (logging.LEVEL) – Level of console logging. Defaults to Warning.

Returns:

Aggregated stats of all output runs.

Return type:

miso_stats(MISO2Stats)

Raises:

  • ValueError – When illegal number of repetitions are to be executed or mismatch in MC configuration

  • TypeError – When wrong object types are passed to function.

model.core.MISO2_multiprocessing.miso_run_pool_sensitivity(config, run_model_arguments, logging_level=30)

Executes sensitivity parameter testing for a given region configuration.

This function performs sensitivity analysis by running the model with different parameter groups and scenarios. It logs the process and outputs the results for each combination of parameter group and scenario.

Parameters:

  • config (MISO2Config) – Configuration object for the region to be analyzed.

  • run_model_arguments (dict) – Arguments required to run the model, including the output path and survival function parametrization.

  • logging_level (int, optional) – Logging level for the file logger. Defaults to logger.WARNING.

Returns:

A dictionary where keys are tuples of (parameter_group, scenario) and values are

the model outputs for those combinations.

Return type:

dict

Notes

  • Creates a timestamped log file in the output directory for the specified region.

  • Iterates over parameter groups and scenarios to perform the sensitivity analysis.

  • Uses a cache to store and reuse computed survival functions.

  • For the “Multiplier” parameter group, temporarily modifies the multiplier values during the analysis and restores them afterward.