Data I/O Manager#

Functions for loading input data and exporting results.

Data Loading#

sdom.io_manager.load_data(input_data_dir: str = '.\\Data\\')[source]#

Load all required SDOM input datasets from CSV files in the specified directory.

Reads and validates all input CSV files needed for SDOM optimization including VRE data, fixed generation profiles, storage characteristics, thermal units, scalars, and formulation specifications. Performs data consistency checks and filters datasets based on completeness.

Parameters:

input_data_dir (str, optional) – Path to directory containing input CSV files. Defaults to ‘.Data'. Should contain all required files defined in constants.INPUT_CSV_NAMES.

Returns:

Dictionary containing loaded and processed data with keys:

  • ’formulations’ (pd.DataFrame): Component formulation specifications

  • ’solar_plants’, ‘wind_plants’ (list): Plant IDs for VRE technologies

  • ’cf_solar’, ‘cf_wind’ (pd.DataFrame): Hourly capacity factors

  • ’cap_solar’, ‘cap_wind’ (pd.DataFrame): Plant CAPEX and capacity data

  • ’load_data’ (pd.DataFrame): Hourly electricity demand

  • ’nuclear_data’ (pd.DataFrame): Hourly nuclear generation

  • ’large_hydro_data’ (pd.DataFrame): Hourly hydropower generation/availability

  • ’large_hydro_max’, ‘large_hydro_min’ (pd.DataFrame): Hydro bounds (if budget formulation)

  • ’other_renewables_data’ (pd.DataFrame): Hourly other renewable generation

  • ’storage_data’ (pd.DataFrame): Storage technology characteristics

  • ’STORAGE_SET_J_TECHS’, ‘STORAGE_SET_B_TECHS’ (list): Storage tech identifiers

  • ’thermal_data’ (pd.DataFrame): Thermal balancing unit parameters

  • ’scalars’ (pd.DataFrame): System-level scalar parameters

  • ’import_cap’, ‘export_cap’, ‘import_prices’, ‘export_prices’ (pd.DataFrame): Trade data (if import/export formulation active)

  • ’complete_solar_data’, ‘complete_wind_data’ (pd.DataFrame): Filtered VRE data

  • ’filtered_cap_solar_dict’, ‘filtered_cap_wind_dict’ (dict): Capacity mappings

Return type:

dict

Raises:

  • FileNotFoundError – If any required input file is missing from input_data_dir.

  • ValueError – If formulation specifications are invalid.

Notes

  • All numeric data rounded to 5 decimal places for consistency

  • VRE plant lists filtered to include only plants with complete data

  • Conditionally loads hydro bounds and import/export data based on formulations

  • Uses flexible filename matching via normalize_string() for CSV files

  • Logs detailed progress at debug level for troubleshooting data loading issues

Formulation Management#

sdom.io_manager.get_formulation(data: dict, component: str = 'hydro')[source]#

Retrieve the selected formulation for a specific model component.

Extracts the formulation name from the loaded formulations DataFrame for a given component (e.g., hydro, imports, exports). Used throughout model initialization to conditionally add constraints based on formulation.

Parameters:

  • data (dict) – Dictionary containing the ‘formulations’ DataFrame loaded from formulations.csv.

  • component (str, optional) – Component name to look up (case-insensitive). Examples: ‘hydro’, ‘Imports’, ‘Exports’. Defaults to ‘hydro’.

Returns:

The formulation name for the specified component (e.g.,

’MonthlyBudgetFormulation’, ‘CapacityPriceNetLoadFormulation’, ‘NotModel’).

Return type:

str

Notes

Performs case-insensitive matching on component name. Returns the first matching formulation (expects unique component names).

sdom.io_manager.check_formulation(formulation: str, valid_formulations)[source]#

Validate that a formulation string is in the list of valid formulations.

Checks if the user-specified formulation (from formulations.csv) is valid for the component being configured. Raises an error with helpful message if invalid.

Parameters:

  • formulation (str) – The formulation name specified by user (e.g., ‘MonthlyBudgetFormulation’, ‘RunOfRiverFormulation’).

  • valid_formulations – Iterable (typically dict.keys()) containing all valid formulation names for the component.

Returns:

None

Raises:

ValueError – If formulation is not in valid_formulations, with a message listing all valid options.

Notes

This function is called during data loading to validate formulation.csv entries.

Results Export#

sdom.io_manager.export_results(results, case: str, output_dir: str = './results_pyomo/')[source]#

Export optimization results to CSV files.

Writes the results from an OptimizationResults object to CSV files in the specified directory. Creates output directory if it doesn’t exist.

Parameters:

  • results (OptimizationResults) – The optimization results object from run_solver().

  • case (str or int) – Case identifier used in output filenames to distinguish between different scenarios or runs.

  • output_dir (str, optional) – Directory path for output files. Defaults to ‘./results_pyomo/’. Directory will be created if it doesn’t exist.

Returns:

  • None

  • Output Files

  • ————

  • OutputGeneration_{case}.csv – Hourly dispatch results containing: Scenario, Hour, Solar PV/Wind generation and curtailment, Thermal, hydro, nuclear, other renewables generation, Storage net charge/discharge, imports, exports, Load.

  • OutputStorage_{case}.csv – Hourly storage operation for each technology: Hour, Technology, Charging power (MW), Discharging power (MW), State of charge (MWh).

  • OutputSummary_{case}.csv – Summary metrics including: Total costs, Installed capacities by technology, Total generation by technology, Demand statistics, Cost breakdowns (VRE, storage, thermal CAPEX/FOM/VOM).

  • OutputThermalGeneration_{case}.csv – Disaggregated hourly thermal generation by plant (only if more than one thermal plant exists).

Notes

This function accepts either an OptimizationResults dataclass (new API) or the legacy tuple return from run_solver (deprecated).

Example: Loading Data#

from sdom import load_data

# Load data from directory
data = load_data('./Data/my_scenario/')

# Examine loaded data
print(f"Solar plants: {len(data['solar_plants'])}")
print(f"Wind plants: {len(data['wind_plants'])}")
print(f"Storage technologies: {data['STORAGE_SET_J_TECHS']}")

# Access formulation settings
hydro_form = data['formulations']
print(hydro_form)

Example: Exporting Results#

from sdom import run_solver, export_results

# Run optimization
results = run_solver(model, solver_config)

# Export results using the OptimizationResults object
if results.is_optimal:
    export_results(
        results=results,
        case="scenario_95pct_clean",
        output_dir="./results_pyomo/"
    )

# This creates three CSV files:
# - OutputGeneration_scenario_95pct_clean.csv
# - OutputStorage_scenario_95pct_clean.csv
# - OutputSummary_scenario_95pct_clean.csv

# You can also access the DataFrames directly from results:
generation_df = results.generation_df
storage_df = results.storage_df
summary_df = results.summary_df

Note

The export_results function now accepts an OptimizationResults object as the first argument. See Results Module for details on the OptimizationResults class.

Data Dictionary Structure#

The load_data() function returns a dictionary with these keys:

  • formulations: Component formulation specifications

  • solar_plants, wind_plants: Lists of plant IDs

  • cf_solar, cf_wind: DataFrames with capacity factors

  • cap_solar, cap_wind: DataFrames with CAPEX and capacity data

  • load_data: Hourly demand DataFrame

  • nuclear_data: Hourly nuclear generation

  • large_hydro_data: Hourly hydro generation

  • other_renewables_data: Hourly other renewables

  • storage_data: Storage technology parameters

  • STORAGE_SET_J_TECHS: List of all storage technologies

  • STORAGE_SET_B_TECHS: List of coupled storage technologies

  • thermal_data: Thermal unit parameters

  • scalars: System-level parameters

  • import_cap, export_cap (optional): Trade capacity limits

  • import_prices, export_prices (optional): Trade prices