Utility Functions#

Helper functions and utilities used throughout SDOM.

Pyomo Utilities#

sdom.common.utilities.safe_pyomo_value(var)[source]#

Safely extract the value from a Pyomo variable or expression.

This function attempts to retrieve the value of a Pyomo variable or expression after optimization. It handles cases where the variable may be uninitialized or None.

Parameters:: var – A Pyomo variable, expression, or parameter from which to extract the value. Can be None.
Returns:: The numeric value of the variable/expression if it is initialized and not None, otherwise returns None.

Notes

This function is essential when collecting results from solved models, as it prevents ValueErrors when accessing uninitialized or optional model components.

File Utilities#

sdom.common.utilities.check_file_exists(filepath, file_name, file_description='')[source]#

Verify that a required input file exists in the specified directory.

This function searches for a file using flexible name matching via get_complete_path() and raises an informative error if the file is not found.

Parameters:

filepath (str) – Directory path where the file should be located.
file_name (str) – Filename to search for (typically a CSV file).
file_description (str, optional) – Human-readable description of the file’s purpose, used in error messages. Defaults to empty string.

Returns:

Complete path to the verified file.

Return type:

str

Raises:

FileNotFoundError – If the specified file cannot be found in the directory, with an error message including the file description.

Notes

This function logs an error message before raising the exception to help with debugging input data configuration issues.

sdom.common.utilities.get_complete_path(filepath, file_name)[source]#

Search for a CSV file in a directory using flexible name matching.

This function performs case-insensitive matching of CSV filenames, ignoring spaces, hyphens, and underscores. It searches the specified directory for a file that matches the base_name pattern.

Parameters:

filepath (str) – Directory path where the file should be located.
file_name (str) – Base filename to search for (should include .csv extension).

Returns:

Complete path to the matched file if found, otherwise an empty string.

Return type:

str

Notes

Only searches for files with .csv extension. Uses normalize_string() for flexible matching to handle variations in input file naming conventions.

sdom.common.utilities.normalize_string(name: str) → str[source]#

Normalize a string for case-insensitive filename comparison.

Removes spaces, hyphens, and underscores from the input string and converts it to lowercase. This function is used to match CSV filenames flexibly, ignoring common formatting variations.

Parameters:

name (str) – The string to normalize (typically a filename or identifier).

Returns:

The normalized string with spaces, hyphens, and underscores removed,: converted to lowercase.

Return type:

str

Example

>>> normalize_string("Load_hourly-2025.csv")
'loadhourly2025.csv'

Data Utilities#

sdom.common.utilities.compare_lists(list1, list2, text_comp='', list_names=['', ''])[source]#

Compare two lists for consistency in length and elements.

Validates that two lists have the same length and contain the same elements, logging warnings for any discrepancies. Used to verify consistency between related datasets (e.g., capacity factors vs. CAPEX data for the same plants).

Parameters:

list1 (list) – First list to compare.
list2 (list) – Second list to compare.
text_comp (str, optional) – Description of what is being compared, used in warning messages. Defaults to empty string.
list_names (list of str, optional) – Two-element list containing names/labels for the lists being compared (e.g., [‘CF’, ‘Capex’]). Defaults to [‘’, ‘’].

Returns:

True if lists have the same length and elements, False otherwise.

Return type:

bool

Notes

Uses set comparison to check for element equality, so order is ignored. Logs warnings via logging module when discrepancies are found.

sdom.common.utilities.concatenate_dataframes(df: DataFrame, new_data_dict: dict, run=1, unit='$US', metric='')[source]#

Append optimization results from a dictionary to an existing DataFrame.

This function converts a dictionary of results into a DataFrame row and appends it to the existing DataFrame with additional metadata columns (Run, Unit, Metric). Used for collecting results across multiple optimization runs or scenarios.

Parameters:

df (pd.DataFrame) – The DataFrame to which the new data will be appended.
new_data_dict (dict) – Dictionary containing technology names as keys and optimal values as values (e.g., {‘Li-Ion’: 1250.5, ‘CAES’: 800.0}).
run (int, optional) – Run or scenario identifier. Defaults to 1.
unit (str, optional) – Unit of measurement for the values. Defaults to ‘$US’.
metric (str, optional) – Metric name or description (e.g., ‘Capacity’, ‘Cost’). Defaults to empty string.

Returns:

Updated DataFrame with the new row(s) appended, containing: columns: [‘Technology’, ‘Optimal Value’, ‘Run’, ‘Unit’, ‘Metric’].

Return type:

pd.DataFrame

Notes

The new_data_dict is pivoted so each key becomes a separate row in the resulting DataFrame, all sharing the same Run, Unit, and Metric values.

sdom.common.utilities.get_dict_string_void_list_from_keys_in_list(keys: list) → dict[source]#

Create a dictionary with string keys mapped to empty lists.

Generates a dictionary where each element from the input list becomes a string key associated with an empty list value. Used for initializing result containers that will be populated during model execution.

Parameters:

keys (list) – List of items to use as dictionary keys (typically plant IDs or technology identifiers).

Returns:

Dictionary with string-converted keys from the input list, each mapped: to an empty list.

Return type:

dict

Example

>>> get_dict_string_void_list_from_keys_in_list([101, 202, 303])
{'101': [], '202': [], '303': []}

Constants#

Key constants defined in sdom.constants:

from sdom.constants import (
    MW_TO_KW,                                      # 1000.0
    INPUT_CSV_NAMES,                               # Dict of CSV filenames
    VALID_HYDRO_FORMULATIONS_TO_BUDGET_MAP,       # Hydro formulation types
    VALID_IMPORTS_EXPORTS_FORMULATIONS_TO_DESCRIPTION_MAP,  # Trade formulations
)

Unit Conversion#

MW_TO_KW = 1000.0: Conversion factor from MW to kW

Input File Names#

The INPUT_CSV_NAMES dictionary maps logical names to expected filenames:

{
    'formulations': 'formulations.csv',
    'load_data': 'Load_hourly.csv',
    'nuclear_data': 'Nucl_hourly.csv',
    'large_hydro_data': 'lahy_hourly.csv',
    'cf_solar': 'CFSolar.csv',
    'cf_wind': 'CFWind.csv',
    'cap_solar': 'CapSolar.csv',
    'cap_wind': 'CapWind.csv',
    'storage_data': 'StorageData.csv',
    'thermal_data': 'Data_BalancingUnits.csv',
    'scalars': 'scalars.csv',
    # ... and more
}

Hydro Formulations#

VALID_HYDRO_FORMULATIONS_TO_BUDGET_MAP = {
    "MonthlyBudgetFormulation": 730,   # Hours per month (approx)
    "DailyBudgetFormulation": 24,      # Hours per day
    "RunOfRiverFormulation": 1         # No aggregation
}

Import/Export Formulations#

VALID_IMPORTS_EXPORTS_FORMULATIONS_TO_DESCRIPTION_MAP = {
    "NotModel": "No imports/exports considered",
    "CapacityPriceNetLoadFormulation": "Price-based import/export optimization"
}

Example Usage#

from sdom.common.utilities import safe_pyomo_value, compare_lists
from sdom.constants import MW_TO_KW

# Safely extract variable values
total_cost = safe_pyomo_value(model.Obj.expr)
wind_capacity = safe_pyomo_value(model.wind.total_installed_capacity)

# Convert units
wind_capacity_kw = wind_capacity * MW_TO_KW

# Compare plant lists
solar_cf_plants = ['101', '102', '103']
solar_capex_plants = ['101', '102', '103']
compare_lists(solar_cf_plants, solar_capex_plants, 
              text_comp='solar plants',
              list_names=['CF', 'CAPEX'])

Utility Functions#

Pyomo Utilities#

File Utilities#

Data Utilities#

Constants#

Unit Conversion#

Input File Names#

Hydro Formulations#

Import/Export Formulations#

Example Usage#

This Page