Unit checking for v4 databases #232
Conversation
WalkthroughAdds an opt-in Pint-based unit-checking subsystem: database Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes
Possibly related PRs
Suggested labels
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (22)
data_files/my_configs/config_sample.toml(1 hunks)data_files/temoa_schema_v4.sql(6 hunks)docs/source/index.rst(1 hunks)docs/source/unit_checking.rst(1 hunks)pyproject.toml(1 hunks)requirements-dev.txt(3 hunks)requirements.txt(3 hunks)temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/core/config.py(3 hunks)temoa/extensions/modeling_to_generate_alternatives/mga_sequencer.py(2 hunks)temoa/extensions/myopic/myopic_sequencer.py(1 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/entry_checker.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)temoa/model_checking/unit_checking/temoa_units.txt(1 hunks)temoa/tutorial_assets/config_sample.toml(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-10-27T15:53:41.829Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 177
File: temoa/model_checking/commodity_network.py:26-33
Timestamp: 2025-10-27T15:53:41.829Z
Learning: The Temoa project requires Python 3.12 or above as the minimum supported version, so PEP 695 `type` syntax for type aliases is appropriate and preferred over `TypeAlias`.
Applied to files:
requirements.txtrequirements-dev.txt
🧬 Code graph analysis (10)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-392)temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(122-124)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/extensions/modeling_to_generate_alternatives/mga_sequencer.py (3)
temoa/extensions/myopic/myopic_sequencer.py (1)
_run_unit_check(140-156)temoa/cli.py (1)
check_units(293-392)temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
temoa/extensions/myopic/myopic_sequencer.py (1)
temoa/extensions/modeling_to_generate_alternatives/mga_sequencer.py (2)
_run_unit_check(123-139)start(141-339)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
temoa/model_checking/unit_checking/entry_checker.py (1)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(122-124)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(146-154)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-64)validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(115-118)consolidate_lines(146-154)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/core/config.py (1)
temoa/cli.py (1)
check_units(293-392)
🪛 Ruff (0.14.7)
tests/test_unit_checking.py
33-33: Wrong type passed to first argument of pytest.mark.parametrize; expected tuple
Use a tuple for the first argument
(PT006)
37-37: Boolean-typed positional argument in function definition
(FBT001)
37-37: Unused function argument: expected
(ARG001)
63-63: Wrong type passed to first argument of pytest.mark.parametrize; expected tuple
Use a tuple for the first argument
(PT006)
67-67: Boolean-typed positional argument in function definition
(FBT001)
83-83: Wrong type passed to first argument of pytest.mark.parametrize; expected tuple
Use a tuple for the first argument
(PT006)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/screener.py
43-43: Avoid specifying long messages outside the exception class
(TRY003)
253-253: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/model_checking/unit_checking/entry_checker.py
47-47: Consider moving this statement to an else block
(TRY300)
58-58: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/table_checker.py
128-128: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/common.py
107-107: Boolean positional value in function call
(FBT003)
107-107: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
110-110: Boolean positional value in function call
(FBT003)
110-110: Boolean positional value in function call
(FBT003)
temoa/model_checking/unit_checking/relations_checker.py
149-149: Consider moving this statement to an else block
(TRY300)
187-189: Possible SQL injection vector through string-based query construction
(S608)
193-193: Possible SQL injection vector through string-based query construction
(S608)
201-201: Possible SQL injection vector through string-based query construction
(S608)
203-203: Possible SQL injection vector through string-based query construction
(S608)
205-205: Avoid specifying long messages outside the exception class
(TRY003)
211-213: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
246-246: Avoid specifying long messages outside the exception class
(TRY003)
348-351: Avoid specifying long messages outside the exception class
(TRY003)
353-354: Possible SQL injection vector through string-based query construction
(S608)
359-361: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
404-404: Do not catch blind exception: Exception
(BLE001)
temoa/core/config.py
62-62: Boolean-typed positional argument in function definition
(FBT001)
62-62: Boolean default positional argument in function definition
(FBT002)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (37)
data_files/temoa_schema_v4.sql (3)
106-106: Schema changes are syntactically correct and follow consistent placement patterns.All six modified tables have the new nullable
units TEXTcolumns positioned consistently (after the primary data/value column, before thenotescolumn), use proper SQL syntax, and avoid breaking changes by remaining nullable with no additional constraints. The placement aligns with existing patterns in the schema and supports the opt-in unit checking feature.Also applies to: 116-117, 261-261, 360-360, 383-383, 393-393
261-261: Efficiencyunitscolumn stores commodity unit references—semantics are clear from validation logic.The
unitscolumn added on line 261 stores references to commodity units (option b), not efficiency ratio units. Validation logic intemoa/model_checking/unit_checking/relations_checker.pyexplicitly validates thatefficiency.unitsmatch the units of the associatedinput_commandoutput_commcommodities, enforcing consistency across the model. The validation generates error messages like "Efficiency units conflict with associated commodity for Technology X" when mismatches occur. This design aligns with the PR objective to ensure efficiency table units match commodity units.
106-106: No actionable issues identified. The 6 modified lines in this diff represent the changes being introduced, and the full schema file contains units columns across approximately 29 table locations, aligning with the PR's stated objective of supporting units across the v4 schema. The original review comment's concern about a 28-versus-6 discrepancy stems from conflating the number of diff changes with the total intended scope.temoa/extensions/myopic/myopic_sequencer.py (1)
158-161: start() pre-check hook looks appropriateRunning
_run_unit_check()at the very beginning ofstart()ensures myopic flows can perform unit checks even when used independently of the main sequencer. Once the duplication concern withTemoaSequenceris resolved, this sequencing is reasonable.requirements-dev.txt (1)
30-33: New dev dependencies align with unit-checking featureThe additions of
flexcache,flexparser,pint,platformdirs, and the updatedtyping-extensions“via” comments are consistent with the new Pint-based unit-checking stack and Typer usage. Given this file is uv-generated, keeping these changes here (rather than hand-editing) is appropriate.Also applies to: 105-108, 201-205
temoa/extensions/modeling_to_generate_alternatives/mga_sequencer.py (2)
141-152: Prepending unit check to start() is sensibleHooking
_run_unit_check()at the start ofstart()ensures MGA runs benefit from unit validation before data loading and solving, particularly when invoked directly. Once duplication with the core sequencer is addressed, this ordering is appropriate.
123-140: Directory creation is already handled by the screener; focus on the duplicate unit check coordinationThe first concern about missing directory creation is incorrect. The
screen()function inscreener.pyalready handles directory creation via_write_report()(lines 254–255), which checks and creates the directory withmkdir(parents=True)if needed. The suggested diff to addmkdirin_run_unit_check()is redundant.However, the duplicate unit check concern has merit but is broader than stated: when MGA is launched via
TemoaSequencer.start(), unit checks fire twice—once in_run_preliminary_checks()and again inMgaSequencer._run_unit_check(). The same issue exists for MyopicSequencer. This is an architectural coordination problem that should be addressed at theTemoaSequencerlevel (by conditionally skipping_run_preliminary_checks()for modes that handle their own checks), not by adding redundant mkdir calls to individual sequencers.tests/test_unit_checking.py (2)
46-76: Expression and registry tests look correct once format validation is fixedThe parametrized cases for
validate_units_expression(including aliases likedollar/dollars/USDand custom units likeethos) and the equality checks againstureginstances are well-constructed and should provide good coverage of the registry behavior. No changes needed here beyond the format-test fix above.
79-92: Time-dimension locator test is a good sanity checkUsing
test_value.dimensionality.get('[time]')to validate exponents forkW,kWh,PJ, andPJ/his a nice way to lock in the expected temporal dimensionality in the unit registry. The cases and expected exponents (-3 vs -2) look consistent with physical dimensions.temoa/cli.py (1)
292-392: CLIcheck-unitscommand is well-structured and robustThe new
check-unitscommand:
- Validates the database path via Typer’s argument options.
- Creates the report directory up front and surfaces creation errors cleanly.
- Delegates unit checking to
screen(...)and provides clear user feedback, including a concise report summary when issues are found.- Handles both missing DBs and unexpected failures with appropriate logging and non-zero exit codes.
This fits well with the rest of the CLI design.
temoa/core/config.py (1)
37-65: Newcheck_unitsflag is properly integrated and poses no actual API stability riskThe new
check_unitsparameter has been correctly added to__init__(line 37), properly assigned toself.check_units(line 140), and appropriately displayed in__repr__as "Unit checking" (line 251). The implementation is complete and consistent across all three locations.The theoretical concern about positional argument shifting is not a practical risk in this codebase. An exhaustive search reveals only two active
TemoaConfiginstantiations:
- Line 231: constructed with keyword arguments (
output_path=,config_file=,silent=,**data)- Line 192: constructed with no arguments, relying on defaults
No code anywhere uses positional arguments to construct
TemoaConfig, so inserting the new parameter cannot silently rebind existing calls. The implementation is safe as-is.temoa/model_checking/unit_checking/screener.py (2)
247-258: LGTM - timezone warning is acceptable here.The function correctly handles directory creation and file writing with proper encoding. The static analysis warning about
datetime.now()lacking a timezone is acceptable in this context since the timestamp is used for local file naming where local time is appropriate.
262-270: LGTM - standalone execution helper is useful.The
__main__block provides a convenient way to test the screener independently, which is useful for development.temoa/model_checking/unit_checking/entry_checker.py (3)
24-36: LGTM!The function correctly validates unit format using regex patterns and returns appropriate tuple results for both valid and invalid cases.
39-49: LGTM - static analysis hint is a false positive.The function correctly validates unit expressions against the registry. The
returnstatement in thetryblock (line 47) is appropriate here - the static analysis suggestion to move it to anelseblock would not improve clarity or correctness.
52-63: Validate table name to prevent SQL injection.The function constructs SQL with an f-string using the
tableparameter (line 58), which could allow SQL injection if an untrusted value is passed. While the current callers only pass values fromtables_with_units, the function itself doesn't enforce this constraint.Apply this diff to add validation:
def gather_from_table(conn: sqlite3.Connection, table: str) -> dict[str, list[int]]: """gather all unique "units" entries from a table and collect the row indices""" + # Validate table name against known safe list + from temoa.model_checking.unit_checking.common import tables_with_units + if table not in tables_with_units: + raise ValueError(f"Invalid table name: {table}") res = defaultdict(list)Alternatively, use parameterized queries if SQLite supports table name parameters in your version, or use an identifier quoting mechanism.
⛔ Skipped due to learnings
Learnt from: ParticularlyPythonicBS Repo: TemoaProject/temoa PR: 184 File: temoa/data_processing/database_util.py:109-118 Timestamp: 2025-11-03T13:34:35.907Z Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.Learnt from: ParticularlyPythonicBS Repo: TemoaProject/temoa PR: 184 File: temoa/data_processing/database_util.py:109-118 Timestamp: 2025-11-03T13:34:35.907Z Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.temoa/model_checking/unit_checking/table_checker.py (2)
29-129: LGTM - comprehensive unit validation with excellent documentation.The function implements thorough validation with particularly good handling of the capacity vs. energy units distinction (lines 91-116). The detailed comments explaining the physics behind the check are very helpful for maintainability.
The error messages are clear and actionable, including suggestions for fixes.
101-117: Excellent capacity vs. energy unit validation.The check for energy units in capacity tables is well-implemented with clear physics-based reasoning. The suggested fix (
unit_obj / ureg.year) is helpful guidance, though users may need to consider their data more broadly.temoa/model_checking/unit_checking/common.py (4)
10-50: LGTM - well-documented table list.The list correctly identifies tables with units columns in v4 schema. The commented-out sections are well-explained:
- Growth/degrowth tables use
seed_unitsinstead ofunits- Output tables don't have units in current v4 schema
97-112: LGTM - cost table definitions are correct.The
CostTableDataentries correctly specify the properties of each cost table:
cost_invest: One-time investment per capacitycost_emission: Cost per emissioncost_fixed: Fixed O&M per capacity per period (correctly marked as period_based)cost_variable: Variable cost per activityThe static analysis warnings about boolean positional arguments (FBT003) are false positives here - the NamedTuple field names provide sufficient clarity.
121-142: LGTM - well-designed regex patterns for unit validation.The
SINGLE_ELEMENTandRATIO_ELEMENTpatterns correctly handle the expected unit formats, including:
- Support for exponents (numbers allowed)
- Proper isolation of denominators in parentheses for ratios
- Whitespace trimming
The comments reference unit tests which aid understanding.
146-154: LGTM - helpful utility for concise error reporting.The function appropriately limits line number lists to maintain readable error messages while still indicating the full extent of issues.
temoa/model_checking/unit_checking/relations_checker.py (6)
30-43: LGTM!The function safely builds a lookup table from the commodity table using a fixed SQL query.
46-59: LGTM!The function safely builds a C2A lookup table using a fixed SQL query.
68-141: LGTM - comprehensive efficiency table validation.The function thoroughly validates:
- Unit format and registry compliance
- Alignment with commodity units
- Consistency across same-name technologies
The error messages are well-formatted and actionable.
311-327: LGTM!The function provides a standardized format for error messages with clear labeling of table entries, conversions, results, and expectations.
330-476: Address SQL injection risk and improve exception handling.Two issues need attention:
SQL Injection Risk (line 353): The function builds SQL using f-strings with values from
CostTableData. Whilecost_based_tablesis a controlled list, there's no validation at the function level.Overly broad exception handling (line 404): Catching bare
Exceptionshould be replaced with specific exceptions expected from Pint operations.Fix 1: Add input validation
def check_cost_tables( conn: sqlite3.Connection, cost_tables: Iterable[CostTableData], tech_lut: dict[str, IOUnits], c2a_lut: dict[str, Unit], commodity_lut: dict[str, Unit], ) -> list[str]: """ Check all cost tables for (a) alignment of units to tech output (the denominator) and (b) 100% commonality in the cost units (numerator) Note: we'll *assume* the first passing entry in the first table establishes the common cost units and check for consistency """ + from temoa.model_checking.unit_checking.common import cost_based_tables + valid_tables = {ct.table_name for ct in cost_based_tables} + common_cost_unit = None # Expectation: MUSD. Something with a prefix and currency dimension error_msgs = [] for ct in cost_tables: + if ct.table_name not in valid_tables: + raise ValueError(f"Invalid table name: {ct.table_name}") table_grouped_errors = defaultdict(list)Fix 2: Use specific exception types
try: if '[currency]' in cost_units.dimensionality: common_cost_unit = cost_units else: # Units don't contain currency dimension - invalid for cost table error_msgs.append( f'{ct.table_name}: Unprocessed row ' f'(units lack currency dimension): {cost_units} at row: {idx}' ) continue - except Exception: + except (AttributeError, TypeError) as e: error_msgs.append( - f'{ct.table_name}: Could not check cost units: {cost_units} at row: {idx}' + f'{ct.table_name}: Could not check cost units: {cost_units} at row: {idx} ({e})' ) continueAlso fix the logging on line 359-361:
except sqlite3.OperationalError: - logger.error( + logger.exception( 'failed to process query: %s when processing table %s', query, ct.table_name )⛔ Skipped due to learnings
Learnt from: ParticularlyPythonicBS Repo: TemoaProject/temoa PR: 184 File: temoa/data_processing/database_util.py:109-118 Timestamp: 2025-11-03T13:34:35.907Z Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.Learnt from: ParticularlyPythonicBS Repo: TemoaProject/temoa PR: 184 File: temoa/data_processing/database_util.py:109-118 Timestamp: 2025-11-03T13:34:35.907Z Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
154-308: Add input validation and improve error logging.The function builds SQL queries dynamically using f-strings with
table_nameandtech_columnvariables (lines 182-203). While current callers pass controlled values, the function doesn't validate inputs, creating potential SQL injection risks.Additionally, line 211-213 should use
logging.exception()instead oflogging.error()to capture the full exception traceback.Apply these improvements:
def check_inter_table_relations( conn: sqlite3.Connection, table_name: str, tech_lut: dict[str, IOUnits], comm_lut: dict[str, Unit], relation_type: RelationType, ) -> list[str]: """Check the tech and units in the given table vs. baseline (expected) values for the tech. Fixed: Made SQL queries more robust to handle: - Missing columns (e.g., 'region' may not exist in all tables) - Missing tables (e.g., some databases may not have all limit tables) - Schema variations between v3.1 and v4.0 """ + # Validate table_name against known safe tables + from temoa.model_checking.unit_checking.common import ( + activity_based_tables, capacity_based_tables, commodity_based_tables + ) + valid_tables = activity_based_tables + capacity_based_tables + commodity_based_tables + if table_name not in valid_tables: + raise ValueError(f"Invalid table name: {table_name}") + grouped_errors = defaultdict(list)And fix the logging:
except sqlite3.OperationalError as e: # Log the error but don't fail the entire check - logger.error( + logger.exception( 'failed to process query: %s when processing table %s: %s', query, table_name, e )⛔ Skipped due to learnings
Learnt from: ParticularlyPythonicBS Repo: TemoaProject/temoa PR: 184 File: temoa/data_processing/database_util.py:109-118 Timestamp: 2025-11-03T13:34:35.907Z Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.Learnt from: ParticularlyPythonicBS Repo: TemoaProject/temoa PR: 184 File: temoa/data_processing/database_util.py:109-118 Timestamp: 2025-11-03T13:34:35.907Z Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.temoa/tutorial_assets/config_sample.toml (1)
49-52: LGTM! Clear configuration option with appropriate tutorial default.The new
check_unitsconfiguration option is well-documented and appropriately enabled in the tutorial config to showcase the feature. The placement afterplot_commodity_networkmaintains logical grouping of database validation options.requirements.txt (1)
15-18: LGTM! Auto-generated dependency resolution for Pint and its transitive dependencies.The additions correctly reflect the new
pint>=0.25.2dependency and its transitive requirements (flexcache, flexparser, platformdirs, typing-extensions). The exact version pins (==) in this lock file complement the minimum version constraint (>=) in pyproject.toml, which is standard practice for reproducible builds.Also applies to: 68-71, 118-122
docs/source/unit_checking.rst (1)
1-407: LGTM! Comprehensive and well-structured documentation.This documentation provides excellent coverage of the unit checking subsystem, including:
- Clear usage instructions (config and CLI)
- Detailed format requirements with examples
- Common pitfalls with solutions
- Troubleshooting guidance
- Reference tables for schema coverage
The inclusion of warnings for critical format requirements (lines 74-89) and the extensive "Common Footguns and Pitfalls" section (lines 111-231) will significantly help users avoid common mistakes.
docs/source/index.rst (1)
7-7: LGTM! Documentation index updated correctly.The addition of
unit_checkingto the toctree properly integrates the new documentation into the Sphinx structure.data_files/my_configs/config_sample.toml (1)
50-53: LGTM! Appropriate opt-in default for sample configuration.The
check_units = falsedefault in the sample config is appropriate for a new optional feature, allowing users to opt-in when ready. This contrasts appropriately with the tutorial config (which enables it to showcase the feature). The documentation clearly explains the purpose and recommended usage.temoa/model_checking/unit_checking/temoa_units.txt (3)
4-6: Custom unit definitions are appropriate for energy system modeling.The definitions of
passenger,seat, andvehicleas custom dimensional units are well-suited for transportation and energy system modeling contexts. These provide necessary units for passenger-miles, seat-miles, and vehicle-based metrics.
11-12: Currency unit definitions with appropriate documentation.The
dollarandeurocurrency definitions are correctly structured, and the comment on line 8 appropriately warns users about currency conversion limitations in Pint. The alias syntax (dollar = [currency] = USD) properly establishes both the base unit and common alias.
15-15: Changeethos = [empty]toethos = []— Pint uses empty brackets for dimensionless units.The syntax
[empty]creates a base dimension named "empty", but Pint's standard syntax for dimensionless units is[](empty brackets). Useethos = []instead to properly define ethos as a dimensionless unit.Likely an incorrect or invalid review comment.
pyproject.toml (1)
33-33: No action required. Pint 0.25.2 is the current latest release (released Nov 6, 2025) with no known security vulnerabilities. The version constraint is appropriate and valid.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
966da4a to
16e80e1
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (1)
temoa/model_checking/unit_checking/__init__.py (1)
5-9: Add defensive error handling aroundtemoa_units.txtloading.If the packaged definitions file is missing or malformed,
ureg.load_definitions(...)will currently fail at import time with a low‑signal traceback. Wrapping the resource lookup and load in a smalltry/exceptwith a clearRuntimeErrorwill make installation/config issues much easier to diagnose.For example:
-from pint import UnitRegistry - -# UnitRegistry is generic but doesn't require type args at instantiation -ureg: UnitRegistry = UnitRegistry() # type: ignore[type-arg] -# Load custom unit definitions from the package resources -data = importlib_resources.files('temoa.model_checking.unit_checking').joinpath('temoa_units.txt') -ureg.load_definitions(str(data)) +from pint import UnitRegistry +from pint.errors import DefinitionSyntaxError + +# UnitRegistry is generic but doesn't require type args at instantiation +ureg: UnitRegistry = UnitRegistry() # type: ignore[type-arg] + +# Load custom unit definitions from the package resources +try: + data = importlib_resources.files( + 'temoa.model_checking.unit_checking' + ).joinpath('temoa_units.txt') + # Ensure we have a real filesystem path (handles zipped resources too) + with importlib_resources.as_file(data) as path: + ureg.load_definitions(path) +except (FileNotFoundError, OSError, DefinitionSyntaxError) as exc: + raise RuntimeError( + f'Failed to load custom Temoa unit definitions from {data!r}' + ) from excThis keeps the global registry behavior but turns packaging problems into actionable errors instead of obscure crashes.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (5)
temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(122-124)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/relations_checker.py
152-152: Consider moving this statement to an else block
(TRY300)
174-174: Avoid specifying long messages outside the exception class
(TRY003)
195-197: Possible SQL injection vector through string-based query construction
(S608)
201-201: Possible SQL injection vector through string-based query construction
(S608)
209-209: Possible SQL injection vector through string-based query construction
(S608)
211-211: Possible SQL injection vector through string-based query construction
(S608)
213-213: Avoid specifying long messages outside the exception class
(TRY003)
220-220: Redundant exception object included in logging.exception call
(TRY401)
254-254: Avoid specifying long messages outside the exception class
(TRY003)
359-362: Avoid specifying long messages outside the exception class
(TRY003)
364-365: Possible SQL injection vector through string-based query construction
(S608)
370-372: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
415-415: Do not catch blind exception: Exception
(BLE001)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
68-68: Wrong type passed to first argument of pytest.mark.parametrize; expected tuple
Use a tuple for the first argument
(PT006)
72-72: Boolean-typed positional argument in function definition
(FBT001)
88-88: Wrong type passed to first argument of pytest.mark.parametrize; expected tuple
Use a tuple for the first argument
(PT006)
🔇 Additional comments (1)
tests/test_unit_checking_integration.py (1)
17-23: Good use of a module-level skip to gate DB-dependent integration tests.The
DBS_PRESENT+pytestmark = skipif(...)pattern cleanly avoidsFileNotFoundErrors when the test databases aren’t present, while still guiding contributors to runcreate_unit_test_dbs.py. This resolves the earlier fragility around missing fixtures.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
16e80e1 to
59c63e6
Compare
There was a problem hiding this comment.
Actionable comments posted: 8
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (6)
temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (3)
tests/create_unit_test_dbs.py (2)
temoa/_internal/data_brick.py (1)
name(50-51)temoa/utilities/graphviz_generator.py (1)
connect(173-187)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(96-102)RelationType(114-117)consolidate_lines(145-153)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/__init__.py
18-21: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
232-232: Redundant exception object included in logging.exception call
(TRY401)
266-266: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
382-384: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
temoa/model_checking/unit_checking/common.py
106-106: Boolean positional value in function call
(FBT003)
106-106: Boolean positional value in function call
(FBT003)
107-107: Boolean positional value in function call
(FBT003)
107-107: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
🔇 Additional comments (15)
tests/test_unit_checking.py (2)
1-46: LGTM! Good coverage of format validation edge cases.The test cases cover important scenarios including whitespace handling, parentheses semantics, and invalid characters. The assertion fix from the previous review has been properly implemented.
49-81: Good unit expression validation coverage.The test cases effectively cover the Pint registry validation including custom units (ethos, passenger, seat) and currency aliases (dollar, dollars, USD, EUR).
tests/create_unit_test_dbs.py (1)
32-35: Consider clearing existing test databases for idempotent runs.If a test database already exists with different content (e.g., from a previous schema version),
shutil.copywill overwrite it. This is fine, but for databases where only a single row is modified, stale modifications from prior runs won't be cleared if the script is run multiple times with a changing source database.This is minor since copying the source each time resets the state, but you might want to log when overwriting existing files.
tests/test_unit_checking_integration.py (2)
17-23: Well-implemented module-level skip condition.The
pytestmarkapproach cleanly gates all tests when databases are missing, with a helpful message guiding users to run the creation script. This properly addresses the previous review feedback.
103-117: Consider movingtest_databases_existto run first.Since other tests will fail with
FileNotFoundError(caught by the screen function) if databases are missing, having this existence check run last provides limited value. You could rename it with a leadinga_or usepytest-orderingto ensure it runs first for clearer diagnostics.-def test_databases_exist() -> None: +def test_a_databases_exist() -> None:Alternatively, keep as-is since the module-level skip already handles the missing database case.
Likely an incorrect or invalid review comment.
temoa/model_checking/unit_checking/common.py (2)
9-48: Good documentation of table categories with clear NYI markers.The commented-out tables with explanatory notes (growth/degrowth tables, output tables) provide clear documentation of scope and future work. This helps maintainability.
96-110: LGTM! NamedTuple provides clear structure for cost table metadata.The
CostTableDataNamedTuple with explicit field names makes the boolean flags self-documenting when accessed (e.g.,entry.capacity_based). The static analysis warnings about boolean positional values (FBT003) are acceptable here since NamedTuple constructors conventionally use positional arguments.temoa/model_checking/unit_checking/relations_checker.py (8)
1-31: LGTM: Imports and module setup are well-structured.The imports are appropriately organized and include all necessary dependencies for unit validation.
33-46: LGTM: Safe and correct commodity LUT construction.The function correctly filters and validates commodity units before adding them to the lookup table.
65-68: LGTM: IOUnits dataclass is appropriately designed.The frozen dataclass correctly encapsulates input and output units for a technology.
159-166: LGTM: SQL injection concern mitigated at call sites.While this helper uses f-string interpolation for table names, the validation happens at caller level (see line 184-186 in
check_inter_table_relations). Given current usage patterns, this is acceptable.
184-226: SQL injection mitigated by whitelist validation.The dynamic query construction at lines 207-223 is protected by table name validation against a whitelist (lines 184-186) and column names derived from schema introspection. While parameterized queries would be ideal, the current approach is safe given the validation layer.
334-350: LGTM: Error formatting helper is clear and consistent.The standardized error message format will help users understand validation failures.
420-458: LGTM: common_cost_unit logic correctly fixed.The restructured logic now properly validates all rows:
- First valid row establishes
common_cost_unitand continues to measure unit validation- Subsequent rows are checked against the established unit and also proceed to measure validation (unless incompatible)
This resolves the bug from the past review where subsequent rows were silently skipped.
375-378: SQL construction is safe with validated inputs.The dynamic query at lines 376-377 uses
ct.table_nameandct.commodity_referencefrom the typedCostTableDatastructure, which are controlled by the codebase rather than user input.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
59c63e6 to
cb700b6
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
temoa/model_checking/unit_checking/relations_checker.py (2)
49-62: Misleading variable name in loop.The loop variable
common line 56 is misleading since this iterates over thecapacity_to_activitytable, which usestechas the key column. Renaming totechwould improve clarity.Apply this diff:
def make_c2a_lut(conn: sqlite3.Connection) -> dict[str, Unit]: """Get a dictionary of the units for each capacity to activity entry""" res: dict[str, Unit] = {} cursor = conn.cursor() query = 'SELECT tech, units FROM capacity_to_activity' cursor.execute(query) rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return res
112-128: Use already-retrieved variables instead of redundant lookups.Lines 125 and 127 re-index
comm_units[ic]andcomm_units[oc], but these values were already retrieved asexpected_inputandexpected_outputat lines 100-101. Using the stored variables is more efficient and defensive.Apply this diff:
if invalid_input_flag: - msg += f'\n Invalid input units: {comm_units[ic]}' + msg += f'\n Invalid input units: {expected_input}' if invalid_output_flag: - msg += f'\n Invalid output units: {comm_units[oc]}' + msg += f'\n Invalid output units: {expected_output}'
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (6)
temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (2)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(122-124)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
369-372: Avoid specifying long messages outside the exception class
(TRY003)
374-375: Possible SQL injection vector through string-based query construction
(S608)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/common.py
107-107: Boolean positional value in function call
(FBT003)
107-107: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
110-110: Boolean positional value in function call
(FBT003)
110-110: Boolean positional value in function call
(FBT003)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
🔇 Additional comments (5)
temoa/model_checking/unit_checking/common.py (1)
106-112: LGTM! Cost table definitions are clear.The boolean positional arguments are well-documented by the
CostTableDataNamedTuple definition, and the inline comment on line 109 confirms a deliberate correction was made. The static analysis warning (FBT003) is a false positive in this context.temoa/model_checking/unit_checking/__init__.py (1)
1-22: LGTM! Robust initialization with proper error handling.The unit registry initialization includes comprehensive error handling for missing or malformed resource files. The use of
as_fileensures compatibility with zipped resources, and the error message provides clear diagnostics.tests/test_unit_checking.py (1)
1-100: LGTM! Comprehensive test coverage with proper assertions.The tests properly validate unit format matching, expression validation, and time dimension handling. All past review issues have been addressed, including proper assertion destructuring and tuple-style parametrize signatures.
tests/test_unit_checking_integration.py (1)
1-117: LGTM! Well-structured integration tests.The module-level skip marker ensures tests fail gracefully when test databases are missing, and the coverage includes all key error scenarios. The test structure is clear and maintainable.
tests/create_unit_test_dbs.py (1)
1-110: LGTM! Robust test database generation.The script properly uses context managers for all database operations and includes error checking for critical updates. The rowcount validation ensures the intended modifications are applied, preventing silent failures.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
cb700b6 to
2bb632c
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (2)
tests/test_unit_checking.py (1)
84-100: Minor: Consider moving the docstring to a comment above the list.The docstring on line 85 appears after the variable assignment, which is syntactically valid but unconventional. It would be clearer as a comment above the list:
+# Time dimension exponents: power units have [time]^-3, energy units have [time]^-2 time_dimension_cases = [('kW', -3), ('kWh', -2), ('PJ', -2), ('PJ/h', -3)] -"""Time dimension exponents: power units have [time]^-3, energy units have [time]^-2 """The test logic itself is correct and well-structured.
temoa/model_checking/unit_checking/relations_checker.py (1)
49-62: Rename loop variable inmake_c2a_lutfor clarity.Here the SELECT returns
tech, but the loop variable is namedcomm, which is easy to confuse with commodities and withmake_commodity_lutjust above. Renaming totechclarifies intent without changing behavior.- rows = cursor.fetchall() - for comm, units in rows: + rows = cursor.fetchall() + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (6)
temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (2)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(122-124)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(30-244)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/common.py
107-107: Boolean positional value in function call
(FBT003)
107-107: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
108-108: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
109-109: Boolean positional value in function call
(FBT003)
110-110: Boolean positional value in function call
(FBT003)
110-110: Boolean positional value in function call
(FBT003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
283-283: Unpacked variable is_valid is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
369-372: Avoid specifying long messages outside the exception class
(TRY003)
374-375: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (8)
tests/create_unit_test_dbs.py (1)
1-103: LGTM! Test database generation script is well-structured.The script properly:
- Uses context managers for all SQLite connections (addressing previous feedback)
- Validates rowcount for the complex UPDATE query (lines 81-85)
- Checks source database existence before proceeding
- Provides clear output messages and usage hints
The static analysis hint (TRY003) about the long exception message on lines 82-85 is acceptable here since this is a test setup script where a descriptive error message aids debugging.
temoa/model_checking/unit_checking/__init__.py (1)
1-22: LGTM! Unit registry initialization with proper error handling.The implementation addresses all previous feedback:
- Type ignore rationale documented (line 6)
_resource_pathdefined before try block to avoidUnboundLocalError(line 10)- Proper error handling for
FileNotFoundError,OSError, andDefinitionSyntaxError- Uses
as_filecontext manager to handle zipped resources correctlyThe TRY003 static analysis hint is acceptable here since descriptive error messages are critical for diagnosing installation issues.
tests/test_unit_checking.py (2)
36-46: LGTM! Format validation test correctly asserts on the boolean result.The test now properly destructures the return value and asserts the validity against the expected boolean, addressing the previous feedback about the broken assertion.
51-82: LGTM! Expression validation tests are comprehensive.Good coverage of:
- Standard units (kg, m/s, kWh)
- Custom Temoa units (ethos, passenger, seat)
- Currency units with aliases (dollar/dollars/USD, EUR)
- Invalid units (dog_food)
The static analysis hint (FBT001) about boolean positional arguments is acceptable for pytest parametrized tests.
tests/test_unit_checking_integration.py (2)
17-23: LGTM! Module-level skip properly gates all tests.The
pytestmarkapproach cleanly skips all tests when databases are missing, with a helpful message directing developers to run the database creation script.
26-117: LGTM! Comprehensive integration test coverage.The tests cover all seven database variants created by
create_unit_test_dbs.py:
- Valid units (baseline)
- Invalid currency dimension
- Energy units in capacity tables
- Missing ratio parentheses
- Unknown/unregistered units
- Mismatched tech outputs
- Bad composite currency
The
test_databases_existfunction provides a helpful sanity check that all expected test databases are present.temoa/model_checking/unit_checking/common.py (2)
131-143: LGTM! Regex patterns are well-documented.The patterns correctly handle:
SINGLE_ELEMENT: Captures unit expressions with optional whitespace, exponents, and divisionsRATIO_ELEMENT: Specifically captures ratio format with parenthesized denominatorThe inline comments and docstrings clearly explain the expected formats.
146-154: LGTM! Helper function for readable line number output.The function correctly truncates long lists of line numbers to prevent verbose error messages, showing at most 5 numbers with a count of remaining items.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
2bb632c to
3e04463
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (7)
temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (3)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(129-132)temoa/model_checking/unit_checking/relations_checker.py (5)
check_cost_tables(353-518)check_efficiency_table(71-156)check_inter_table_relations(169-331)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(29-129)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(251-310)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/screener.py
97-97: Unused function argument: c2a_units
(ARG001)
265-265: Avoid specifying long messages outside the exception class
(TRY003)
319-319: datetime.datetime.now() called without a tz argument
(DTZ005)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.13)
🔇 Additional comments (6)
temoa/model_checking/unit_checking/__init__.py (1)
1-22: LGTM! Error handling and resource loading are well-implemented.The error handling properly addresses the potential issues flagged in previous reviews:
_resource_pathis defined before the try block, avoiding UnboundLocalError- Specific exceptions are caught (FileNotFoundError, OSError, DefinitionSyntaxError)
- Clear error message guides users to diagnose installation issues
- Type ignore comment is properly documented
The TRY003 static analysis warning about long messages is acceptable here since this is a critical initialization failure that benefits from a detailed error message.
tests/test_unit_checking.py (1)
1-100: LGTM! Test structure is comprehensive and past issues are resolved.The tests properly cover:
- Unit format validation (SINGLE_ELEMENT and RATIO_ELEMENT patterns)
- Unit expression validation against the Pint registry
- Time dimension checking for power vs energy units
Past critical issue with the broken assertion (always passing due to truthy tuple) has been properly fixed in commit 16e80e1. The FBT001 static analysis warnings about boolean positional arguments in test functions are acceptable for pytest test functions.
tests/create_unit_test_dbs.py (1)
1-110: LGTM! Database creation script is well-structured with proper error handling.The script properly addresses past review comments:
- All database connections use context managers (lines 41, 49, 57, 66, 74, 92)
- Critical UPDATE statement (lines 74-86) includes rowcount verification to catch silent failures
- Clear error messages guide users when failures occur
The TRY003 warning about the long RuntimeError message (lines 82-85) is acceptable since this is a test setup failure that benefits from a detailed explanation.
temoa/model_checking/unit_checking/screener.py (1)
30-310: Well-structured screening workflow with clear separation of concerns.The refactoring to extract helper functions (_check_db_version, _check_units_entries, etc.) addresses the previous review comment about modularity. Each helper:
- Has a clear, single responsibility
- Returns a boolean success indicator
- Appends structured messages to report_entries
- Includes proper docstrings
The five-stage screening process is easy to follow and the early-exit behavior on DB version failure is appropriate.
The DTZ005 warning about timezone-naive datetime (line 319) is acceptable for local file timestamps.
tests/test_unit_checking_integration.py (1)
1-117: LGTM! Integration tests are comprehensive with proper skip guards.The module-level
pytestmark(lines 20-23) properly addresses the previous review comment about missing skip conditions. All tests will now skip gracefully when test databases are not present, with clear guidance to run the database creation script.Test coverage is thorough, validating:
- Valid units (baseline)
- Invalid currency dimensions
- Energy vs capacity unit mismatches
- Format violations (missing parentheses)
- Unknown/unregistered units
- Mismatched tech output units
- Nonsensical composite units (rejected)
- Valid composite units (accepted)
temoa/model_checking/unit_checking/common.py (1)
1-168: LGTM! Common definitions are well-structured and past issues are resolved.The module properly addresses previous review comments:
consolidate_lines(line 160) usesSequence[int]for flexibility while maintaining type safetyCostTableDatainstantiations (lines 107-124) use keyword arguments for clarity, making the boolean flags self-documentingThe organization is logical:
- Table categorizations are clearly documented
- Type definitions (CostTableData, RelationType, UnitsFormat) are properly structured
- Constants (SINGLE_ELEMENT, RATIO_ELEMENT) provide reusable patterns
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
3e04463 to
365f5f8
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
temoa/model_checking/unit_checking/relations_checker.py (1)
49-62: Rename loop variable inmake_c2a_lutfor clarity
capacity_to_activityreturns(tech, units), but the loop currently unpacks intofor comm, units in rows:and keys the LUT bycomm. This works functionally, but the name is misleading in a tech-based table and can confuse future readers.Renaming to something like
tech(and usingres[tech]) would better reflect the schema and match the surrounding comments.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (7)
temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (2)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(251-308)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/screener.py
265-265: Avoid specifying long messages outside the exception class
(TRY003)
317-317: datetime.datetime.now() called without a tz argument
(DTZ005)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (6)
temoa/model_checking/unit_checking/__init__.py (1)
6-22: Unit registry initialization and error handling look appropriateGlobal
uregcreation plus theimportlib_resources.as_fileblock gives a single, shared registry and works with zipped resources. The targeted exception handling and RuntimeError message make failures easy to diagnose; no changes needed here.tests/test_unit_checking.py (1)
17-100: Tests correctly exercise format, expression, and time-dimension handlingThe tests now assert on the boolean validity flag rather than the whole return tuple, and they check both the resolved Pint Units and the time exponent where relevant. This closes the earlier gap in negative coverage and provides solid regression tests for the unit parser.
tests/create_unit_test_dbs.py (1)
32-103: Deterministic and safe creation of unit-test databasesThe script cleanly creates all DB variants using
with sqlite3.connect(...)blocks, and the explicitrowcountcheck for the mismatched-output case ensures test setup fails fast if the expected record is missing. This is a solid basis for the integration tests.tests/test_unit_checking_integration.py (1)
26-117: Integration coverage for all DB variants with safe skipping behaviorThe suite exercises each generated database through
screen, asserting the expected pass/fail outcome and verifying composite-currency acceptance on the valid DB. The module-level skip condition cleanly handles missing fixtures while still documenting how to create them. This is a solid integration layer on top of the unit-checking infrastructure.temoa/model_checking/unit_checking/common.py (1)
10-168: Shared unit-checking metadata and helpers are consistent with the v4 designThe table groupings,
CostTableDatadefinitions, and unit-format regexes line up with how the checkers and tests use them, andconsolidate_linesis correctly generalized toSequence[int]. This file forms a clear, maintainable backbone for the rest of the unit-checking code.temoa/model_checking/unit_checking/relations_checker.py (1)
239-303: Confirm behavior whenunitscolumns are NULL in inter-table checks
check_inter_table_relationspassestable_unitsandc2a_unitsdirectly intovalidate_units_format. Given the v4 schema keeps newunitscolumns nullable for backward compatibility, it’s likely that some deployments will haveNULLin these fields for a while.This is safe only if
validate_units_formatexplicitly handlesNone(e.g., treating it as invalid and returning(False, None)) rather than assuming astr. If it doesn’t, runningscreen()against a partially-updated database could hit aTypeErrorinstead of reporting structured “invalid format” errors.Please double-check that
validate_units_format(and, transitively,validate_units_expression) are robust toNoneinputs, or guard these calls with a simpleif table_units is None:/if c2a_units is None:branch that records an “invalid/missing units” error and continues.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
365f5f8 to
fe3f9ef
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
temoa/_internal/temoa_sequencer.py (1)
71-99: Unit-check integration in_run_preliminary_checkslooks solidThe gating on
config.check_units, use ofscreen(...)with a dedicatedunit_check_reportssubdirectory, and non-fatal behavior on failures all align well with the described workflow. No functional issues stand out.If you want to trim redundancy later, note that the database version is now checked here via
check_database_version(...)and again insidebuild_model(), so you could rely on just the pre-flight check in a follow-up cleanup.
♻️ Duplicate comments (2)
temoa/model_checking/unit_checking/relations_checker.py (2)
56-56: Rename misleading loop variable.Line 56 unpacks the tech value into a variable named
comm, which is misleading since this function queries thecapacity_to_activitytable (which uses atechcolumn), not thecommoditytable.Apply this diff:
rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return res
368-378: Add validation forct.table_nameandct.commodity_referencebefore SQL interpolation.Lines 376-377 construct SQL with unvalidated identifiers via f-string. While current usage is safe (callers pass the static
cost_based_tablesconstant), the function signature accepts arbitraryIterable[CostTableData], creating a potential injection vector if future callers pass externally-influenced data.Based on learnings, SQL injection vulnerabilities in database utility classes are considered low concern in this project, but adding defensive validation would improve safety. Consider either:
- A strict identifier regex:
^[A-Za-z_][A-Za-z0-9_]*$- A whitelist check against known cost tables/columns (similar to
check_inter_table_relationsat lines 184-186)Apply validation before the query construction:
def check_cost_tables( conn: sqlite3.Connection, cost_tables: Iterable[CostTableData], tech_lut: dict[str, IOUnits], c2a_lut: dict[str, Unit], commodity_lut: dict[str, Unit], ) -> list[str]: + import re + common_cost_unit = None error_msgs = [] for ct in cost_tables: + # Validate table name and column name to prevent SQL injection + if not re.match(r'^[A-Za-z_][A-Za-z0-9_]*$', ct.table_name): + raise ValueError(f'Invalid table name: {ct.table_name}') + if ct.commodity_reference and not re.match(r'^[A-Za-z_][A-Za-z0-9_]*$', ct.commodity_reference): + raise ValueError(f'Invalid column name: {ct.commodity_reference}') + table_grouped_errors = defaultdict(list) if ct.commodity_reference and ct.capacity_based:
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (9)
temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (5)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-392)temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/screener.py
259-259: Avoid specifying long messages outside the exception class
(TRY003)
309-309: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (6)
temoa/model_checking/unit_checking/__init__.py (1)
6-22: GlobalUnitRegistryinitialization and resource loading look correctThe shared
ureginstance plusas_file(...)-based loading oftemoa_units.txtis robust, and the narrowed exception handling around missing/bad definitions provides a clear, actionable RuntimeError. No changes needed here.tests/test_unit_checking.py (1)
17-100: Unit-checking tests now exercise the right semanticsSwitching
test_format_validationto assert on the boolean result and adding explicit unit and time-dimension checks gives good coverage of both regex parsing and registry behavior. The parametrization is clear and matches the intended edge cases.temoa/cli.py (1)
292-392:check-unitsCLI wiring is consistent and user-friendlyArgument validation, defaulting/creation of the report directory, invocation of
screen(...), and the report-summary logic all look correct and align with the screener’s API. Error handling viatyper.Exitis appropriate, and keeping errors visible even withsilent=Trueis a reasonable choice.tests/create_unit_test_dbs.py (1)
14-110: Test database variant creation is deterministic and fails fast on schema driftUsing context managers for all SQLite updates plus the explicit
rowcountcheck for the E01/ELC efficiency row makes the fixture generation robust and easy to diagnose if the tutorial DB ever changes. This should keep the integration tests stable.temoa/model_checking/unit_checking/common.py (1)
10-168: Shared unit-checking metadata and formats are coherent and appropriately typedThe table categorizations,
CostTableDatadefinition, unit-format regexes, andconsolidate_lineshelper form a clear, reusable contract for the rest of the unit-checking stack. The move toSequence[int]forconsolidate_linesimproves flexibility without changing behavior.tests/test_unit_checking_integration.py (1)
14-117: Integration tests cleanly exercise all DB variants and are safely gatedThe module-level
pytestmarkskip, one-test-per-variant structure, and the explicit existence check intest_databases_existtogether give good end-to-end coverage of the screener while avoiding brittle failures when the test DBs haven’t been generated. This looks ready to ship.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
fe3f9ef to
0869ffa
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
♻️ Duplicate comments (3)
temoa/model_checking/unit_checking/screener.py (1)
305-313: Consider timezone-aware timestamps for report filenames.The
datetime.now()call on line 309 creates a timezone-naive timestamp. This is fine for local development, but could cause confusion in multi-timezone deployments or when correlating logs across systems.Consider making the timestamp timezone-aware:
def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S') report_dir.mkdir(parents=True, exist_ok=True) report_file_path = report_dir / f'units_check_{timestamp}.txt'This would make report timestamps unambiguous and consistent across deployments.
temoa/model_checking/unit_checking/relations_checker.py (2)
49-62: Loop variable namecommis misleading for capacity_to_activity table.This was flagged in a previous review but not addressed. The query selects
techfromcapacity_to_activity, but the loop unpacks it ascomm, which is confusing.Apply this diff for clarity:
rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return res
368-378: Missing validation forct.table_nameandct.commodity_referencebefore SQL interpolation.Unlike
check_inter_table_relationswhich validatestable_nameagainst a whitelist (lines 184-186), this function directly interpolatesct.table_nameandct.commodity_referenceinto SQL without validation. While currently safe (callers pass the staticcost_based_tables), the function signature accepts arbitraryIterable[CostTableData].Add validation consistent with
check_inter_table_relations:+ # Validate against known cost tables + valid_cost_tables = {ctd.table_name for ctd in cost_based_tables} + valid_commodity_refs = {'emis_comm', None} # Known commodity reference columns + for ct in cost_tables: + if ct.table_name not in valid_cost_tables: + raise ValueError(f'Invalid cost table name: {ct.table_name}') + if ct.commodity_reference not in valid_commodity_refs: + raise ValueError(f'Invalid commodity reference: {ct.commodity_reference}') + table_grouped_errors = defaultdict(list)Note: You'll need to import
cost_based_tablesfrom common if not already available in scope for the validation set.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (21)
data_files/my_configs/config_sample.toml(1 hunks)data_files/temoa_schema_v4.sql(6 hunks)docs/source/index.rst(1 hunks)docs/source/unit_checking.rst(1 hunks)pyproject.toml(1 hunks)requirements-dev.txt(3 hunks)requirements.txt(3 hunks)temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/core/config.py(3 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/entry_checker.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)temoa/model_checking/unit_checking/temoa_units.txt(1 hunks)temoa/tutorial_assets/config_sample.toml(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2025-10-27T15:53:41.829Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 177
File: temoa/model_checking/commodity_network.py:26-33
Timestamp: 2025-10-27T15:53:41.829Z
Learning: The Temoa project requires Python 3.12 or above as the minimum supported version, so PEP 695 `type` syntax for type aliases is appropriate and preferred over `TypeAlias`.
Applied to files:
requirements.txtrequirements-dev.txt
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (8)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
gather_from_table(52-64)validate_units_format(24-36)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/entry_checker.py (1)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/core/config.py (1)
temoa/cli.py (1)
check_units(293-392)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-392)temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/table_checker.py
128-128: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/entry_checker.py
47-47: Consider moving this statement to an else block
(TRY300)
58-58: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/screener.py
259-259: Avoid specifying long messages outside the exception class
(TRY003)
309-309: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/core/config.py
62-62: Boolean-typed positional argument in function definition
(FBT001)
62-62: Boolean default positional argument in function definition
(FBT002)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
🔇 Additional comments (26)
data_files/temoa_schema_v4.sql (3)
1-1118: AI summary is incomplete; describes only 6 of 28 table schema changes.The AI-generated summary states "Added a new column units TEXT to six tables," but the provided code shows schema updates to approximately 28 tables across three categories: (1) core data tables (commodity, efficiency, demand, cost_, emission_, end_of_life_output, etc.), (2) limit constraint tables with both
unitsandseed_unitscolumns, and (3) lifetime/capacity metadata tables. This aligns with the PR summary claim of 28 tables.
100-118: Schema updates maintain backwards compatibility and consistency.The additions across all 28 tables follow a uniform pattern: new nullable TEXT columns for units metadata, consistently positioned before the
notescolumn. This design avoids breaking existing queries or imposing constraints on legacy data. The commodity table correctly adapts the trailing comma afterdescription(line 116) to accommodate the newunitscolumn (line 117).
352-396: Migration script automatically handles schema changes for all 28 modified tables.The migration programmatically copies data from v3.1 to v4 by applying the complete v4 schema (which includes all 28 tables with
unitsorseed_unitscolumns) and selectively inserting columns that exist in both old and new schemas. The 6limit_*_capacityvariants are new in v4 and don't exist in v3.1, so theirseed_unitscolumns will be NULL/empty after migration—this is expected behavior, not a coverage gap. The migration is verified to run viatemoa migratecommand in the CLI.data_files/my_configs/config_sample.toml (1)
50-53: Opt‑outcheck_unitsflag in generic sample config looks appropriatePlacing
check_units = falsewith clear commentary alongside the other data/model checks keeps existing behavior unchanged while advertising the new unit‑checking capability for users who are ready to populate units.requirements-dev.txt (1)
30-33: Pint‑related dev dependencies are consistent withpyproject.tomland uv workflowThe additions of
flexcache,flexparser,pint,platformdirsand the updatedtyping-extensionsprovenance look consistent with the Pint dependency tree and withpint>=0.25.2inpyproject.toml. Given this file is uv‑generated, just ensure future dependency changes continue to go throughuv pip compileso these entries stay in sync.Also applies to: 105-108, 200-205
requirements.txt (1)
15-18: Runtime Pint dependency pins and transitive requirements look correctThe new entries for
flexcache,flexparser,pint==0.25.2, andplatformdirsmatch the unit‑checking feature and are in sync withpyproject.tomland the dev requirements. Since this file is uv‑compiled, continuing to manage versions viapyproject.toml+uv pip compileshould keep everything coherent.Also applies to: 68-71, 118-122
docs/source/index.rst (1)
4-7: Addingunit_checkingto the main toctree makes the new docs discoverableThe new
unit_checkingentry is correctly indented under the existing toctree and will pull the unit‑checking documentation into the main navigation, assuming the correspondingunit_checking.rstfile is present.temoa/tutorial_assets/config_sample.toml (1)
49-52: Enablingcheck_unitsin the tutorial config aligns with the new workflowTurning
check_units = trueon for the tutorial configuration (with clear explanatory comments) is a good default: it exercises the unit‑checking pipeline on the reference database while leaving the generic sample config opt‑out.pyproject.toml (1)
33-34: Review comment is based on non-existent code and cannot be verifiedThe review comment references a unit-checking subsystem that does not exist in the repository:
temoa/model_checking/unit_checking/__init__.pydoes not existtemoa_units.txtis not present anywhere in the codebase- No code uses
UnitRegistryorimportlib.resourcesto load unit definitionsThe pint dependency addition at line 33 is present (
"pint>=0.25.2"), but the code that would consume it has not been implemented. The package-data recommendation formodel_checking/unit_checking/temoa_units.txtaddresses infrastructure for code that is not yet in this repository.Likely an incorrect or invalid review comment.
temoa/model_checking/unit_checking/temoa_units.txt (1)
1-15: LGTM! Unit definitions are clean and well-documented.The custom unit definitions appropriately extend Pint's registry for Temoa-specific needs. The warning about currency conversion (line 8) is helpful to prevent users from expecting automatic exchange rate handling.
temoa/_internal/temoa_sequencer.py (1)
71-89: Unit check integration looks good, but clarify whether failures should block execution.The implementation correctly gates the unit check on
config.check_unitsand logs outcomes appropriately. The report directory path is constructed correctly. However, there's a design question:Currently, unit check failures log a warning but do not set
checks_ok = False(line 91), meaning the model build proceeds regardless of unit check results. This is fine if unit checking is purely advisory, but if it's meant to catch critical input errors, you may want to consider:
- Making it blockable via a config flag (e.g.,
check_units_strict: bool)- Documenting in the config/docs that unit check failures are non-blocking
The screener's
_write_reportalready handles directory creation defensively withmkdir(parents=True, exist_ok=True), so no action needed there.Do unit check failures represent critical input errors that should halt execution, or are they advisory warnings? If the former, consider adding a strict mode; if the latter, please document this behavior in the configuration docs.
docs/source/unit_checking.rst (1)
1-407: Excellent comprehensive documentation for the unit checking feature.The documentation is well-structured and covers all essential aspects:
- Clear warnings about format requirements (especially parentheses for ratios)
- Comprehensive troubleshooting section
- Good examples of common pitfalls
- Testing guidance with code snippets
This will be very helpful for users adopting the unit checking feature.
tests/test_unit_checking.py (1)
1-100: Test coverage is comprehensive and well-structured.The test suite properly validates:
- Unit format regex matching (SINGLE_ELEMENT and RATIO_ELEMENT)
- Registry membership for various unit expressions
- Time dimension exponents for power vs. energy units
The parametrized test approach is clean and makes it easy to add new test cases. Past review comments have been addressed (assertion destructuring, tuple parametrize format).
temoa/model_checking/unit_checking/entry_checker.py (1)
52-64: SQL injection warning is a false positive, but consider a minor refactor for exception handling.The static analysis tool flags line 58 for potential SQL injection due to f-string query construction. However, this is a false positive because:
- The
tableparameter comes from hardcoded module constants (tables_with_unitsincommon.py)- Table names are never sourced from user input
For the TRY300 hint on line 47 in
validate_units_expression, the suggestion to movereturn False, Noneto anelseblock is purely stylistic and optional.temoa/cli.py (1)
292-392: CLI command implementation is solid with good error handling.The
check-unitscommand follows existing CLI patterns and includes:
- Proper path validation via typer annotations
- Defensive directory creation with error handling
- Clear user feedback with rich formatting
- Helpful report summary display
The logic to show the first 40 lines or until a double newline (lines 372-379) is a reasonable approach for report summaries.
temoa/core/config.py (1)
62-62: Configuration parameter addition follows existing patterns.The new
check_unitsparameter is properly integrated:
- Defaults to
False(opt-in behavior)- Follows the same pattern as other boolean flags like
price_checkandsource_trace- Added to
__repr__for visibilitytemoa/model_checking/unit_checking/screener.py (1)
247-302: Screen function is well-structured and correctly implements the five-stage check.The orchestration of checks is clean:
- Proper early exit on DB version failure
- Correct aggregation of failures via
all_clearflag- Report writing is conditional and defensive
The helper functions extracted in past reviews make the code maintainable and testable.
temoa/model_checking/unit_checking/table_checker.py (1)
29-34: LGTM on function signature and initialization.The function signature with typed return
tuple[dict[str, Unit], list[str]]is clear, and the initialization oferrorsandresis appropriate.temoa/model_checking/unit_checking/common.py (3)
145-157: Regex patterns are well-documented and correctly structured.The
SINGLE_ELEMENTandRATIO_ELEMENTpatterns are appropriately designed:
SINGLE_ELEMENTcaptures a single unit expression with allowed charactersRATIO_ELEMENTrequires the denominator to be parenthesized, which is a clear design choice documented in the docstringThe accompanying
ACCEPTABLE_CHARACTERSpattern aligns withSINGLE_ELEMENTfor preliminary validation.
97-125: Well-structured cost table metadata.The
CostTableDataNamedTuple with keyword argument instantiation provides clear, self-documenting configuration for each cost table's properties. The field semantics (commodity_reference,capacity_based,period_based) appropriately capture the validation requirements.
160-168: LGTM onconsolidate_linesimplementation.The function correctly truncates long lists while providing helpful context about remaining items. The
Sequence[int]type hint appropriately generalizes the input type.tests/test_unit_checking_integration.py (2)
20-23: LGTM on module-level skip condition.The
pytestmarkapproach cleanly applies the skip condition to all tests in the module, addressing the previous review feedback about inconsistent skip handling.
103-117: LGTM on comprehensive fixture verification.This test provides useful diagnostic information when test databases are partially present, helping developers identify which specific fixtures need to be regenerated.
temoa/model_checking/unit_checking/relations_checker.py (3)
86-97: LGTM on efficiency table validation flow.The early-exit pattern (
if not valid ... continue) correctly handles invalid rows without attempting further validation. The reuse of thevalidflag across output and input validation is intentional to short-circuit on first failure.
183-186: Good defensive validation of table name against whitelist.The validation at lines 184-186 ensures only known table names are used in SQL queries, mitigating injection concerns per project policy.
501-511: LGTM on measure unit mismatch reporting.The error message correctly uses
tech(the actual row value) rather than the column name, providing actionable diagnostic information.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
0869ffa to
2124d76
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (3)
temoa/model_checking/unit_checking/relations_checker.py (3)
56-61: Misleading loop variable name:commshould betech.The query selects from
capacity_to_activitywith columnstech, units, but the loop unpacks intocomm, units. This is confusing since the value represents a technology, not a commodity.Apply this diff:
rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return res
159-166: Minor: Exception handling structure.Static analysis suggests moving
return column in columnsto anelseblock, but the current implementation is clear and functionally correct.
375-377: Consider defensive validation for SQL identifiers (unaddressed from prior review).The function interpolates
ct.table_nameandct.commodity_referencedirectly into SQL. While current usage only passes staticcost_based_tablesdata, the function signature accepts arbitraryIterable[CostTableData].Based on learnings, SQL injection in Temoa's database utilities is considered low concern due to data access patterns. However, for consistency with
check_inter_table_relations(which validates against a whitelist at line 184-186), consider adding similar validation here for defensive programming.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (10)
temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (5)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-64)validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/table_checker.py
128-128: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/screener.py
259-259: Avoid specifying long messages outside the exception class
(TRY003)
309-309: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (13)
temoa/_internal/temoa_sequencer.py (1)
71-92: Unit-check integration looks good; confirminput_databaseis always aPath.The pre-flight integration and logging behavior here are solid and match the intended “advisory-only” semantics (warnings but no change to
checks_ok). One thing to double‑check:screen()expectsPathobjects and calls.is_file()ondb_path, soself.config.input_databasereally needs to be aPath, not a raw string. If there are any callers constructingTemoaConfigmanually, consider normalizinginput_databasetoPathinTemoaConfig.build_config/__post_init__to make this guarantee explicit.temoa/model_checking/unit_checking/__init__.py (1)
1-22: Global unit registry initialization is robust and clear.The shared
uregplusimportlib.resources.as_fileloading oftemoa_units.txtwith targeted exception handling is a solid, fail‑fast setup. The inline comment on thetype: ignorealso documents the mypy suppression well.tests/test_unit_checking.py (1)
17-100: Unit-format and registry tests give good coverage.The combination of
cases,expression_cases, andtime_dimension_casesexercises both the regex formats and Pint dimensionality behavior in a focused way, and the tests now correctly assert on the boolean/Unit parts of the helpers.tests/create_unit_test_dbs.py (1)
14-110: Test DB generator is well-structured and exception-safe.The use of context-managed
sqlite3.connectcalls, clear per-variant comments, and therowcountguard on the E01/ELC update make this script robust and easy to debug; it cleanly supports the integration tests’ scenarios.temoa/model_checking/unit_checking/screener.py (1)
30-314: Screening pipeline and report handling look consistent with the 5‑stage design.The decomposition into per-stage helpers and the
screen(*db_paths, report_dir=...)orchestrator is clear, with correct early exit on DB-version failure and sensible aggregation ofall_clearacross databases. Report writing via_write_reportmatches the CLI and sequencer expectations.tests/test_unit_checking_integration.py (1)
14-107: Integration tests comprehensively exercise unit-checking scenarios.The module-level skip guard plus the per-DB tests for valid/invalid currencies, capacity vs energy units, ratio format, unknown units, mismatched outputs, and composite currencies give good end-to-end coverage of
screen()against the generated Utopia variants.temoa/model_checking/unit_checking/common.py (1)
1-168: LGTM! Well-structured utility module for unit checking.The module provides a clean, well-documented set of shared types and constants. The categorization of tables by their unit-checking requirements is logical and clearly documented with explanatory comments for excluded tables.
temoa/model_checking/unit_checking/relations_checker.py (6)
1-31: LGTM!Imports are well-organized and all used within the module.
33-46: LGTM!The lookup table construction correctly validates both format and expression before storing, gracefully skipping invalid entries.
65-156: LGTM!The efficiency table checker is well-implemented with robust error handling:
- Safe dictionary lookups with
.get()- Clear error messages distinguishing expected vs found units
- Consistency validation across same-tech rows
- Consolidated reporting for invalid rows
169-331: LGTM!The inter-table relation checker is well-designed with:
- Whitelist validation for table names (mitigating SQL injection concerns)
- Robust column existence checks for schema variations
- Graceful fallbacks when required tables/columns are missing
- Correct unit conversion logic for capacity-based tables
334-351: LGTM!Clean helper function for consistent, well-formatted error messages.
353-516: LGTM! Cost table validation logic is now correct.The restructured
common_cost_unitlogic properly separates initialization (line 434-435) from validation (lines 437-458), ensuring all rows are checked for consistency. The measure unit validation correctly handles:
- Commodity-referenced vs tech-referenced tables
- Capacity-based adjustments with C2A conversion
- Period-based adjustments
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
2124d76 to
32873bd
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (3)
temoa/model_checking/unit_checking/table_checker.py (1)
101-117: Fix capacity-table time-dimension check:.realaccessor and comparison threshold.Two issues remain from past reviews:
.realaccessor incompatibility: Pint's dimensionality can returnFractionwhich lacks a.realattribute. This will raiseAttributeError. Usefloat(time_exponent)directly.Comparison threshold
> 0misses energy units: Pertests/test_unit_checking.py, energy units (kWh,PJ) have time exponent-2, while power units (kW,GW) have-3. The current> 0check catches nothing since both are negative. The comments on lines 105-107 are also incorrect (they claimkWhhastime=1).Apply this diff:
- # Positive time exponent indicates energy (capacity * time), not capacity - # Examples: GW*year (time=1), kWh (time=1) are WRONG for capacity - # Examples: GW (time=-3), MW (time=-3) are CORRECT for capacity - # Convert to float for comparison (dimensionality can be complex) - if float(time_exponent.real) > 0: + # Capacity tables should use power-like units (time exponent -3). + # Energy units (time exponent -2 or greater) are incorrect. + # Examples: kWh (time=-2), GWh (time=-2) are WRONG for capacity + # Examples: kW (time=-3), GW (time=-3) are CORRECT for capacity + # Convert to float for comparison + if float(time_exponent) > -3:temoa/model_checking/unit_checking/relations_checker.py (1)
49-62: Rename loop variable for clarity.Line 56 uses
commto unpack the tech value, which is misleading since this is thecapacity_to_activitytable where the first column istech, not a commodity. This was flagged in a previous review but appears unaddressed.Apply this diff:
rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return restemoa/model_checking/unit_checking/screener.py (1)
305-313: Consider using timezone-aware timestamp for report filenames.The
datetime.now()call lacks a timezone argument, which could cause ordering issues in cross-timezone environments. This is a minor polish item.Apply this diff:
def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S_UTC') report_dir.mkdir(parents=True, exist_ok=True) report_file_path = report_dir / f'units_check_{timestamp}.txt' with open(report_file_path, 'w', encoding='utf-8') as report_file: report_file.writelines(report_entries)
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (10)
temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (6)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-395)temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-64)validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
🪛 Ruff (0.14.7)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/screener.py
259-259: Avoid specifying long messages outside the exception class
(TRY003)
309-309: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/model_checking/unit_checking/table_checker.py
128-128: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (19)
temoa/_internal/temoa_sequencer.py (1)
73-88: LGTM: Unit checking integration with soft-failure behavior.The unit checking is properly gated behind
config.check_units, uses a lazy import to avoid loading the unit-checking subsystem when not needed, and correctly continues with model build even when unit checks fail (logging warnings rather than aborting). This aligns with the PR objective of making unit checking opt-in and non-blocking.temoa/model_checking/unit_checking/__init__.py (1)
1-22: LGTM: Robust unit registry initialization with proper error handling.The implementation correctly:
- Uses
_resource_pathto avoidUnboundLocalErrorin the exception handler- Handles zipped resources via
importlib_resources.as_file- Catches the appropriate exception types (
FileNotFoundError,OSError,DefinitionSyntaxError)- Provides a clear error message with the resource path for debugging
The Ruff TRY003 warning about long exception messages can be ignored per the repo's linting configuration.
tests/test_unit_checking.py (1)
1-100: LGTM: Well-structured unit tests with comprehensive coverage.The test module properly validates:
- Format regex matching for
SINGLE_ELEMENTandRATIO_ELEMENTpatterns- Unit registry expression validation against both known and unknown units
- Time dimension exponent verification for power vs energy units
Past review concerns have been addressed: the assertion now correctly destructures the return value,
pytest.mark.parametrizeuses tuple form, and time dimension cases include explanatory documentation.The Ruff FBT001 warnings about boolean positional arguments are false positives for test functions—boolean test parameters are standard pytest practice.
temoa/cli.py (1)
292-395: LGTM: CLI command properly returns non-zero exit code on failure.The
check-unitscommand correctly:
- Returns exit code 1 when unit checks fail (lines 394-395)
- Handles
FileNotFoundErrorand general exceptions with exit code 1- Creates output directory defensively with
exist_ok=True- Provides informative console output with report summary
The past review concern about missing non-zero exit codes has been addressed. The Ruff FBT002 warning is a false positive for Typer option defaults.
tests/create_unit_test_dbs.py (1)
14-103: LGTM: Test database generator with robust error handling.The implementation correctly:
- Uses context managers for all SQLite connections (exception-safe)
- Validates row modification with
cursor.rowcountcheck (lines 81-85)- Creates a comprehensive set of test variants covering different unit error scenarios
- Provides clear output and pytest run instructions
All past review concerns have been addressed. The Ruff TRY003 warning for the long exception message can be ignored per repo configuration.
temoa/model_checking/unit_checking/table_checker.py (1)
119-128: LGTM: Result assembly logic is correct.The result dictionary construction properly handles both
SINGLE_ELEMENT(single unit) andRATIO_ELEMENT(numerator/denominator division) formats. The ValueError for unknown formats now uses an f-string as previously suggested.temoa/model_checking/unit_checking/screener.py (2)
247-302: LGTM!The
screen()function correctly orchestrates the five-stage validation process with proper early exit on DB version failure and aggregates results across multiple databases. The report writing is appropriately conditional.
30-54: LGTM!The DB version check is straightforward and correctly requires v4.0+ for unit checking to proceed.
tests/test_unit_checking_integration.py (2)
17-23: LGTM!Good implementation of the module-level skip condition. This ensures all tests are skipped gracefully when test databases are missing, with a clear message guiding developers to run the creation script.
26-107: LGTM!The test suite comprehensively covers the unit checking scenarios with clear docstrings and descriptive assertion messages. Each test targets a specific validation path in the screener.
temoa/model_checking/unit_checking/common.py (4)
10-49: LGTM!The
tables_with_unitslist is well-documented with comments explaining why certain tables are excluded (e.g., growth/degrowth tables using different column names, output tables lacking units columns).
106-125: LGTM!The
CostTableDatainstances now use keyword arguments for clarity, properly addressing the previous review feedback. The table configurations appear correct for each cost type.
141-157: LGTM!The regex patterns are well-designed for parsing unit expressions. The
RATIO_ELEMENTpattern correctly enforces parentheses around the denominator to ensure unambiguous parsing of ratio expressions likeUSD / (kW * year).
160-167: LGTM!The
consolidate_linesfunction provides good UX by limiting output to 5 line numbers with a+N moreindicator for longer lists.temoa/model_checking/unit_checking/relations_checker.py (5)
71-156: LGTM!The
check_efficiency_tablefunction properly validates technology I/O units against commodity units. The.get()lookups preventKeyErrorcrashes, and error messages correctly distinguish expected vs. found values.
159-166: LGTM!The
_column_existshelper is clear and functional. The static analysis suggestion (TRY300) is a minor style preference; the current implementation is acceptable.
169-331: LGTM!The
check_inter_table_relationsfunction handles schema variations robustly with dynamic column detection and appropriate error handling. Previous review feedback regardingKeyErrorhandling, index typing, and entity labeling has been addressed.Based on learnings, the SQL injection warnings (S608) are low concern for this project's data access patterns.
410-458: LGTM!The cost commonality validation has been fixed to properly check all rows against the established common unit. The logic correctly distinguishes between incompatible units (conversion failure) and compatible but non-standard units.
460-512: LGTM!The measure unit validation correctly handles the different cost table structures: commodity-referenced tables (like
cost_emission) lookup directly from the commodity LUT, while tech-referenced tables derive expected units from the tech's output, optionally applying C2A conversion for capacity-based tables and year division for period-based tables.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
2 times, most recently
from
1e52fcd to
20419dc
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
temoa/model_checking/unit_checking/screener.py (2)
193-202: Remove redundant assignment on line 199.Same issue as in
_check_units_entries: the assignmenterrors_test4 = Trueon line 199 is redundant because the variable is alreadyTruewhen errors were detected (lines 152, 169, 186).Apply this diff:
if not errors_test4: msg = 'Units Check 4: (Related Tables): Passed' logger.info(msg) report_entries.extend((msg, '\n')) - - else: - errors_test4 = True report_entries.append('\n') return not errors_test4
78-87: Remove redundant assignment on line 84.The assignment
errors_test2 = Trueon line 84 is redundant because the variable is alreadyTruewhen the else block is reached (it was set toTrueon line 73 when errors were detected).Apply this diff:
if not errors_test2: msg = 'Units Check 2 (Units Entries in Tables): Passed' logger.info(msg) report_entries.extend((msg, '\n')) - - else: - errors_test2 = True report_entries.append('\n') return not errors_test2temoa/model_checking/unit_checking/table_checker.py (1)
100-116: Critical: Capacity table time dimension check is incorrect and misses energy units.The current condition on line 108 checks
if float(cast('float', time_exponent)) > 0, but this will not detect energy units in capacity tables. According to the physics and the test cases intests/test_unit_checking.py:
- Power units (correct for capacity):
kW,GWhave time exponent -3- Energy units (wrong for capacity):
kWh,GWhhave time exponent -2The test database
utopia_energy_in_capacity.sqlitecontainsGWh(time=-2) and expects it to be flagged as an error, but the condition> 0won't catch it.Apply this fix:
- # Positive time exponent indicates energy (capacity * time), not capacity - # Examples: GW*year (time=1), kWh (time=1) are WRONG for capacity - # Examples: GW (time=-3), MW (time=-3) are CORRECT for capacity - # Convert to float for comparison (dimensionality can be complex) - if float(cast('float', time_exponent)) > 0: + # Capacity tables should use power units (time exponent -3). + # Energy units (time exponent -2 or higher) are incorrect. + # Examples: kWh (time=-2), GWh (time=-2) are WRONG for capacity + # Examples: kW (time=-3), GW (time=-3) are CORRECT for capacity + if float(cast('float', time_exponent)) > -3:Additionally, the
cast('float', time_exponent)is redundant—float()directly handles all numeric types that Pint returns (int, float, Fraction, Decimal):- if float(cast('float', time_exponent)) > -3: + if float(time_exponent) > -3:temoa/model_checking/unit_checking/relations_checker.py (1)
49-62: Rename loop variable for clarity:comm→tech.The query selects
tech, units FROM capacity_to_activity, but the unpacking usescommwhich is misleading since this table maps technologies, not commodities.rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return res
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (10)
temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (6)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-64)validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-395)temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/screener.py
259-259: Avoid specifying long messages outside the exception class
(TRY003)
309-309: datetime.datetime.now() called without a tz argument
(DTZ005)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/table_checker.py
127-127: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.12)
🔇 Additional comments (17)
tests/test_unit_checking.py (1)
1-100: Test module is well-structured and comprehensive.The test coverage for unit checking is thorough:
- Format validation tests cover both SINGLE_ELEMENT and RATIO_ELEMENT patterns with edge cases
- Expression validation tests verify registry compliance and expected unit objects
- Time dimension tests validate physics-based dimensionality (power vs energy units)
The parametrize decorators now use the preferred tuple-of-names format, and the time dimension comment on line 85 adds helpful context. The Ruff FBT001 warnings about boolean positional arguments are false positives—these are standard pytest test parameters, not function design issues.
tests/create_unit_test_dbs.py (1)
14-103: Test database generator is robust and well-documented.The script creates seven database variants systematically, each with a specific unit error scenario for testing. Good practices:
- Uses
sqlite3.connectcontext managers for automatic cleanup- Validates source database existence with clear error messaging
- Includes explicit
rowcountcheck for the E01/ELC update to catch schema drift- Provides helpful console output with pytest run instructions
The Ruff TRY003 warning about the long exception message is stylistic—since you've chosen to ignore this rule repo-wide, no action needed.
temoa/_internal/temoa_sequencer.py (1)
71-89: Unit checking integration is clean and appropriately gated.The integration follows a sensible pattern:
- Conditionally runs unit checks based on
config.check_unitsflag- Logs warnings but continues with model build even if unit checks fail (appropriate for optional validation)
- Unit checks run before system checks, maintaining clear separation of concerns
- Report directory is created under
config.output_pathfor consistencyThe previous concern about duplicate unit checks for MYOPIC/MGA modes has been addressed (past comments indicate resolution in commit 16e80e1).
temoa/model_checking/unit_checking/__init__.py (1)
1-22: Unit registry initialization is robust and handles edge cases well.The implementation properly handles resource loading:
- Uses
importlib.resources.as_filecontext manager to handle zipped resources- Catches specific exceptions (FileNotFoundError, OSError, DefinitionSyntaxError)
- Uses
_resource_pathstring in error messages to avoid referencing potentially unbound variables- Includes helpful comment explaining the
type: ignore[type-arg]directiveThe error handling provides clear diagnostics for broken installations without exposing implementation details to end users.
tests/test_unit_checking_integration.py (1)
1-107: Integration test suite provides comprehensive scenario coverage.The test module is well-organized:
- Module-level
pytestmarkensures all tests skip gracefully when test databases are absent- Covers all seven database variants created by
create_unit_test_dbs.py- Tests validate both positive case (valid units pass) and negative cases (various error scenarios are detected)
test_databases_existprovides clear diagnostics when setup is incompleteThe skip condition and error messages guide developers to run the database creation script, making the test suite maintainable.
temoa/cli.py (1)
292-395: CLI command implementation is thorough and user-friendly.The
check-unitscommand provides excellent UX:
- Validates database path via typer.Argument constraints
- Creates output directory defensively with clear error handling
- Displays success/failure status with appropriate formatting (green/yellow/red)
- Shows brief report summary on failure, with pointer to full report
- Correctly exits with code 1 on failures (lines 347, 388, 392, 395) for CI/automation compatibility
- Handles exceptions appropriately with logging and user-facing messages
The Ruff FBT002 warning about the
silentparameter is a false positive—usingbooldefaults withtyper.Optionis the standard Typer pattern.temoa/model_checking/unit_checking/table_checker.py (1)
30-89: Entry validation logic is comprehensive and well-structured.The validation pipeline correctly handles each stage:
- Blank entries are flagged with consolidated line ranges
- Character validation uses
ACCEPTABLE_CHARACTERSregex appropriately- Format validation distinguishes between
RATIO_ELEMENTandSINGLE_ELEMENTwith context-specific error messages- Registry compliance check handles both success and failure cases, including the
Noneunit_obj scenarioThe error messages provide actionable guidance (e.g., "Check illegal chars/operators and that denominator is isolated in parentheses").
temoa/model_checking/unit_checking/screener.py (1)
247-314: Screening orchestration is well-structured with clear separation of concerns.The
screen()function provides a clean public API and properly orchestrates the five-stage validation:
- DB version check with early exit on failure
- Units entries validation across all tables
- Efficiency table validation
- Related tables validation (activity/capacity/commodity-based)
- Cost tables validation
Good practices:
- Builds LUTs (commodity units, c2a units) once and reuses across checks
- Aggregates errors across databases and returns cumulative result
- Writes timestamped reports when
report_diris provided- Raises
FileNotFoundErrorfor missing databasesThe Ruff DTZ005 warning about
datetime.now()without a timezone (line 309) is minor—for local file timestamps, the local timezone is often acceptable. If you want timezone-aware timestamps for consistency across deployments, you could usedatetime.now(timezone.utc).temoa/model_checking/unit_checking/common.py (4)
10-49: Well-organized table metadata with helpful inline comments.The commented-out table names with explanations (e.g., growth/degrowth tables using
seed_unitscolumn, output tables lacking units in v4 schema) provide good documentation for future maintainers.
97-125: LGTM! Keyword arguments improve clarity.The
CostTableDatainstantiations now use explicit keyword arguments, making the boolean flags self-descriptive.
160-167: LGTM! Type hint corrected toSequence[int].The function signature now correctly accepts any sequence type, and the truncation logic at 5 items is reasonable for error message readability.
145-157: No changes needed. The regex patterns handle unit expressions correctly as validated by comprehensive unit tests. The non-greedy quantifiers (+?) are appropriate in these patterns since they are anchored to string boundaries (^and$), which forces the regex engine to match all required content regardless of greediness. Test cases confirm the patterns correctly handle complex expressions likekWh/day/(cycle)and properly reject invalid formats like(kWh/day)/(cycle).temoa/model_checking/unit_checking/relations_checker.py (5)
71-156: LGTM! Robust efficiency table validation.The function properly:
- Handles missing commodity units gracefully using
.get()with appropriate error messages (lines 100-110)- Validates both input and output units against commodity definitions
- Detects inconsistent units across rows for the same technology (lines 132-144)
- Consolidates invalid rows into a single summary message for readability
183-186: Good defensive programming with table whitelist validation.Validating
table_nameagainst known safe tables before interpolating into SQL mitigates injection risks even though the callers use controlled inputs.
191-225: Schema-adaptive query building handles v3.1/v4.0 variations well.The code appropriately:
- Checks for column existence before building queries
- Uses
LEFT JOINto handle missing C2A matches gracefully- Falls back to simpler queries when
capacity_to_activitytable is unavailable- Handles both
techandtech_or_groupcolumn namingBased on learnings, SQL injection concerns in this project's database utilities are low priority.
433-458: Cost unit validation logic is now correct.The refactored logic properly:
- Initializes
common_cost_uniton the first valid row- Validates subsequent rows against the established common unit
- Distinguishes between incompatible units (error + continue) and compatible-but-different units (non-standard warning)
334-350: LGTM! Clean helper for consistent error formatting.The
_ding_labelfunction produces well-structured diagnostic output with clear field labels and optional conversion details.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
20419dc to
0c8fea3
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (7)
tests/test_unit_checking.py (1)
84-85: Minor: Docstring placement is unconventional.The docstring-style comment on line 85 appears after the
time_dimension_casesassignment. While syntactically valid (it's just a string literal), the typical convention is to place such documentation either as a comment above the list or as a proper module/function docstring.Consider moving it to a comment above:
+# Time dimension exponents: power units have [time]^-3, energy units have [time]^-2 time_dimension_cases = [('kW', -3), ('kWh', -2), ('PJ', -2), ('PJ/h', -3)] -"""Time dimension exponents: power units have [time]^-3, energy units have [time]^-2 """temoa/model_checking/unit_checking/screener.py (3)
193-202: Redundant assignment on line 199.Same issue as
_check_units_entries: theerrors_test4 = Trueassignment is redundant.if not errors_test4: msg = 'Units Check 4: (Related Tables): Passed' logger.info(msg) report_entries.extend((msg, '\n')) - - else: - errors_test4 = True report_entries.append('\n') return not errors_test4
305-314: Consider timezone-aware timestamps for cross-host consistency.The
datetime.now()call without a timezone argument (line 309) may cause issues if reports from different timezones need to be correlated. This is a minor polish item.def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(tz=timezone.utc).strftime('%Y-%m-%d_%H%M%S')
78-87: Redundant assignment on line 84.The
errors_test2 = Trueon line 84 is redundant since it was already set toTrueon line 73 when errors were found.if not errors_test2: msg = 'Units Check 2 (Units Entries in Tables): Passed' logger.info(msg) report_entries.extend((msg, '\n')) - - else: - errors_test2 = True report_entries.append('\n') return not errors_test2temoa/model_checking/unit_checking/relations_checker.py (2)
49-63: Loop variable name is misleading in make_c2a_lut.Line 56 uses
commto unpack the tech value fromcapacity_to_activitytable, which is misleading since this is a tech-based lookup, not commodity-based.rows = cursor.fetchall() - for comm, units in rows: + for tech, units in rows: valid, group = validate_units_format(units, SINGLE_ELEMENT) if valid and group is not None: valid, unit_obj = validate_units_expression(group[0]) if valid and unit_obj is not None: - res[comm] = unit_obj + res[tech] = unit_obj return res
491-500: Consider explicit handling when both measure_units are None.The current logic sets
matched = Falsewhen eithermeasure_unitsorexpected_measure_unitsisNone. If both areNone, this may produce a misleading "non-matching" error. Consider treating both-None as a separate case:+ # Handle case where both are None (could indicate data/schema issue) + if measure_units is None and expected_measure_units is None: + label = f'{ct.table_name}: Unable to determine measure units for tech/comm: {tech}' + table_grouped_errors[label].append(idx) + continue + # Check if measure_units matches expected matched = ( measure_units == expected_measure_units if (measure_units and expected_measure_units) else False )temoa/model_checking/unit_checking/table_checker.py (1)
91-115: Incorrect physics comments in capacity validation logic.The comments on lines 104-106 are misleading:
- "GWyear (time=1), kWh (time=1) are WRONG" — Actually, energy units like kWh have time exponent of -2 (energy = mass × length² × time⁻²), and GWyear would have time exponent of -2 (power × time = time⁻³ × time¹ = time⁻²)
- "GW (time=-3), MW (time=-3) are CORRECT" — This is correct; power units have time⁻³
The condition
> -3is correct for detecting energy units, but the comments should be updated:- # Positive time exponent indicates energy (capacity * time), not capacity - # Examples: GW*year (time=1), kWh (time=1) are WRONG for capacity - # Examples: GW (time=-3), MW (time=-3) are CORRECT for capacity + # Energy units have time^-2 (e.g., kWh, GWh, J), power units have time^-3 (e.g., kW, GW, W) + # Examples: GWh (time=-2), kWh (time=-2) are WRONG for capacity + # Examples: GW (time=-3), MW (time=-3) are CORRECT for capacity
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (10)
temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)tests/create_unit_test_dbs.py(1 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (5)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-395)temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(136-138)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(247-302)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-64)validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(97-103)RelationType(129-132)consolidate_lines(160-168)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
🪛 Ruff (0.14.7)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/table_checker.py
126-126: Avoid specifying long messages outside the exception class
(TRY003)
tests/create_unit_test_dbs.py
82-85: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/screener.py
259-259: Avoid specifying long messages outside the exception class
(TRY003)
309-309: datetime.datetime.now() called without a tz argument
(DTZ005)
🔇 Additional comments (31)
temoa/model_checking/unit_checking/__init__.py (1)
1-22: Well-implemented unit registry initialization.The implementation correctly:
- Uses
importlib.resources.as_fileto handle both installed packages and zipped resources- Defines
_resource_pathbefore the try block to avoidUnboundLocalErrorin the exception handler- Catches the appropriate exceptions (
FileNotFoundError,OSError,DefinitionSyntaxError)- Provides a clear error message indicating installation issues
The Ruff TRY003 hint about long exception messages can be ignored per the user's request to permanently disable that rule for this repository.
temoa/_internal/temoa_sequencer.py (1)
73-88: Clean integration of unit checking into preliminary checks.The implementation:
- Uses lazy import to avoid loading the unit checking module unless needed
- Correctly passes the input database path and output directory to
screen()- Appropriately logs warnings but continues execution when unit checks fail, matching the opt-in, non-blocking design intent
- Runs before system checks (Python/DB version), which is sensible since unit issues are data-level concerns
The previous concern about duplicate runs for MYOPIC/MGA modes was addressed in commit 16e80e1.
temoa/cli.py (1)
292-395: Solid CLI command implementation for unit checking.The command properly:
- Validates input via Typer's argument constraints (exists, file_okay, readable)
- Creates the output directory with appropriate error handling
- Returns non-zero exit code (
raise typer.Exit(1)) when checks fail, enabling CI/pipeline integration- Provides user-friendly output with report summaries in non-silent mode
The Ruff FBT002 hint is a false positive—
silentis defined as atyper.Optionwith a default, making it a keyword argument, not a boolean positional argument.tests/test_unit_checking.py (3)
36-46: Good test structure with proper assertion handling.The test correctly destructures the
(bool, tuple | None)return value and assertsis_valid is expected, addressing the previous concern about always-truthy tuple assertions.The Ruff FBT001 hint is a false positive—these are pytest parametrized test arguments, not function design choices.
67-81: Thorough expression validation tests.Good coverage of:
- Standard SI units (
kg,m/s)- Custom Temoa units (
ethos,passenger,seat)- Currency units with aliases (
dollar,dollars,USD,EUR)- Invalid/unregistered units (
dog_food)The assertions properly verify both validity and the resulting unit object.
88-100: Well-structured time dimension tests with clear assertions.The test properly validates that time dimensionality exponents are correctly detected for power (-3) vs energy (-2) units. The assertion messages provide useful debugging information.
tests/test_unit_checking_integration.py (2)
17-23: Clean module-level skip pattern.The
pytestmarkapproach cleanly skips all tests when databases are missing, providing a helpful message directing developers to run the creation script. This addresses the previous concern aboutFileNotFoundErroron fresh checkouts.
93-107: Useful existence verification test.This test provides a clear inventory of expected test databases and fails with actionable messages. However, note that with the module-level
pytestmarkskip, this test will also be skipped when databases are missing—meaning it will never actually fail. This is fine since the skip message already directs users to run the creation script.tests/create_unit_test_dbs.py (4)
14-30: Well-structured test database generator with proper path handling.The function correctly uses
Path(__file__)for reliable source location, creates the output directory withexist_ok=True, and provides clear error handling when the source database is missing.
41-68: Context managers properly ensure connection cleanup.Each database variant uses
with sqlite3.connect(...)ensuring connections are properly closed even if exceptions occur. This addresses the earlier review feedback.
74-86: Good defensive check for UPDATE rowcount.The explicit
rowcount == 0check ensures the test setup fails fast with a clear message if the expected E01/ELC record doesn't exist, preventing silent test failures later.
106-110: Clean main guard with proper exit codes.The script correctly returns exit code 0 on success and 1 on failure, making it suitable for CI/CD integration.
temoa/model_checking/unit_checking/table_checker.py (3)
1-27: Clean module structure with appropriate imports.The imports are well-organized, separating standard library, third-party (pint), and local imports. The logger setup follows standard patterns.
30-56: Robust entry validation with consolidated error reporting.The function efficiently groups entries by expression, validates characters against the regex pattern, and uses
consolidate_linesto produce readable error messages. The error message now correctly mentions "letters, digits, underscore" matching theACCEPTABLE_CHARACTERSregex.
117-127: Result assembly handles both single and ratio formats correctly.The logic properly builds the result dictionary based on format type and raises
ValueErrorfor unexpected formats. The Ruff TRY003 warning about long exception messages is a minor stylistic concern that doesn't affect correctness.temoa/model_checking/unit_checking/screener.py (3)
30-55: DB version check is clean and well-documented.The function properly validates the database version requirement and provides clear pass/fail messages for the report.
90-118: Efficiency table check properly returns both LUT and status.The function correctly builds the tech_io_lut and reports errors, returning both the lookup table and success indicator for downstream checks.
247-302: Well-structured screening orchestration with proper error handling.The
screen()function properly:
- Validates file existence before processing
- Runs checks in sequence with early exit on DB version failure
- Aggregates results across multiple databases
- Writes report only when
report_diris providedtemoa/model_checking/unit_checking/common.py (5)
1-9: Clean imports with appropriate typing constructs.The imports are well-organized, using
Sequencefromcollections.abcfor modern type hints.
10-51: Comprehensive table listing with helpful comments.The
tables_with_unitslist clearly documents which tables have units columns in the v4 schema, with commented-out entries explaining why certain tables are excluded (e.g., growth/degrowth tables use different column names).
97-127: CostTableData uses explicit keyword arguments for clarity.The
CostTableDatainstantiations now use keyword arguments (e.g.,table_name='cost_invest'), making the boolean flags' meanings immediately clear without referencing the NamedTuple definition.
135-157: UnitsFormat dataclass and regex patterns are well-documented.The frozen dataclass ensures immutability, and the regex patterns have clear comments explaining their purpose. The
RATIO_ELEMENTpattern correctly requires parentheses around the denominator.
160-168: consolidate_lines helper is concise and effective.The function properly limits output to 5 line numbers plus a count of remaining items, preventing overly long error messages.
temoa/model_checking/unit_checking/relations_checker.py (8)
1-31: Well-organized imports with clear module purpose.The imports properly separate standard library, third-party (pint), and local modules. The docstring clearly states the module's purpose.
65-157: Efficiency table check is robust with proper error handling.The function correctly:
- Uses
.get()for commodity lookups to avoid KeyError- Logs warnings for missing commodity units
- Validates both input and output units against commodity expectations
- Checks for consistency across same-tech entries
- Consolidates invalid rows into a single error message
159-167: _column_exists helper is functional but could be slightly cleaner.The function works correctly. The Ruff TRY300 suggestion to move
return Falseto an else block is purely stylistic and doesn't affect correctness.
169-238: Inter-table relations check has robust schema handling.The function properly:
- Validates table names against a whitelist (lines 184-186)
- Handles missing columns gracefully with
_column_existschecks- Uses LEFT JOIN for capacity-to-activity lookups
- Catches SQL errors and returns structured error messages
Per project learnings, SQL injection concerns are low priority for this codebase due to data access patterns.
239-331: Row processing logic is comprehensive with proper error grouping.The match-case structure cleanly handles CAPACITY, ACTIVITY, and COMMODITY relation types. The error messages now correctly distinguish between "tech" and "commodity" based on relation type (line 266).
353-409: Cost table validation has proper structure and error handling.The function:
- Validates that capacity-based tables don't have commodity references
- Handles SQL errors gracefully with logging
- Screens for empty/missing unit expressions
- Validates ratio format for cost units
410-459: Cost commonality validation now correctly checks all rows.The refactored logic properly:
- Checks currency dimension for each row
- Initializes
common_cost_uniton the first valid row- Validates subsequent rows against the established unit
- Distinguishes between incompatible and compatible-but-non-standard units
460-490: Measure unit validation properly handles capacity-based and period-based adjustments.The logic correctly:
- Looks up expected units from commodity_lut or tech_lut based on table structure
- Applies C2A conversion for capacity-based tables
- Adjusts for period-based tables by dividing by year
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
2 times, most recently
from
20e6dce to
11c4f1d
Compare
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
2 times, most recently
from
e50c0fd to
d375b14
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
temoa/_internal/table_writer.py (2)
347-377: Avoid repeatedself.unit_propagatorproperty calls inside loops; cache to a local.This improves readability and ensures any future side effects (logging, initialization) aren’t repeatedly triggered.
@@ def write_storage_level(self, model: TemoaModel, iteration: int | None = None) -> None: @@ - records = [] + unit_prop = self.unit_propagator + records = [] for sli, val in storage_levels.items(): @@ 'level': val, - 'units': ( - self.unit_propagator.get_storage_units(sli.t) - if self.unit_propagator - else None - ), + 'units': unit_prop.get_storage_units(sli.t) if unit_prop else None, } ) @@ def write_emissions(self, iteration: int | None = None) -> None: @@ + unit_prop = self.unit_propagator for ei, val in self.emission_register.items(): @@ - 'units': ( - self.unit_propagator.get_emission_units(ei.e) if self.unit_propagator else None - ), + 'units': unit_prop.get_emission_units(ei.e) if unit_prop else None, } @@ def _insert_capacity_results(self, cap_data: CapData, iteration: int | None) -> None: @@ + unit_prop = self.unit_propagator @@ - 'units': ( - self.unit_propagator.get_capacity_units(t) if self.unit_propagator else None - ), + 'units': unit_prop.get_capacity_units(t) if unit_prop else None, } ) @@ def _get_flow_units(self, flow_type: FlowType, input_comm: str, output_comm: str) -> str | None: @@ - if not self.unit_propagator: + unit_prop = self.unit_propagator + if not unit_prop: return None @@ - return self.unit_propagator.get_flow_in_units(input_comm) + return unit_prop.get_flow_in_units(input_comm) else: - return self.unit_propagator.get_flow_out_units(output_comm) + return unit_prop.get_flow_out_units(output_comm) @@ def _write_cost_rows(...): @@ + unit_prop = self.unit_propagator @@ - 'units': ( - self.unit_propagator.get_cost_units() if self.unit_propagator else None - ), + 'units': unit_prop.get_cost_units() if unit_prop else None, } )Also applies to: 397-430, 435-498, 500-577, 692-731
95-151: Don't swallow all exceptions when lazily initializingUnitPropagator(and log tracebacks).Catching
Exceptionswallows real bugs (e.g., schema drift, programming errors, module import failures) and degrades silently to "units=None", making failures hard to diagnose. Additionally, the currentlogger.debug(..., e)call drops the traceback.@@ def unit_propagator(self) -> UnitPropagator | None: @@ if self._unit_propagator is None: try: from temoa.model_checking.unit_checking.unit_propagator import ( UnitPropagator, ) @@ self._unit_propagator = UnitPropagator(self.connection) if not self._unit_propagator.has_unit_data: logger.debug('No unit data available in database') - except Exception as e: - logger.debug('Could not initialize unit propagator: %s', e) + except (ImportError, sqlite3.Error, RuntimeError) as e: + logger.debug('Could not initialize unit propagator: %s', e, exc_info=True) # Leave as None - units will be None in output return self._unit_propagatorNarrow the exception scope to catch only expected failures (adjust the exception list based on what
UnitPropagatorcan actually raise), and addexc_info=Trueto preserve tracebacks in debug logs for troubleshooting.
♻️ Duplicate comments (10)
requirements.txt (1)
15-18: Remove flexcache and flexparser—these are not Pint dependencies.As noted in previous reviews, Pint 0.25.2 has no required install dependencies and depends only on the Python standard library. The entries
flexcache==0.3andflexparser==0.4marked as "via pint" are incorrect.Run the following script to verify Pint's actual dependencies:
#!/bin/bash # Check Pint 0.25.2 dependencies from PyPI curl -s https://pypi.org/pypi/pint/0.25.2/json | jq -r '.info.requires_dist // [] | .[]'requirements-dev.txt (1)
30-33: Remove flexcache and flexparser—these are not Pint dependencies.Same issue as in
requirements.txt: flexcache and flexparser are not dependencies of Pint 0.25.2. These should be removed or their actual source in the dependency tree should be identified.tests/conftest.py (1)
96-109: Consider using context manager for database connection.The manual
conn = None/conn.close()pattern works but is less idiomatic than using a context manager. This would automatically handle commits and ensure cleanup even on exceptions:- # Apply modifications with proper connection management - conn = None - - try: - conn = sqlite3.connect(temp_db) - for sql, params in modifications: - conn.execute(sql, params) - conn.commit() - - finally: - if conn is not None: - conn.close() - - close_all_connections(temp_db) + # Apply modifications with proper connection management + with sqlite3.connect(temp_db) as conn: + for sql, params in modifications: + conn.execute(sql, params) + conn.commit() + + close_all_connections(temp_db)temoa/_internal/temoa_sequencer.py (1)
71-72: Docstring does not reflect the non-fatal unit check behavior.The docstring still states "raises an error if any fail," but unit check failures only produce warnings. This was flagged in a prior review.
- def _run_preliminary_checks(self) -> None: - """Runs pre-flight checks and raises an error if any fail.""" + def _run_preliminary_checks(self) -> None: + """Runs pre-flight checks. + + Unit-check failures are logged as warnings but do not abort the run. + Python/database version check failures raise RuntimeError. + """docs/source/unit_checking.rst (1)
166-171: Dimensional notation remains inconsistent with the Dimension Reference section.The shorthand
time^-3for power andtime^1for energy conflicts with the derived dimensions shown later (lines 417-418):
- Energy:
[length]^2 * [mass] / [time]^2→ time exponent is -2, not +1- Power:
[length]^2 * [mass] / [time]^3→ time exponent is -3 ✓Consider aligning with the formal dimensionality:
- ✅ existing_capacity units: "GW" (time^-3 = power) - ❌ existing_capacity units: "GWh" (time^1 = energy) + ✅ existing_capacity units: "GW" ([time]^-3 = power) + ❌ existing_capacity units: "GWh" ([time]^-2 = energy) **Physics**: -- Capacity = Power (W, kW, MW, GW) → ``time^-3`` -- Energy = Power x Time (Wh, kWh) → ``time^1`` +- Capacity = Power (W, kW, MW, GW) → ``[time]^-3`` +- Energy = Power × Time (Wh, kWh) → ``[time]^-2``This was flagged in a prior review cycle but appears not fully addressed.
temoa/model_checking/unit_checking/entry_checker.py (1)
52-64: Add defensive handling for missingunitscolumn.If a table doesn't have a
unitscolumn (e.g., schema not fully upgraded), this will raisesqlite3.OperationalErrorand crash the screening process. The past review flagged this as a critical issue that causes CI failures.def gather_from_table(conn: sqlite3.Connection, table: str) -> dict[str, list[int]]: """gather all unique "units" entries from a table and collect the row indices""" res = defaultdict(list) with conn: cur = conn.cursor() - cur.execute(f'SELECT units FROM {table}') + try: + cur.execute(f'SELECT units FROM {table}') + except sqlite3.OperationalError as exc: + logger.error( + 'Table %s does not contain a "units" column: %s', + table, + exc, + ) + return {} + for idx, result in enumerate(cur.fetchall(), start=1): # note: this will put in "blank" entries which is OK, we want to mark blank rows too entry = result[0] res[entry].append(idx) return resThe caller should then treat an empty dict from a table that should have units as a reportable failure condition.
temoa/model_checking/unit_checking/table_checker.py (2)
47-56: Allowed-characters error message still doesn’t match the regex (spaces are allowed).
ACCEPTABLE_CHARACTERSallows whitespace (\s), but the message omits “spaces”.@@ errors.append( f'Invalid character(s): {expr if expr else "<no recognized entry>"} ' - f'[only letters, digits, underscore and "*, /, ^, ()" operators allowed] ' + f'[only letters, digits, underscore, spaces, and "*, /, ^, ()" operators allowed] ' f'at rows: {listed_lines} ' )
91-103: Drop the incorrectcast('float', ...)and just convert the exponent.
cast('float', ...)is a typing no-op at runtime and passing'float'as a string is not a valid type for static checkers.@@ unit_dimensionality = unit_obj.dimensionality time_exponent = unit_dimensionality.get('[time]', 0) - if float(cast('float', time_exponent)) > -3: + if float(time_exponent) > -3: listed_lines = consolidate_lines(line_nums) errors.append(temoa/model_checking/unit_checking/screener.py (1)
309-318: Consider timezone-aware report timestamps for reproducibility across hosts.@@ def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S%z')temoa/model_checking/unit_checking/relations_checker.py (1)
273-301: Drop unused_is_validlocals to make intent explicit.@@ entry_format_valid, units_data = validate_units_format(table_units, SINGLE_ELEMENT) if entry_format_valid and units_data is not None and len(units_data) >= 1: - _is_valid, valid_table_units = validate_units_expression(units_data[0]) + _, valid_table_units = validate_units_expression(units_data[0]) @@ # further ensure the conversion is valid and retain the appropriate units object - _is_valid, valid_c2a_units = validate_units_expression(units_data[0]) + _, valid_c2a_units = validate_units_expression(units_data[0])
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (26)
data_files/my_configs/config_sample.toml(1 hunks)data_files/temoa_schema_v4.sql(15 hunks)docs/source/index.rst(1 hunks)docs/source/unit_checking.rst(1 hunks)pyproject.toml(1 hunks)requirements-dev.txt(3 hunks)requirements.txt(3 hunks)temoa/_internal/table_writer.py(11 hunks)temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(1 hunks)temoa/core/config.py(3 hunks)temoa/db_schema/temoa_schema_v4.sql(15 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/entry_checker.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)temoa/model_checking/unit_checking/temoa_units.txt(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/config_sample.toml(1 hunks)tests/conftest.py(2 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)tests/test_unit_propagation.py(1 hunks)tests/testing_outputs/.gitignore(1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2025-10-27T15:53:41.829Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 177
File: temoa/model_checking/commodity_network.py:26-33
Timestamp: 2025-10-27T15:53:41.829Z
Learning: The Temoa project requires Python 3.12 or above as the minimum supported version, so PEP 695 `type` syntax for type aliases is appropriate and preferred over `TypeAlias`.
Applied to files:
requirements.txtrequirements-dev.txt
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (12)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(249-306)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(140-142)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(249-306)
tests/conftest.py (2)
temoa/utilities/graphviz_generator.py (1)
connect(173-187)temoa/_internal/table_writer.py (1)
close(120-128)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-395)temoa/model_checking/unit_checking/screener.py (1)
screen(249-306)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(133-136)temoa/model_checking/unit_checking/relations_checker.py (5)
check_cost_tables(353-522)check_efficiency_table(71-156)check_inter_table_relations(169-331)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
temoa/model_checking/unit_checking/entry_checker.py (1)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(140-142)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
temoa/_internal/table_writer.py (2)
temoa/model_checking/unit_checking/unit_propagator.py (7)
has_unit_data(273-282)get_storage_units(252-270)get_emission_units(228-241)get_capacity_units(214-226)get_flow_in_units(187-200)get_flow_out_units(172-185)get_cost_units(243-250)temoa/types/model_types.py (1)
FlowType(275-282)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(164-172)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-64)validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/relations_checker.py (2)
temoa/model_checking/unit_checking/common.py (3)
CostTableData(101-107)RelationType(133-136)consolidate_lines(164-172)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/core/config.py (1)
temoa/cli.py (1)
check_units(293-395)
🪛 Ruff (0.14.8)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
tests/conftest.py
129-131: Avoid specifying long messages outside the exception class
(TRY003)
148-151: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/screener.py
59-59: Unused function argument: major_version
(ARG001)
59-59: Unused function argument: minor_version
(ARG001)
261-261: Avoid specifying long messages outside the exception class
(TRY003)
313-313: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/model_checking/unit_checking/entry_checker.py
47-47: Consider moving this statement to an else block
(TRY300)
58-58: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/unit_propagator.py
135-135: Possible SQL injection vector through string-based query construction
(S608)
temoa/_internal/table_writer.py
147-147: Do not catch blind exception: Exception
(BLE001)
temoa/model_checking/unit_checking/table_checker.py
115-115: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
temoa/core/config.py
62-62: Boolean-typed positional argument in function definition
(FBT001)
62-62: Boolean default positional argument in function definition
(FBT002)
🔇 Additional comments (29)
pyproject.toml (1)
33-33: LGTM: Pint dependency appropriately added for unit checking.The addition of
pint>=0.25.2as a runtime dependency is appropriate for the new unit-checking subsystem. The version constraint is reasonable and aligns with the PR objectives.temoa/model_checking/unit_checking/temoa_units.txt (1)
1-15: LGTM: Custom unit definitions are well-structured.The custom Pint unit definitions appropriately extend the default registry for Temoa-specific needs. The warning about currency conversions (line 8) is important—Pint treats currencies as dimensional units but doesn't perform exchange rate conversions.
tests/conftest.py (2)
148-151: Error message formatting violates TRY003 but is acceptable.The static analyzer flags this as a long message outside the exception class. While creating a custom exception class would be more robust, the current approach is acceptable for test infrastructure where the message provides clear context about the missing source database.
215-222: LGTM: pytest_configure hook properly handles database setup.Using
pytest_configureto create test databases avoids import-time failures and gracefully handles missing source databases by catchingFileNotFoundError. The# noqa: ARG001annotation appropriately suppresses the unused parameter warning.docs/source/index.rst (1)
7-7: LGTM: Documentation navigation updated for unit checking.The addition of
unit_checkingto the toctree is appropriate and aligns with the new documentation file for the unit-checking subsystem.temoa/tutorial_assets/config_sample.toml (1)
49-52: LGTM: check_units option appropriately added with clear documentation.The new
check_units = trueconfiguration option is well-documented. Setting it totruein the tutorial sample makes sense as a reference implementation, ensuring users are aware of the feature.data_files/my_configs/config_sample.toml (1)
50-53: LGTM: check_units defaults to false for non-breaking compatibility.The
check_units = falsedefault in this config sample is appropriate, maintaining backward compatibility while making the feature discoverable. This aligns with the PR's goal of making unit checking opt-in to avoid breaking changes.temoa/db_schema/temoa_schema_v4.sql (1)
106-109: Schema additions for unit tracking look good.The
unitscolumn additions across all relevant tables are appropriately nullable, ensuring backward compatibility with existing databases. The placement before thenotescolumn maintains a logical column ordering convention.docs/source/unit_checking.rst (1)
1-8: Comprehensive documentation for the unit checking subsystem.The documentation thoroughly covers the feature's purpose, configuration, validation steps, format requirements, common pitfalls, and best practices. The explicit warning about pre-processing validation (not runtime enforcement) sets appropriate expectations.
temoa/_internal/temoa_sequencer.py (1)
73-88: Unit checking integration is well-structured.The lazy import keeps the unit checking dependency optional, and the warning-only approach correctly implements opt-in validation without blocking model execution. The report output location aligns with other outputs.
temoa/cli.py (2)
292-318: Well-structured CLI command following existing patterns.The
check-unitscommand correctly follows the established patterns in the CLI (similar tovalidate,run, etc.) with proper argument/option handling, error handling, and output formatting. The static analysis hint (FBT002) is a false positive sincesilentis a keyword-only parameter viatyper.Option, not a positional argument.
350-395: Comprehensive error handling and correct exit codes.The command properly:
- Handles
FileNotFoundErrorand general exceptions- Returns exit code 1 on failure (line 394-395)
- Provides helpful report summaries when issues are found
This aligns with the past review feedback about non-zero exit codes for CI/script usage.
temoa/model_checking/unit_checking/__init__.py (1)
1-22: Robust UnitRegistry initialization with proper error handling.The implementation addresses all prior review feedback:
_resource_pathis defined before the try block, avoidingUnboundLocalError- Error handling catches the relevant exceptions with a clear message
importlib_resources.as_filecorrectly handles zipped resources- The type ignore comment includes the rationale (line 6)
The Ruff TRY003 hint about long exception messages was discussed in prior reviews and agreed to be ignored for this repository.
tests/test_unit_checking.py (3)
1-16: LGTM!The imports are correctly organized and the module docstring clearly describes the test purpose.
17-46: LGTM!The format validation test cases cover a good range of scenarios including whitespace handling, allowed/disallowed characters, and ratio format semantics. The test function now correctly destructures the return value and asserts against the
expectedparameter.
49-81: LGTM!Good coverage of unit expression validation including standard units, custom Temoa units (ethos, passenger, dollar, euro), and invalid expressions. The conditional assertion logic correctly handles both valid and invalid cases.
tests/test_unit_propagation.py (2)
32-74: LGTM!The individual test functions provide good coverage for the propagator's methods including positive cases and graceful handling of missing data (nonexistent commodity/tech returning
None).
25-29: Connection is not used after initialization—the fixture is safe as-is.
UnitPropagatorstores the connection inself._connbut only uses it during__init__to build lookup tables (via_build_lookups()). All subsequent getter method calls operate on cached data (dictionaries and Unit objects), not the connection. The context manager closing the connection after the fixture returns does not affect tests since they only invoke getter methods that reference pre-cached lookups. The proposed change toyieldis unnecessary.Likely an incorrect or invalid review comment.
temoa/model_checking/unit_checking/entry_checker.py (3)
1-22: LGTM!Module docstring clearly describes the three validation checks. Imports are appropriate and the logger is correctly configured.
24-36: LGTM!The function correctly handles empty strings and uses regex group capture to return both validity and extracted elements.
39-49: LGTM!Clean exception handling for undefined units. The TRY300 hint about moving the return to an
elseblock is a minor style preference and the current form is readable.tests/test_unit_checking_integration.py (3)
1-25: LGTM!Module docstring correctly references
conftest.pyfor DB creation. The skip condition and message are accurate.
28-52: LGTM!Good test structure. The
test_invalid_currency_units_detectedproperly cleans up report artifacts using try/finally as suggested in past reviews.
55-97: LGTM!Comprehensive coverage of error detection scenarios: energy units in capacity tables, missing parentheses, unknown units, mismatched outputs, and bad composite currency. Each test targets a specific validation rule.
temoa/core/config.py (2)
62-62: LGTM!The
check_unitsparameter follows the established pattern for boolean configuration options in this class. The default ofFalsemakes unit checking opt-in, which aligns with the PR's design to avoid breaking changes.Also applies to: 139-139
253-253: LGTM!The
__repr__output correctly includes the new "Unit checking" field, grouped logically with related validation options (price_check, source_trace).temoa/model_checking/unit_checking/common.py (1)
10-172: Centralized unit-checking metadata/constants look consistent and readable.data_files/temoa_schema_v4.sql (1)
100-109: Schema additions look aligned with unit-checking/propagation, but verify migrations + anySELECT *consumers.Given the wide addition of nullable
unitscolumns, please confirm:
- v4 migration/backfill scripts add these columns consistently for existing DBs, and
- no code relies on positional
SELECT *ordering for affected tables (inputs and outputs).Also applies to: 110-118, 141-155, 155-206, 207-218, 235-248, 249-265, 334-345, 352-396, 703-772, 773-840, 995-1070
temoa/model_checking/unit_checking/relations_checker.py (1)
353-521: Cost-table checks look coherent (currency dimension + commonality + measure alignment).Please just sanity-check with Pint 0.25.2 that
(1.0 * cost_unit_obj).to(common_cost_unit)behaves as expected for your custom currency units (dollar/euro) and any prefixes you define.
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (5)
docs/source/unit_checking.rst (1)
166-171: Dimensional notation inconsistency persists.The shorthand
time^-3for power andtime^1for energy at lines 166-171 conflicts with the Dimension Reference section (lines 417-418), which correctly shows energy as[time]^-2and power as[time]^-3. The notationtime^1for energy is incorrect—energy is[time]^-2(one power of time higher than power's[time]^-3).Consider updating for consistency:
- ✅ existing_capacity units: "GW" (time^-3 = power) - ❌ existing_capacity units: "GWh" (time^1 = energy) + ✅ existing_capacity units: "GW" ([time]^-3 = power) + ❌ existing_capacity units: "GWh" ([time]^-2 = energy) **Physics**: -- Capacity = Power (W, kW, MW, GW) → ``time^-3`` -- Energy = Power x Time (Wh, kWh) → ``time^1`` +- Capacity = Power (W, kW, MW, GW) → ``[time]^-3`` +- Energy = Power × Time (Wh, kWh) → ``[time]^-2``tests/conftest.py (1)
177-184: Consider logging when unit test DB creation is skipped.The
pytest_configurehook silently catchesFileNotFoundError. While the inline comment explains the intent, logging a warning would help developers understand why unit tests are being skipped in CI or fresh environments.try: create_unit_test_dbs() except FileNotFoundError: # Source DB not available; unit tests will be skipped via pytestmark - pass + logger.warning( + 'Unit test databases not created: source SQL not found. ' + 'Unit checking tests will be skipped.' + )temoa/model_checking/unit_checking/unit_propagator.py (1)
150-174: Fragile numerator extraction persists from previous review.The
split('/')[0]approach at line 169 remains unchanged from the previous review. While existing cost data uses simple formats, edge cases with slashes in the numerator (e.g., "kWh/day / (GW)") would extract incorrectly.The SQL injection warning at line 163 is a false positive – the table name comes from a hardcoded list (
cost_tables), not user input.However, consider making the exception handling more specific:
- except sqlite3.OperationalError: + except sqlite3.OperationalError as e: + logger.debug('Cost table %s not found or query failed: %s', table, e) continuetests/test_unit_propagation.py (2)
195-214: Add connection cleanup for consistency.Similar to the previous tests, wrap the connection in a context manager.
def test_units_propagated_to_cost_table(solved_db_with_units: Path) -> None: """Verify that output_cost table has units populated.""" - conn = sqlite3.connect(solved_db_with_units) - - row = conn.execute( + with sqlite3.connect(solved_db_with_units) as conn: + row = conn.execute( + # ... rest of test - conn.close()
149-171: Add connection cleanup for consistency.While the test logic is correct, the connection should be closed or wrapped in a context manager for consistency with other tests.
def test_units_propagated_to_output_flow_tables(solved_db_with_units: Path) -> None: """Verify that output flow tables have units populated.""" - conn = sqlite3.connect(solved_db_with_units) - - # Check output_flow_out - row = conn.execute( - 'SELECT COUNT(*), SUM(CASE WHEN units IS NOT NULL THEN 1 ELSE 0 END) FROM output_flow_out' - ).fetchone() + with sqlite3.connect(solved_db_with_units) as conn: + # Check output_flow_out + row = conn.execute( + 'SELECT COUNT(*), SUM(CASE WHEN units IS NOT NULL THEN 1 ELSE 0 END) FROM output_flow_out' + ).fetchone() + # ... rest of test - conn.close()
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (8)
docs/source/unit_checking.rst(1 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/cli.py(2 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_propagation.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(133-136)temoa/model_checking/unit_checking/relations_checker.py (5)
check_cost_tables(353-522)check_efficiency_table(71-156)check_inter_table_relations(169-331)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
temoa/_internal/table_writer.py (2)
temoa/model_checking/unit_checking/unit_propagator.py (7)
has_unit_data(306-315)get_storage_units(285-303)get_emission_units(261-274)get_capacity_units(247-259)get_flow_in_units(220-233)get_flow_out_units(205-218)get_cost_units(276-283)temoa/types/model_types.py (1)
FlowType(275-282)
tests/test_unit_propagation.py (3)
temoa/_internal/table_writer.py (1)
unit_propagator(131-150)temoa/model_checking/unit_checking/unit_propagator.py (6)
UnitPropagator(28-315)has_unit_data(306-315)get_flow_out_units(205-218)get_flow_in_units(220-233)get_capacity_units(247-259)get_cost_units(276-283)temoa/core/config.py (2)
TemoaConfig(32-327)build_config(199-236)
🪛 GitHub Actions: CI
tests/test_unit_propagation.py
[error] 190-190: Unit propagation test failed: Expected capacity unit 'GW' but found 'PJ^2 / GW' in output_built_capacity table.
🪛 Ruff (0.14.8)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
263-263: Unpacked variable major_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
263-263: Unpacked variable minor_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
306-306: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
592-595: Avoid specifying long messages outside the exception class
(TRY003)
598-601: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/unit_propagator.py
143-144: try-except-pass detected, consider logging the exception
(S110)
143-143: Do not catch blind exception: Exception
(BLE001)
163-163: Possible SQL injection vector through string-based query construction
(S608)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_propagation.py
179-179: Possible SQL injection vector through string-based query construction
(S608)
187-187: Possible SQL injection vector through string-based query construction
(S608)
🔇 Additional comments (22)
docs/source/unit_checking.rst (1)
1-427: Comprehensive and well-structured documentation.The documentation thoroughly covers the unit checking subsystem including configuration, validation tests, common pitfalls, and best practices. The table coverage matrix and quick reference sections are particularly helpful for users.
temoa/cli.py (2)
292-396: Well-implemented CLI command with proper error handling.The
check-unitscommand implementation is solid:
- Proper argument validation via Typer decorators
- Graceful directory creation with error handling
- Non-zero exit code on failure (line 394-395) correctly addresses the previous review concern
- Comprehensive exception handling for different failure modes
- Report summary display for user feedback
Note: The Ruff FBT002 warning at line 314 is a false positive—
silentis a Typer Option with explicit flags (--silent,-q), not a boolean positional argument.
552-612: Good refactoring to single source of truth for tutorial database.The change from copying a pre-built SQLite database to generating it from the SQL source file (
utopia.sql) ensures consistency and maintains a single source of truth. The approach properly:
- Removes existing database before regeneration (avoiding stale data)
- Handles both package resources and development fallback paths
- Provides clear error messages when resources are missing
tests/conftest.py (2)
74-96: Clean implementation with proper resource management.The
create_unit_test_db_from_sqlhelper properly uses context managers for both the SQLite connection and file operations, addressing previous review concerns about connection cleanup.
116-169: Comprehensive test variations for unit checking scenarios.The test database variations effectively cover the key validation scenarios documented in
unit_checking.rst:
- Valid baseline (no modifications)
- Invalid currency dimension in cost tables
- Energy units in capacity tables
- Missing parentheses in ratio format
- Unknown/unregistered units
- Mismatched technology output units
- Invalid composite currency
This provides good coverage for integration testing the unit checker.
temoa/_internal/table_writer.py (3)
130-150: Well-designed lazy initialization with graceful fallback.The
unit_propagatorproperty implements a clean lazy initialization pattern:
- Defers import until first access (avoids circular imports and startup cost)
- Caches the instance for reuse
- Catches multiple exception types for robustness
- Logs at debug level (appropriate for optional feature)
- Returns
Nonegracefully when unit data is unavailableThis ensures backward compatibility with databases lacking unit information.
545-568: Correct flow unit selection logic.The
_get_flow_unitshelper correctly implements the documented unit propagation rules:
FlowType.IN→ uses input commodity unitsFlowType.OUT,FlowType.CURTAIL,FlowType.FLEX→ use output commodity unitsThis aligns with the PR objectives specifying that
output_flow_inuses input commodity units whileoutput_flow_outandoutput_curtailmentuse output commodity units.
594-614: Verify intentionalNoneunits in summary flow output.The
_insert_summary_flow_resultsmethod hardcodes'units': None(line 609) rather than using the unit propagator like other write paths. If this is intentional (e.g., because summaries aggregate across different commodities/units), consider adding a comment to clarify. Otherwise, this may need unit propagation similar towrite_flow_tables.temoa/model_checking/unit_checking/screener.py (1)
244-299: Well-structured screening pipeline.The
screen()function implements a clean 5-stage validation pipeline:
- Database version check (with early exit on failure)
- Units entry validation
- Efficiency table alignment
- Related tables consistency
- Cost tables validation
The implementation correctly:
- Accumulates errors across all stages
- Builds lookup tables once and reuses them
- Supports multiple database paths
- Optionally writes consolidated reports
- Returns a clear success/failure indicator
temoa/model_checking/unit_checking/unit_propagator.py (6)
1-60: LGTM!The module documentation is clear, imports are appropriate, and the initialization pattern is sound. The use of TYPE_CHECKING for conditional imports and defensive initialization with None values ensures graceful degradation.
61-102: LGTM!The graceful error handling with try/except blocks and logger.debug is appropriate for optional unit data. The conditional logic to only build tech_io_units when commodity_units exists prevents cascading failures.
176-203: LGTM!The storage commodity lookup correctly identifies storage technologies through multiple criteria (seas_stor flag, storage_duration table, or 'ps' flag) and gracefully handles missing tables.
205-274: LGTM!The getter methods correctly use Pint's compact format specifier
f"{unit:~}"to preserve unit symbols, addressing the previous review's concern. Defensive None checks ensure graceful handling of missing data.
276-303: LGTM!Both methods follow the established pattern with proper None checks and Pint format specifiers.
305-315: LGTM!The property provides a clean API for checking unit data availability.
tests/test_unit_propagation.py (3)
1-75: LGTM!The test structure is sound, with proper use of pytest fixtures and skipif markers. The previous review's concern about connection leaks has been addressed – line 80 correctly uses a context manager.
77-92: LGTM!The previous review's recommendation to use a context manager has been implemented (line 80), ensuring the connection is properly closed even if assertions fail.
99-146: LGTM!The integration test fixture is well-designed, properly using pytest's temp directory facilities and cleaning up after itself. The config file generation is appropriate for end-to-end testing.
temoa/tutorial_assets/utopia.sql (4)
162-177: LGTM!The capacity_to_activity table units are consistently formatted as 'PJ / (GW * year)', which correctly represents energy-per-capacity-per-time conversion factors.
178-200: LGTM!Commodity units are consistently set to 'PJ' for all energy and emission commodities, providing a simple and uniform basis for flow unit derivation.
396-467: LGTM!The remaining tables (cost_variable, demand, efficiency, emission_activity, existing_capacity, and others) use straightforward unit formats that align with expected physical dimensions. These simple formats work correctly with the current parsing logic.
Also applies to: 531-594, 635-648, 703-717
255-381: The review comment references files that cannot be verified in the current repository state.The review targets
temoa/tutorial_assets/utopia.sql(lines 255-381) andunit_propagator.py, but these files are not accessible for verification. The specific concerns raised about complex unit denominators ('Mdollar / (PJ^2 / GW)'and'Mdollar / (PJ^2 / GW / year)') and alleged parsing failures cannot be confirmed without access to:
- The actual SQL file containing these cost table definitions
- The parsing logic in
unit_propagator.pythat allegedly fails- The test failures mentioned as pipeline validation
To proceed, please ensure:
- The files are committed to the repository and accessible
- The file paths are correct
- Any relevant test cases showing the failure are included
Once the files are available, the dimensional analysis of cost unit formats and the impact on unit parsing can be properly evaluated.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
2 times, most recently
from
a3884e3 to
674423e
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
temoa/_internal/table_writer.py (2)
95-150: Prevent repeated (failing) UnitPropagator init attempts.
If init fails once (ImportError/sqlite error), every subsequent access retries and re-logs; consider an “attempted” sentinel.class TableWriter: @@ def __init__(...): @@ - self._unit_propagator: UnitPropagator | None = None + self._unit_propagator: UnitPropagator | None = None + self._unit_propagator_attempted: bool = False @@ def unit_propagator(self) -> UnitPropagator | None: @@ - if self._unit_propagator is None: + if self._unit_propagator is None and not self._unit_propagator_attempted: + self._unit_propagator_attempted = True try: from temoa.model_checking.unit_checking.unit_propagator import ( UnitPropagator, ) @@ except (ImportError, sqlite3.Error, RuntimeError) as e: logger.debug('Could not initialize unit propagator: %s', e, exc_info=True) # Leave as None - units will be None in output return self._unit_propagator
574-615: Consider propagatingunitsforoutput_flow_out_summarytoo (if the table has aunitscolumn).
Right now it’s always written asNone, unlike the other flow tables.
♻️ Duplicate comments (3)
tests/conftest.py (1)
37-72: Use context managers for SQLite connections inrefresh_databases()(exception safety).
This is the same robustness improvement previously applied elsewhere in this file.- con = sqlite3.connect(data_output_path / db) - with open(source_dir / src) as script: - con.executescript(script.read()) - con.close() + with sqlite3.connect(data_output_path / db) as con, open(source_dir / src) as script: + con.executescript(script.read())temoa/model_checking/unit_checking/screener.py (2)
244-300: Prefix unused unpacked DB version variables with underscores.
This matches Ruff RUF059 and makes intent explicit.- db_version_ok, major_version, minor_version = _check_db_version(conn, report_entries) + db_version_ok, _major_version, _minor_version = _check_db_version(conn, report_entries)
302-321: Use UTC timestamps for report filenames + replace the triple-quoted “comment” with# ....def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone @@ - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S') @@ -"""Preserve a way to run this quickly/independently as long as that is useful...""" +# Preserve a way to run this quickly/independently as long as that is useful... if __name__ == '__main__':
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (9)
docs/source/unit_checking.rst(1 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/cli.py(2 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_propagation.py(1 hunks)tests/test_unit_scavenging.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
temoa/_internal/table_writer.py (2)
temoa/model_checking/unit_checking/unit_propagator.py (8)
UnitPropagator(29-337)has_unit_data(328-337)get_storage_units(307-325)get_emission_units(283-296)get_capacity_units(269-281)get_flow_in_units(242-255)get_flow_out_units(227-240)get_cost_units(298-305)temoa/types/model_types.py (1)
FlowType(275-282)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
tests/test_unit_propagation.py (2)
temoa/_internal/table_writer.py (1)
unit_propagator(131-150)temoa/model_checking/unit_checking/unit_propagator.py (6)
UnitPropagator(29-337)has_unit_data(328-337)get_flow_out_units(227-240)get_flow_in_units(242-255)get_capacity_units(269-281)get_cost_units(298-305)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(133-136)temoa/model_checking/unit_checking/relations_checker.py (3)
check_cost_tables(353-522)check_efficiency_table(71-156)check_inter_table_relations(169-331)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
🪛 Ruff (0.14.8)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
188-188: Unnecessary pass statement
Remove unnecessary pass
(PIE790)
temoa/model_checking/unit_checking/unit_propagator.py
178-178: Possible SQL injection vector through string-based query construction
(S608)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
592-595: Avoid specifying long messages outside the exception class
(TRY003)
598-601: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_scavenging.py
34-34: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
tests/test_unit_propagation.py
187-187: Possible SQL injection vector through string-based query construction
(S608)
195-195: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
263-263: Unpacked variable major_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
263-263: Unpacked variable minor_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
306-306: datetime.datetime.now() called without a tz argument
(DTZ005)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
🔇 Additional comments (17)
docs/source/unit_checking.rst (2)
10-34: Clear statement that unit checking is validation-only (not enforced in Pyomo).
This sets correct expectations and avoids users assuming dimensional enforcement during solve.
151-213: Capacity vs energy dimensionality examples look consistent with Pint ([time]^-3 vs [time]^-2).temoa/_internal/table_writer.py (1)
347-569: Units propagation wiring across storage/emissions/capacity/flows looks consistent and safely nullable.
The_bulk_insertcolumn intersection keeps this compatible with older schemas.temoa/cli.py (2)
292-396:check-unitscommand behavior (report generation + non-zero exit on failures) is solid for CI/scripting.
552-613: Generating the tutorial DB fromutopia.sqlis the right “single source of truth” move.tests/test_unit_propagation.py (2)
99-147: Verify CI/runtime has the configured solver (appsi_highs) and that a full solve belongs in the default unit test suite.
If solver availability/perf varies by environment, consider marking this as@pytest.mark.integration/@pytest.mark.slowand gating via an env var.
182-199: Silence Ruff S608 by whitelisting table names before f-string SQL.
This is safe here (hardcoded table list), but linters may still fail the build.def test_units_propagated_to_capacity_tables(solved_db_with_units: Path) -> None: @@ with sqlite3.connect(solved_db_with_units) as conn: - for table in ['output_built_capacity', 'output_net_capacity']: + allowed_tables = {'output_built_capacity', 'output_net_capacity'} + for table in sorted(allowed_tables): row = conn.execute( f'SELECT COUNT(*), SUM(CASE WHEN units IS NOT NULL THEN 1 ELSE 0 END) FROM {table}' ).fetchone()Likely an incorrect or invalid review comment.
temoa/model_checking/unit_checking/unit_propagator.py (6)
1-61: LGTM! Clean initialization and proper type checking.The module structure is well-organized with proper imports, clear documentation, and appropriate use of TYPE_CHECKING to avoid runtime Pint dependencies. The initialization pattern correctly sets up instance state before building lookups.
62-103: LGTM! Robust error handling with graceful fallbacks.The method properly isolates failures for each lookup table, allowing partial unit data to be available even if some lookups fail. The dependency check at lines 71-72 ensures tech I/O units are only built when commodity units exist. Debug-level logging is appropriate since databases without unit metadata are expected.
104-163: LGTM! Capacity unit extraction properly handles complex formats.The regex-based approach in
_extract_capacity_unit(lines 141-163) correctly addresses the previous critical issue where nested denominators like'Mdollar / (PJ^2 / GW)'were incorrectly parsed. Word boundaries prevent false matches, and the precedence order (existing_capacity → cost_invest) is appropriate.
165-196: LGTM! Robust cost unit extraction handles multiple formats.The regex-based split at lines 183-187 properly addresses the previous concern about fragile parsing. The pattern
r'\s*/\s*\('correctly extracts numerators from both simple ratios and complex nested formats like'kWh/day / (GW)', with appropriate fallback for legacy formats.
198-225: LGTM! Storage technology identification is comprehensive.The query at lines 212-218 correctly identifies storage technologies through multiple pathways (seas_stor flag, storage_duration table, or 'ps' flag), using appropriate LEFT JOINs. The fallback to empty dict on OperationalError ensures compatibility with databases lacking storage-related tables.
227-337: LGTM! Unit getters correctly preserve short symbols.All methods returning units properly use the
f'{unit:~}'format specifier (lines 240, 255, 296, 324), ensuring short symbols like "GW" are preserved instead of canonical names like "gigawatt". The defensive checks for None/empty lookups ensure graceful degradation for databases without unit metadata.temoa/tutorial_assets/utopia.sql (4)
156-177: LGTM! Capacity-to-activity units consistently defined.The
units TEXTcolumn addition at line 162 and consistent use of'PJ / (GW * year)'format across all inserts (lines 166-177) properly demonstrates the expected ratio format with parenthesized compound denominators.
178-200: LGTM! Commodity units consistently use PJ.The schema additions at lines 184-186 and uniform
'PJ'unit assignment across all commodity types (physical, demand, emissions, source) in lines 187-200 is consistent with an energy system model. The added descriptions improve documentation.
241-381: Cost unit formats test complex parsing logic.The cost table units demonstrate the range of formats the unit checking system must handle:
- cost_fixed:
'Mdollar / (PJ^2 / GW / year)'(lines 255-336)- cost_invest:
'Mdollar / (PJ^2 / GW)'(lines 349-381)- cost_variable:
'Mdollar / (PJ)'(lines 396-446)The nested structure in cost_fixed and cost_invest (with PJ² in denominators) provides good test coverage for the regex-based capacity extraction in
unit_propagator._extract_capacity_unit, which should correctly identify 'GW' in these strings.
447-841: LGTM! Comprehensive unit coverage across input and output tables.The tutorial database demonstrates proper unit annotation across all major table categories:
- Demand: PJ (lines 459-467)
- Efficiency: PJ / (PJ) dimensionless ratios (lines 531-594)
- Emissions: Mt / (PJ) mass per energy (lines 635-648)
- Capacity: GW consistently (lines 703-717, 831-841)
- Lifetimes: year (lines 730-734, 757-775)
- Output tables: units columns added for propagation (lines 1167, 1186, 1211, 1236, 1260, 1284, 1315, 1338, 1358)
The nullable TEXT columns maintain backward compatibility while enabling the unit checking framework.
Also applies to: 1156-1360
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
temoa/cli.py (1)
552-613: Tutorial DB generation from SQL looks solid; consider explicit transaction + clearer failure mode.
executescript()wraps statements, but ifutopia.sqlgrows, an explicitBEGIN/COMMIT(or catchingsqlite3.Errorto print a “DB generation failed” with file context) may make failures easier to diagnose.
♻️ Duplicate comments (10)
docs/source/unit_checking.rst (1)
94-119: Clarify that the strict ratio format applies only to efficiency and cost tables.The warning currently reads as a global rule. As noted in a previous review, adding a brief scope clarifier would reduce confusion for users working with other tables.
Consider adding one sentence to clarify:
.. warning:: - **CRITICAL**: The unit checker uses regex parsing with strict format requirements! + **CRITICAL**: The unit checker uses regex parsing with strict format requirements + for **efficiency** and **cost** tables! -Units in **efficiency** and **cost** tables MUST use ratio format: +Units in these tables MUST use ratio format:tests/test_unit_scavenging.py (2)
4-19: Consider importing from production code instead of duplicating.This function duplicates
UnitPropagator._extract_capacity_unit. If the production logic changes, these tests won't catch regressions. Consider either:
- Making
_extract_capacity_unita module-level function that both production and tests import- Testing through
UnitPropagator.get_capacity_units()directly-def extract_capacity_unit(unit_str: str) -> str | None: - """Scavenge for a capacity unit within a complex unit string.""" - patterns = [ - r'\bGW\b', - r'\bMW\b', - r'\bkW\b', - r'\bTW\b', - r'\bgigawatt\b', - r'\bmegawatt\b', - r'\bkilowatt\b', - ] - for pat in patterns: - match = re.search(pat, unit_str) - if match: - return match.group(0) - return None +from temoa.model_checking.unit_checking.unit_propagator import UnitPropagator + +# Use a mock or direct method access for testing +_propagator_instance = type('MockPropagator', (), {'_extract_capacity_unit': staticmethod( + UnitPropagator._extract_capacity_unit.__func__ +)})() +extract_capacity_unit = _propagator_instance._extract_capacity_unitOr better yet, refactor
_extract_capacity_unitto be a standalone utility function.
34-36: Remove unusednoqadirective.The
# noqa: N802directive is flagged as unused by Ruff since N802 is not enabled.-def test_extract_MW(): # noqa: N802 +def test_extract_mw() -> None: assert extract_capacity_unit('Mdollar / (MW)') == 'MW'temoa/model_checking/unit_checking/unit_propagator.py (1)
198-225: Consider adding error logging for storage lookup failures.The storage commodity lookup silently passes on
sqlite3.OperationalError. While graceful fallback is appropriate, a debug log would aid troubleshooting when storage units are unexpectedly missing.except sqlite3.OperationalError: - pass + logger.debug('Could not query storage technologies from efficiency/technology tables') return resulttests/conftest.py (1)
182-188: Remove unnecessarypassstatement.The
passafter thelogger.warning()call is redundant and flagged by static analysis.except FileNotFoundError: # Source DB not available; unit tests will be skipped via pytestmark logger.warning( 'Unit test databases not created: source SQL not found. ' 'Unit checking tests will be skipped.' ) - passtests/test_unit_propagation.py (2)
25-35: Fixture returns aUnitPropagatorbacked by a closed SQLite connection.
This makes the tests brittle (and can outright fail ifUnitPropagatordoes any query after init).@pytest.fixture def propagator() -> UnitPropagator: """Create a UnitPropagator from the valid units database.""" - with sqlite3.connect(VALID_UNITS_DB) as conn: - return UnitPropagator(conn) + conn = sqlite3.connect(VALID_UNITS_DB) + try: + yield UnitPropagator(conn) + finally: + conn.close()
202-217: Make cost-unit assertion case-insensitive and include USD/EUR variants.- assert 'dollar' in sample[0] or 'Eur' in sample[0] + units = sample[0].lower() + assert ('dollar' in units) or ('usd' in units) or ('euro' in units) or ('eur' in units)temoa/model_checking/unit_checking/screener.py (3)
244-300: Prefix unused unpacked vars with_(silences Ruff RUF059 and clarifies intent).- db_version_ok, major_version, minor_version = _check_db_version(conn, report_entries) + db_version_ok, _major_version, _minor_version = _check_db_version(conn, report_entries)
302-321: Use timezone-aware timestamps; replace stray triple-quoted “comment”.def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone @@ - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S') -"""Preserve a way to run this quickly/independently as long as that is useful...""" +# Preserve a way to run this quickly/independently as long as that is useful... if __name__ == '__main__':
30-56: Make_check_db_version()robust to missing/invalidmetadatatable (currently can crash).
Right now, missingmetadata, missingDB_MAJOR, or non-integer values will raise and abort the whole screener instead of reporting “Units Check 1 … Failed”.def _check_db_version(conn: sqlite3.Connection, report_entries: list[str]) -> tuple[bool, int, int]: @@ - data = conn.execute('SELECT element, value FROM metadata').fetchall() - meta_data = dict(data) - major = int(meta_data.get('DB_MAJOR', 0)) - minor = int(meta_data.get('DB_MINOR', 0)) + try: + data = conn.execute('SELECT element, value FROM metadata').fetchall() + meta_data = dict(data) + major = int(meta_data.get('DB_MAJOR', 0)) + minor = int(meta_data.get('DB_MINOR', 0)) + except (sqlite3.Error, ValueError, TypeError) as e: + major, minor = 0, 0 + msg = f'Units Check 1 (DB Version): Failed. Unable to read metadata table: {e}' + report_entries.extend((msg, '\n')) + logger.warning(msg) + return False, major, minor
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (9)
docs/source/unit_checking.rst(1 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/cli.py(2 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_propagation.py(1 hunks)tests/test_unit_scavenging.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
temoa/_internal/table_writer.py (2)
temoa/model_checking/unit_checking/unit_propagator.py (8)
UnitPropagator(29-337)has_unit_data(328-337)get_storage_units(307-325)get_emission_units(283-296)get_capacity_units(269-281)get_flow_in_units(242-255)get_flow_out_units(227-240)get_cost_units(298-305)temoa/types/model_types.py (1)
FlowType(275-282)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(133-136)temoa/model_checking/unit_checking/relations_checker.py (5)
check_cost_tables(353-522)check_efficiency_table(71-156)check_inter_table_relations(169-331)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
tests/test_unit_propagation.py (2)
temoa/_internal/table_writer.py (1)
unit_propagator(131-150)temoa/model_checking/unit_checking/unit_propagator.py (6)
UnitPropagator(29-337)has_unit_data(328-337)get_flow_out_units(227-240)get_flow_in_units(242-255)get_capacity_units(269-281)get_cost_units(298-305)
🪛 Ruff (0.14.8)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
592-595: Avoid specifying long messages outside the exception class
(TRY003)
598-601: Avoid specifying long messages outside the exception class
(TRY003)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
188-188: Unnecessary pass statement
Remove unnecessary pass
(PIE790)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
263-263: Unpacked variable major_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
263-263: Unpacked variable minor_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
306-306: datetime.datetime.now() called without a tz argument
(DTZ005)
tests/test_unit_scavenging.py
34-34: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
temoa/model_checking/unit_checking/unit_propagator.py
178-178: Possible SQL injection vector through string-based query construction
(S608)
tests/test_unit_propagation.py
188-188: Possible SQL injection vector through string-based query construction
(S608)
196-196: Possible SQL injection vector through string-based query construction
(S608)
🔇 Additional comments (15)
docs/source/unit_checking.rst (1)
1-427: Well-structured and comprehensive documentation.The documentation thoroughly covers the unit checking subsystem including enabling, format requirements, common pitfalls, testing guidance, and reference materials. The examples are practical and the troubleshooting section will help users diagnose issues.
tests/conftest.py (3)
44-71: Clean refactoring of database source mapping.The tuple-based mapping
(source_dir, source_file, output_file)cleanly separates the tutorial assets from test data sources, maintaining a single source of truth for unit-compliant data.
74-96: Well-implemented database creation helper.The helper properly uses context managers for connection handling, cleanly separates database creation from modification, and handles the SQL source loading correctly.
116-169: Good test variation coverage for unit checking.The 7 test variations systematically cover key error scenarios: invalid currency, energy units in capacity tables, missing parentheses, unknown units, mismatched outputs, and bad composite currency. This provides solid coverage for the unit validation logic.
temoa/model_checking/unit_checking/unit_propagator.py (4)
175-196: SQL query uses hardcoded table names - no injection risk.The static analysis flags line 178 for potential SQL injection, but
tableis iterated from the hardcodedcost_tableslist, not from user input. This is a false positive.The regex-based cost unit extraction correctly handles complex formats like
"kWh/day / (GW)"by splitting at\s*/\s*\(first.
62-102: Robust lookup building with graceful degradation.The lookup building correctly catches specific exceptions and falls back to empty dictionaries, ensuring the propagator remains functional even with partial data. The conditional building of
_tech_io_unitsbased on_commodity_unitsavailability is a good dependency check.
141-163: Effective regex-based capacity unit extraction.The word-boundary patterns correctly avoid false matches (e.g.,
GWhwon't matchGW). The pattern list covers standard power unit variations and prioritizes common abbreviations.
227-325: Clean getter API with consistent fallback behavior.All getters properly check for lookup availability before access and use the Pint
f'{unit:~}'format to preserve unit symbols. Thehas_unit_dataproperty provides a useful availability check.tests/test_unit_scavenging.py (1)
22-43: Good test coverage for extraction edge cases.The tests cover simple extraction, complex nested expressions, expressions with
/year, different power units (MW), energy unit exclusion (GWh), and no-match cases. This provides solid coverage of the extraction logic.temoa/_internal/table_writer.py (5)
130-150: Well-implemented lazy initialization with graceful fallback.The property correctly:
- Performs lazy import to avoid circular dependencies
- Catches relevant exceptions (ImportError, sqlite3.Error, RuntimeError)
- Logs debug info with stack trace for troubleshooting
- Checks
has_unit_datato provide useful feedback- Returns None on failure, allowing callers to proceed without units
545-568: Correct flow unit routing logic.The helper correctly routes:
FlowType.IN→ input commodity unitsFlowType.OUT,CURTAIL,FLEX→ output commodity unitsThis aligns with the PR documentation stating
output_flow_in → commodity.units (input commodity)andoutput_flow_out → commodity.units (output commodity).
594-614: Summary flow table doesn't propagate units.The
_insert_summary_flow_resultsmethod hardcodes'units': Noneat line 609. If this is intentional (e.g., because the summary aggregates across different commodities), consider adding a comment explaining why. Otherwise, units could be propagated using the same pattern aswrite_flow_tables.records.append( { 'scenario': scenario, 'region': r, 'sector': self.tech_sectors.get(t), 'period': p, 'input_comm': i, 'tech': t, 'vintage': v, 'output_comm': o, 'flow': val, - 'units': None, + # Units omitted for summary table as it aggregates across time slices + # (output commodity units could be added if needed) + 'units': None, } )
347-373: Storage level unit propagation implemented correctly.Units are derived from the stored commodity via
get_storage_units(tech), which looks up the output commodity for storage technologies and returns its units.
431-489: Capacity results correctly propagate units.All three capacity tables (built, net, retired) use
get_capacity_units(tech)to populate the units field, which derives units fromexisting_capacityorcost_investtables.tests/test_unit_propagation.py (1)
32-35: The review comment is incorrect.has_unit_datais defined as a@propertyand should be accessed as an attribute, not called as a method. The current assertionassert propagator.has_unit_data is Trueis correct. The proposed fix would cause a TypeError since properties are not callable.Likely an incorrect or invalid review comment.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
674423e to
32a72e0
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
temoa/_internal/table_writer.py (1)
598-611: Summary flow table missing unit propagation.The
_insert_summary_flow_resultsmethod hardcodes'units': Noneon line 609, while other flow/capacity methods properly callunit_propagator. Consider adding unit propagation here for consistency:records.append( { 'scenario': scenario, 'region': r, 'sector': self.tech_sectors.get(t), 'period': p, 'input_comm': i, 'tech': t, 'vintage': v, 'output_comm': o, 'flow': val, - 'units': None, + 'units': self._get_flow_units(FlowType.OUT, i, o), } )
♻️ Duplicate comments (10)
tests/test_unit_scavenging.py (2)
4-19: Test helper duplicates production parsing logic.This
extract_capacity_unitfunction reimplements the same regex-based extraction that exists inUnitPropagator._extract_capacity_unit. Consider importing and testing the production method directly to avoid "testing the test" and ensure the actual implementation is validated.
34-35: Remove unused# noqa: N802directive or rename function.Ruff flags this as an unused noqa directive. Either remove it or rename the test to snake_case:
-def test_extract_MW(): # noqa: N802 +def test_extract_mw():tests/conftest.py (1)
182-188: Remove unnecessarypassstatement.The
passafter the warning log is redundant since theexceptblock already contains statements.logger.warning( 'Unit test databases not created: source SQL not found. ' 'Unit checking tests will be skipped.' ) - passdocs/source/unit_checking.rst (1)
94-119: Consider clarifying scope of the strict ratio format requirement.The warning about regex parsing with strict format requirements is good, but could benefit from explicitly noting that this format is only enforced for efficiency and cost tables (as mentioned on line 97), not all unit entries. This would reduce confusion for readers.
temoa/cli.py (1)
358-382: Potential race condition when selecting newest report.If multiple
check-unitsruns occur concurrently to the same output directory,reports[0]may not be the report from this invocation. Consider havingscreen()return the report path directly, or generate a deterministic filename based on the database name.temoa/model_checking/unit_checking/screener.py (3)
30-55: Add error handling for missing/invalid metadata table.If the database lacks a
metadatatable or contains non-integer values, this function will raise an unhandled exception, bypassing the intended "DB version failed" reporting path.def _check_db_version(conn: sqlite3.Connection, report_entries: list[str]) -> tuple[bool, int, int]: msg = '======== Units Check 1 (DB Version): Started ========' report_entries.extend((msg, '\n')) logger.info(msg) - data = conn.execute('SELECT element, value FROM metadata').fetchall() - meta_data = dict(data) - major = int(meta_data.get('DB_MAJOR', 0)) - minor = int(meta_data.get('DB_MINOR', 0)) + try: + data = conn.execute('SELECT element, value FROM metadata').fetchall() + meta_data = dict(data) + major = int(meta_data.get('DB_MAJOR', 0)) + minor = int(meta_data.get('DB_MINOR', 0)) + except (sqlite3.Error, ValueError, TypeError) as e: + major, minor = 0, 0 + msg = f'Units Check 1 (DB Version): Failed. Unable to read metadata table: {e}' + report_entries.extend((msg, '\n')) + logger.warning(msg) + return False, major, minor
261-268: Prefix unused version variables with underscore.The
major_versionandminor_versionvariables are unpacked but never used. Prefix them with underscores to indicate intentional non-use:- db_version_ok, major_version, minor_version = _check_db_version(conn, report_entries) + db_version_ok, _major_version, _minor_version = _check_db_version(conn, report_entries)
302-310: Consider timezone-aware timestamp for cross-system consistency.The
datetime.now()call uses local time without timezone info. For consistent report ordering across systems:- from datetime import datetime + from datetime import datetime, timezone - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S')tests/test_unit_propagation.py (2)
25-29: Fixture returns propagator backed by closed connection.The
with sqlite3.connect(...)block closes the connection before returning theUnitPropagator. IfUnitPropagatorbecomes lazier or adds queries later, tests will fail unexpectedly. Use a generator fixture:@pytest.fixture -def propagator() -> UnitPropagator: +def propagator(): """Create a UnitPropagator from the valid units database.""" - with sqlite3.connect(VALID_UNITS_DB) as conn: - return UnitPropagator(conn) + conn = sqlite3.connect(VALID_UNITS_DB) + try: + yield UnitPropagator(conn) + finally: + conn.close()
206-221: Cost unit assertion should be case-insensitive.The assertion
'dollar' in sample[0] or 'Eur' in sample[0]mixes case sensitivity and misses 'USD':- assert 'dollar' in sample[0] or 'Eur' in sample[0] + units_lower = sample[0].lower() + assert 'dollar' in units_lower or 'usd' in units_lower or 'eur' in units_lower
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (9)
docs/source/unit_checking.rst(1 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/cli.py(2 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_propagation.py(1 hunks)tests/test_unit_scavenging.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
tests/test_unit_propagation.py (2)
temoa/_internal/table_writer.py (1)
unit_propagator(131-150)temoa/model_checking/unit_checking/unit_propagator.py (6)
UnitPropagator(29-333)has_unit_data(324-333)get_flow_out_units(223-236)get_flow_in_units(238-251)get_capacity_units(265-277)get_cost_units(294-301)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(133-136)temoa/model_checking/unit_checking/relations_checker.py (3)
check_cost_tables(353-522)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
🪛 Ruff (0.14.8)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
590-593: Avoid specifying long messages outside the exception class
(TRY003)
596-599: Avoid specifying long messages outside the exception class
(TRY003)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
188-188: Unnecessary pass statement
Remove unnecessary pass
(PIE790)
tests/test_unit_propagation.py
116-116: Unpacked variable location is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
192-192: Possible SQL injection vector through string-based query construction
(S608)
200-200: Possible SQL injection vector through string-based query construction
(S608)
tests/test_unit_scavenging.py
34-34: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
temoa/model_checking/unit_checking/unit_propagator.py
178-178: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
263-263: Unpacked variable major_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
263-263: Unpacked variable minor_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
306-306: datetime.datetime.now() called without a tz argument
(DTZ005)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.13)
🔇 Additional comments (27)
tests/test_unit_scavenging.py (1)
22-43: Test coverage looks comprehensive for the extraction patterns.The tests validate key scenarios: simple extraction, complex expressions, parenthesized units, energy unit filtering (
GWhcorrectly returnsNone), and non-matching inputs. Good coverage of edge cases.tests/conftest.py (2)
74-95: Clean database creation helper with proper resource management.The
create_unit_test_db_from_sqlfunction correctly uses a context manager for the SQLite connection, handles file cleanup, and applies modifications atomically. Good implementation.
116-168: Well-structured unit test variations.The test database variations cover key validation scenarios: valid baseline, invalid currency, energy-in-capacity, missing parentheses, unknown units, mismatched outputs, and bad composite currency. This provides good coverage for the unit checking pipeline.
docs/source/unit_checking.rst (1)
1-427: Comprehensive and well-organized documentation.The documentation thoroughly covers:
- Configuration and CLI usage
- Five-stage validation pipeline
- Unit formatting requirements with clear examples
- Common pitfalls with solutions
- Testing guidance and report interpretation
- Quick reference and dimension tables
This will be valuable for users adopting the unit checking feature.
temoa/_internal/table_writer.py (2)
130-150: Well-designed lazy initialization with graceful degradation.The
unit_propagatorproperty correctly:
- Uses lazy initialization to avoid overhead when units aren't needed
- Catches relevant exceptions (
ImportError,sqlite3.Error,RuntimeError)- Logs debug info without failing the write process
- Returns
Nonefor graceful fallbackThis ensures databases without unit data continue to work seamlessly.
545-568: Clean flow units helper with correct type dispatch.The
_get_flow_unitsmethod properly distinguishes between input flows (which use input commodity units) and output/curtailment/flex flows (which use output commodity units). The early return pattern whenunit_propisNoneis clean.temoa/cli.py (2)
292-394: Well-implementedcheck-unitsCLI command.The command properly:
- Validates database existence via typer argument constraints
- Creates output directory with error handling
- Runs the screener and reports results
- Exits with code 1 on failures (enabling CI/script integration)
- Streams report summary efficiently (lines 369-377)
The implementation aligns with other CLI commands (
validate,run) in error handling patterns.
550-610: Tutorial resource generation from SQL source is a good architectural choice.Generating the database from the SQL source file ensures the tutorial always uses the latest schema with unit-compliant data (single source of truth). The fallback path handling for development environments is appropriate.
temoa/model_checking/unit_checking/screener.py (1)
244-299: Well-structured five-stage screening pipeline.The
screen()function properly:
- Validates database file existence before processing
- Runs checks sequentially with appropriate early exit on DB version failure
- Accumulates errors across all checks
- Writes consolidated report when
report_diris provided- Returns clear success/failure indicator
The modular helper functions improve testability and maintainability.
tests/test_unit_propagation.py (2)
187-203: SQL injection warnings are false positives.The Ruff S608 warnings on lines 192 and 200 are false positives since
tablecomes from a hardcoded list (['output_built_capacity', 'output_net_capacity']) at line 190, not user input.
77-91: Good graceful fallback test with proper connection management.The in-memory database test correctly uses a context manager and validates that
UnitPropagatorhandles minimal schemas without crashing. This ensures robustness for databases without unit data.temoa/model_checking/unit_checking/unit_propagator.py (8)
1-27: LGTM!Clean module structure with proper docstring, standard library imports, and appropriate use of
TYPE_CHECKINGto avoid runtime import ofpint.Unit.
29-60: LGTM!Clean class design with proper initialization, type hints, and clear documentation. The lazy-build pattern in
__init__keeps the API simple.
62-102: LGTM!Robust error handling with graceful fallbacks ensures backward compatibility. Debug-level logging is appropriate for diagnosing issues without noise.
104-139: LGTM!The precedence logic (existing_capacity → cost_invest fallback) is clear. The silent exception handling is acceptable here since these tables may legitimately not exist in all databases.
141-163: LGTM!The regex-based extraction with word boundaries correctly addresses the complex unit parsing issue. This safely extracts 'GW' from formats like
'Mdollar / (PJ^2 / GW)'.
165-196: Static analysis false positive - table names are hardcoded.The SQL injection warning at line 178 is a false positive. The
tablevariable iterates over a hardcoded list['cost_invest', 'cost_fixed', 'cost_variable', 'cost_emission'], not user-controlled input. The implementation correctly handles complex numerator formats with the regex split.
198-221: LGTM!The query correctly identifies storage technologies via the
'ps'flag and maps them to their output commodities for unit derivation.
223-333: LGTM!The getter methods correctly use Pint's
{:~}format specifier to preserve unit symbols (e.g., 'GW' instead of 'gigawatt'), addressing the past review feedback. Consistent None-fallback pattern ensures backward compatibility.temoa/tutorial_assets/utopia.sql (8)
196-197: Emission commodity units correctly set to 'Mt'.The past review concern about emission commodities (
co2,nox) incorrectly using'PJ'has been addressed. Mass units ('Mt') are now correctly specified, which is consistent with theemission_activitytable units ('Mt / (PJ)').
162-177: LGTM!The
capacity_to_activityunits use consistent'PJ / (GW * year)'format representing activity per unit capacity per year, following the established ratio notation convention.
349-381: Verify the dimensional correctness of cost_invest units.The cost_invest units use
'Mdollar / (PJ^2 / GW)', where the denominatorPJ^2 / GWis an unusual dimensional combination. While the_extract_capacity_unitmethod will correctly extract'GW'for capacity propagation, consider verifying that this unit representation aligns with the intended physical interpretation (typically cost per unit capacity, e.g.,'Mdollar / GW').
526-594: LGTM!Efficiency units consistently use
'PJ / (PJ)'format, which is dimensionally correct for energy efficiency ratios (output/input).
635-648: LGTM!Emission activity units correctly use
'Mt / (PJ)'(mass per energy), which is consistent with the emission commodity units ('Mt') defined in the commodity table.
703-775: LGTM!Existing capacity uses
'GW'consistently, and lifetime tables correctly use'year'units. These align with the unit propagator's expectations.
1156-1360: LGTM!Output table schema changes consistently add nullable
units TEXTcolumns, enabling unit propagation while maintaining backward compatibility with existing workflows.
831-841: LGTM!Capacity limit units consistently use
'GW', matching the existing_capacity table conventions.
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
32a72e0 to
1041d90
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (5)
tests/test_unit_scavenging.py (2)
4-19: Avoid duplicating production parsing logic in tests.The
extract_capacity_unitfunction duplicates parsing logic that should exist in production code. This creates a "testing the test" scenario where the test implementation might diverge from the actual production behavior.Consider refactoring to either:
- Call the production implementation (e.g.,
UnitPropagator._extract_capacity_unit()or similar utility)- Move this extractor into a shared utility module that both production and tests import
This ensures tests validate actual production behavior rather than a test-specific reimplementation.
34-34: Remove unused noqa directive.The
# noqa: N802comment is unnecessary since the N802 rule (function naming) is not enabled in your Ruff configuration.Apply this diff:
-def test_extract_MW() -> None: # noqa: N802 +def test_extract_mw() -> None: assert extract_capacity_unit('Mdollar / (MW)') == 'MW'Or simply remove the noqa if you want to keep the current naming.
temoa/model_checking/unit_checking/screener.py (2)
263-263: Prefix unused version variables with underscore.The
major_versionandminor_versionvariables are unpacked but never used in the subsequent code. Per Ruff RUF059, prefix them with underscores to indicate intentional non-use.Apply this diff:
- db_version_ok, major_version, minor_version = _check_db_version(conn, report_entries) + db_version_ok, _major_version, _minor_version = _check_db_version(conn, report_entries)
302-310: Consider using timezone-aware timestamp for consistency.The
datetime.now()call uses naive local time. For cross-system consistency and log correlation, consider using UTC:Apply this diff:
def _write_report(report_dir: Path, report_entries: list[str]) -> None: """write out a report if the path is specified""" - from datetime import datetime + from datetime import datetime, timezone - timestamp = datetime.now().strftime('%Y-%m-%d_%H%M%S') + timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%d_%H%M%S')This also addresses the Ruff DTZ005 warning.
tests/conftest.py (1)
177-189: Remove redundant pass statement.Line 188 contains an unnecessary
passstatement after the warning log. The pass adds no value since the except block already has a logger.warning statement.Apply this diff:
except FileNotFoundError: # Source DB not available; unit tests will be skipped via pytestmark logger.warning( 'Unit test databases not created: source SQL not found. ' 'Unit checking tests will be skipped.' ) - pass
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (9)
docs/source/unit_checking.rst(1 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/cli.py(2 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_propagation.py(1 hunks)tests/test_unit_scavenging.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
temoa/_internal/table_writer.py (2)
temoa/model_checking/unit_checking/unit_propagator.py (8)
UnitPropagator(29-333)has_unit_data(324-333)get_storage_units(303-321)get_emission_units(279-292)get_capacity_units(265-277)get_flow_in_units(238-251)get_flow_out_units(223-236)get_cost_units(294-301)temoa/types/model_types.py (1)
FlowType(275-282)
🪛 Ruff (0.14.8)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
263-263: Unpacked variable major_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
263-263: Unpacked variable minor_version is never used
Prefix it with an underscore or any other dummy variable pattern
(RUF059)
306-306: datetime.datetime.now() called without a tz argument
(DTZ005)
temoa/model_checking/unit_checking/unit_propagator.py
178-178: Possible SQL injection vector through string-based query construction
(S608)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
590-593: Avoid specifying long messages outside the exception class
(TRY003)
596-599: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_propagation.py
192-192: Possible SQL injection vector through string-based query construction
(S608)
200-200: Possible SQL injection vector through string-based query construction
(S608)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
188-188: Unnecessary pass statement
Remove unnecessary pass
(PIE790)
tests/test_unit_scavenging.py
34-34: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (18)
docs/source/unit_checking.rst (1)
1-427: LGTM! Comprehensive documentation for the unit checking subsystem.The documentation is thorough and well-structured, covering:
- Overview and enabling unit checking via configuration and CLI
- Five-stage validation pipeline with clear explanations
- Format requirements with warnings for critical gotchas
- Common pitfalls and troubleshooting guidance
- Quick reference and dimension reference sections
The past review comments regarding dimensional notation inconsistencies and scope clarifications have been addressed in previous commits. The documentation provides excellent guidance for users implementing and debugging unit checking.
temoa/_internal/table_writer.py (2)
95-151: Well-designed lazy initialization for unit propagation.The lazy
unit_propagatorproperty correctly:
- Defers import until needed (runtime import guard)
- Handles initialization failures gracefully by returning None
- Checks
has_unit_datato avoid using empty propagators- Logs debug information for troubleshooting
This ensures backward compatibility with databases lacking unit information and prevents crashes in environments where unit checking components are unavailable.
545-569: Clean flow units derivation logic.The
_get_flow_unitshelper correctly distinguishes between flow types:
- Input flows use input commodity units
- Output/curtailment/flex flows use output commodity units
- Returns None gracefully when unit propagator is unavailable
This aligns with the documented unit propagation mappings in the unit checking documentation.
temoa/cli.py (2)
292-394: Solid implementation of the check-units CLI command.The command correctly:
- Determines output directory with sensible defaults
- Runs the unit screening workflow via
screen()- Provides appropriate user feedback (success/issues/errors)
- Exits with non-zero code when issues found or errors occur (addresses past review feedback)
- Handles FileNotFoundError and general exceptions appropriately
The report summary preview (lines 369-377) is helpful for quick issue identification without opening the full report file.
550-611: Good refactoring to SQL-based database generation.Generating the tutorial database from the SQL source (single source of truth) instead of copying a pre-built binary improves:
- Reproducibility across different environments
- Version consistency with the latest schema
- Maintainability (one authoritative source)
The fallback handling for both package resources and development paths is robust.
temoa/model_checking/unit_checking/unit_propagator.py (3)
29-103: Well-structured unit propagator with robust error handling.The UnitPropagator class design is solid:
- Builds all lookup tables once at initialization
- Each lookup construction is wrapped in try-except with debug logging
- Gracefully degrades to empty dicts/None when data unavailable
- Preserves backward compatibility with databases lacking unit information
The use of
logger.debug(rather than warning/error) for lookup failures is appropriate since missing unit data is an expected scenario in legacy databases.
141-163: Capacity unit extraction correctly handles complex formats.The regex-based extraction with word boundaries (
\b) correctly:
- Avoids false matches (e.g., GWh matching GW)
- Handles complex denominators like "Mdollar / (PJ^2 / GW)" → extracts "GW"
- Returns None gracefully when no capacity unit found
This addresses the critical issue from past reviews where simple string splitting failed on nested denominators.
223-236: Unit formatting with :~ preserves input symbols.The use of
f'{unit:~}'correctly applies Pint's compact format to preserve unit symbols (e.g., "GW" instead of "gigawatt"). This ensures output tables display units consistent with input table entries.This addresses the major issue from past reviews where
str(unit)returned canonical names instead of symbols.tests/test_unit_propagation.py (3)
25-30: Note: Fixture returns propagator backed by closed connection.The fixture uses a
withstatement that closes the connection before returning the propagator. While the current implementation works because UnitPropagator builds all lookups eagerly at initialization, this pattern is fragile.If UnitPropagator ever becomes lazier (deferred queries), tests will fail mysteriously. Consider using a yield-based fixture to keep the connection open:
@pytest.fixture def propagator() -> UnitPropagator: """Create a UnitPropagator from the valid units database.""" conn = sqlite3.connect(VALID_UNITS_DB) try: yield UnitPropagator(conn) finally: conn.close()This is a best practice for resource management in pytest fixtures, even though it's not currently breaking.
99-152: Good addition of solver availability check.The solver availability guard (lines 116-118) prevents fixture failures in environments without
appsi_highs. The integration test design is solid:
- Creates isolated working directory
- Copies source database
- Generates config dynamically
- Runs full model solve
- Returns output database for verification
This addresses the past review concern about missing solver handling.
187-204: Comprehensive capacity unit propagation test.The test correctly:
- Verifies both output_built_capacity and output_net_capacity tables
- Checks for non-null units on populated rows
- Validates expected unit format (GW for Utopia)
- Provides clear assertion messages for debugging
The SQL injection warnings from Ruff are false positives—table names come from the hardcoded list on line 190.
temoa/tutorial_assets/utopia.sql (7)
162-200: LGTM! Unit additions are physically consistent.The capacity-to-activity units correctly express the conversion factor (energy per power per time), and the commodity units properly distinguish between energy ('PJ') and mass ('Mt') units. The emission commodities now correctly use 'Mt' as addressed in the previous review.
396-467: LGTM! Cost_variable and demand units are correct.Variable costs use 'Mdollar / (PJ)' (cost per energy), and demand uses 'PJ' (energy). Both are dimensionally consistent and physically appropriate.
531-594: LGTM! Efficiency units are correct.The efficiency table correctly uses 'PJ / (PJ)' representing dimensionless energy conversion ratios, which is the expected format for efficiency parameters.
635-648: LGTM! Emission_activity units are consistent with commodity units.The emission activity units 'Mt / (PJ)' correctly express mass of emissions per unit of energy, and are consistent with the emission commodity units ('Mt') for proper propagation to output_emission.
703-775: LGTM! Capacity and lifetime units are correct.The existing_capacity uses 'GW' (power), and the lifetime tables use 'year' (time). Both are straightforward and dimensionally correct.
831-841: LGTM! Limit_capacity units are correct.The capacity limit constraints correctly use 'GW' (power) units, consistent with the existing_capacity table.
1167-1359: LGTM! Output table unit columns support propagation.The addition of
unitscolumns to output_built_capacity, output_cost, output_curtailment, output_emission, output_flow_in, output_flow_out, output_net_capacity, output_retired_capacity, and output_storage_level correctly enables unit propagation from input tables to output results as described in the PR objectives.
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1970,40.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1980,40.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1990,40.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1970,70.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1980,70.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1990,70.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',2000,70.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',1980,100.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',1990,100.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',2000,100.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',2010,100.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E21',1990,500.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E21',1990,500.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',1990,500.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E21',2000,500.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',2000,500.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',2010,500.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E31',1980,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E31',1990,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',1980,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',1990,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',2000,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',1980,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',1990,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',2000,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',2010,75.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E51',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E51',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',2010,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1960,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1970,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1970,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',2010,30.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1970,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1980,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1990,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',1980,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',1990,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',2000,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',1990,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',2000,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',2010,1.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RL1',1980,9.46,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RL1',1990,9.46,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RL1',2000,9.46,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RL1',2010,9.46,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1970,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1980,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1990,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',1980,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',1990,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',2000,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXD',2000,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXD',2010,52.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXE',1990,100.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXE',1990,90.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXE',2000,90.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXE',2000,80.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXE',2010,80.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1970,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1980,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1990,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',1980,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',1990,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',2000,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXG',2000,48.0,'Mdollar / (PJ^2 / GW / year)',''); | ||
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXG',2010,48.0,'Mdollar / (PJ^2 / GW / year)',''); |
There was a problem hiding this comment.
Correct the cost_fixed units: remove PJ² term.
The units 'Mdollar / (PJ^2 / GW / year)' contain a dimensionally incorrect PJ² (energy squared) term. Fixed O&M costs should be currency per capacity per time, not involving energy squared.
Apply this pattern to correct all cost_fixed entries:
-INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (PJ^2 / GW / year)','');
+INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (GW * year)','');This correction applies to all 82 cost_fixed rows (lines 255-336). The corrected units will properly propagate to output_cost.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1970,40.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1980,40.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1990,40.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1970,70.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1980,70.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1990,70.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',2000,70.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',1980,100.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',1990,100.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',2000,100.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',2010,100.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E21',1990,500.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E21',1990,500.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',1990,500.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E21',2000,500.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',2000,500.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',2010,500.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E31',1980,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E31',1990,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',1980,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',1990,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',2000,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',1980,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',1990,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',2000,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',2010,75.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E51',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E51',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',2010,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1960,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1970,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1970,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',1980,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',1990,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',2000,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',2010,30.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1970,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1980,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1990,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',1980,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',1990,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',2000,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',1990,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',2000,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',2010,1.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RL1',1980,9.46,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RL1',1990,9.46,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RL1',2000,9.46,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RL1',2010,9.46,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1970,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1980,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1990,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',1980,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',1990,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',2000,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXD',2000,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXD',2010,52.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXE',1990,100.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXE',1990,90.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXE',2000,90.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXE',2000,80.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXE',2010,80.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1970,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1980,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1990,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',1980,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',1990,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',2000,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXG',2000,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXG',2010,48.0,'Mdollar / (PJ^2 / GW / year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1970,40.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1980,40.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1990,40.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1970,70.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1980,70.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',1990,70.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E01',2000,70.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',1980,100.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',1990,100.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',2000,100.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E01',2010,100.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E21',1990,500.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E21',1990,500.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',1990,500.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E21',2000,500.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',2000,500.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E21',2010,500.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E31',1980,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E31',1990,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',1980,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',1990,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E31',2000,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',1980,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',1990,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',2000,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E31',2010,75.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E51',1980,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E51',1990,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',1980,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',1990,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E51',2000,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',1980,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',1990,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',2000,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E51',2010,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1960,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1970,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1980,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'E70',1990,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1970,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1980,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',1990,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'E70',2000,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',1980,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',1990,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',2000,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'E70',2010,30.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1970,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1980,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RHO',1990,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',1980,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',1990,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RHO',2000,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',1990,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',2000,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RHO',2010,1.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RL1',1980,9.46,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'RL1',1990,9.46,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'RL1',2000,9.46,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'RL1',2010,9.46,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1970,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1980,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXD',1990,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',1980,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',1990,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXD',2000,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXD',2000,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXD',2010,52.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXE',1990,100.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXE',1990,90.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXE',2000,90.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXE',2000,80.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXE',2010,80.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1970,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1980,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',1990,'TXG',1990,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',1980,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',1990,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2000,'TXG',2000,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXG',2000,48.0,'Mdollar / (GW * year)',''); | |
| INSERT INTO "cost_fixed" VALUES('utopia',2010,'TXG',2010,48.0,'Mdollar / (GW * year)',''); |
🤖 Prompt for AI Agents
In temoa/tutorial_assets/utopia.sql lines 255-336, the cost_fixed unit string
'Mdollar / (PJ^2 / GW / year)' is dimensionally wrong; replace that unit in all
82 cost_fixed INSERT rows with a capacity-time unit such as 'Mdollar / GW /
year' so fixed O&M is currency per capacity per time; update every occurrence in
these lines to exactly the new unit so output_cost receives the corrected units.
| INSERT INTO "cost_invest" VALUES('utopia','E01',1990,2000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E01',2000,1300.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E01',2010,1200.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E21',1990,5000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E21',2000,5000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E21',2010,5000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E31',1990,3000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E31',2000,3000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E31',2010,3000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E51',1990,900.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E51',2000,900.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E51',2010,900.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E70',1990,1000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E70',2000,1000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','E70',2010,1000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','RHE',1990,90.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','RHE',2000,90.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','RHE',2010,90.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','RHO',1990,100.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','RHO',2000,100.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','RHO',2010,100.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','SRE',1990,100.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','SRE',2000,100.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','SRE',2010,100.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXD',1990,1044.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXD',2000,1044.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXD',2010,1044.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXE',1990,2000.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXE',2000,1750.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXE',2010,1500.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXG',1990,1044.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXG',2000,1044.0,'Mdollar / (PJ^2 / GW)',''); | ||
| INSERT INTO "cost_invest" VALUES('utopia','TXG',2010,1044.0,'Mdollar / (PJ^2 / GW)',''); |
There was a problem hiding this comment.
Correct the cost_invest units: remove PJ² term.
The units 'Mdollar / (PJ^2 / GW)' contain a dimensionally incorrect PJ² (energy squared) term. Investment/capital costs should be currency per capacity, not involving energy squared.
Apply this pattern to correct all cost_invest entries:
-INSERT INTO "cost_invest" VALUES('utopia','E01',1990,2000.0,'Mdollar / (PJ^2 / GW)','');
+INSERT INTO "cost_invest" VALUES('utopia','E01',1990,2000.0,'Mdollar / GW','');This correction applies to all 33 cost_invest rows (lines 349-381). The corrected units will properly propagate to output_cost.
Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In temo a/tutorial_assets/utopia.sql around lines 349 to 381, the cost_invest
rows use the incorrect unit string 'Mdollar / (PJ^2 / GW)'; remove the erroneous
energy-squared term and replace that unit with the correct currency-per-capacity
unit 'Mdollar / GW' for all 33 INSERT rows so output_cost receives the corrected
units consistently.
1041d90 to
ee3d3fb
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
tests/test_unit_scavenging.py (1)
21-21: Remove unusednoqadirective.The
# noqa: N802comment is unnecessary becausetest_extract_MWis a valid pytest test function name (thetest_prefix makes it compliant with naming conventions).Apply this diff:
-def test_extract_MW() -> None: # noqa: N802 +def test_extract_MW() -> None: """Test extracting 'MW' as a capacity unit.""" assert extract_capacity_unit_func('Mdollar / (MW)') == 'MW'temoa/model_checking/unit_checking/screener.py (1)
30-56: Add defensive error handling for missing or malformed metadata table.The
_check_db_versionfunction will raise an unhandled exception if themetadatatable is missing or contains non-integer values. This bypasses the intended "DB version failed" reporting path. Wrap the metadata query and parsing in a try-except block:def _check_db_version(conn: sqlite3.Connection, report_entries: list[str]) -> tuple[bool, int, int]: """ Check the database version and return success/failure indicator and version info :param conn: sqlite3 database connection :param report_entries: list to append report messages to :return: tuple of (success_indicator, major_version, minor_version) """ msg = '======== Units Check 1 (DB Version): Started ========' report_entries.extend((msg, '\n')) logger.info(msg) - data = conn.execute('SELECT element, value FROM metadata').fetchall() - meta_data = dict(data) - major = int(meta_data.get('DB_MAJOR', 0)) - minor = int(meta_data.get('DB_MINOR', 0)) + try: + data = conn.execute('SELECT element, value FROM metadata').fetchall() + meta_data = dict(data) + major = int(meta_data.get('DB_MAJOR', 0)) + minor = int(meta_data.get('DB_MINOR', 0)) + except (sqlite3.Error, ValueError, TypeError) as e: + major, minor = 0, 0 + msg = f'Units Check 1 (DB Version): Failed. Unable to read metadata table: {e}' + report_entries.extend((msg, '\n')) + logger.warning(msg) + return False, major, minor🤖 Prompt for AI Agents
In temoa/model_checking/unit_checking/screener.py around lines 30 to 56, the function _check_db_version will raise if the metadata table is missing or contains non-integer values; catch sqlite3.DatabaseError/OperationalError from the SELECT and ValueError/TypeError from int() conversions, and handle missing keys defensively (use defaults), then append the existing "Failed" message to report_entries and logger.warning with the error detail and return (False, 0, 0) so the screener does not crash but reports DB version failure as intended.temoa/tutorial_assets/utopia.sql (2)
255-336: Critical: Dimensionally incorrect PJ² units remain in cost_fixed table.All 82 cost_fixed entries use
'Mdollar / (PJ^2 / GW / year)'which contains a dimensionally incorrect PJ² (energy squared) term. Fixed O&M costs should be currency per capacity per time.The correct dimensional analysis:
- Fixed O&M cost = currency / (capacity × time)
- With given values: Mdollar / (GW × year)
The
PJ^2term appears to be an artifact that doesn't represent a valid physical dimension.Apply this pattern to all 82 cost_fixed rows:
-INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (PJ^2 / GW / year)',''); +INSERT INTO "cost_fixed" VALUES('utopia',1990,'E01',1960,40.0,'Mdollar / (GW * year)','');🤖 Prompt for AI Agents
In temoa/tutorial_assets/utopia.sql lines 255-336, all 82 cost_fixed INSERT statements use the unit string 'Mdollar / (PJ^2 / GW / year)' which contains a dimensionally incorrect energy-squared term; replace this unit in all 82 rows with the correct currency-per-capacity-per-time unit 'Mdollar / (GW * year)' so fixed O&M costs have proper dimensional consistency and output_cost receives correct units.
349-381: Critical: Dimensionally incorrect PJ² units remain in cost_invest table.All 33 cost_invest entries use
'Mdollar / (PJ^2 / GW)'which contains a dimensionally incorrect PJ² (energy squared) term. Investment/capital costs should be currency per capacity.The correct dimensional analysis:
- Investment cost = currency / capacity
- With given values: Mdollar / GW
The
PJ^2term appears to be an artifact that doesn't represent a valid physical dimension.Apply this pattern to all 33 cost_invest rows:
-INSERT INTO "cost_invest" VALUES('utopia','E01',1990,2000.0,'Mdollar / (PJ^2 / GW)',''); +INSERT INTO "cost_invest" VALUES('utopia','E01',1990,2000.0,'Mdollar / GW','');🤖 Prompt for AI Agents
In temoa/tutorial_assets/utopia.sql lines 349-381, all 33 cost_invest INSERT statements use the unit string 'Mdollar / (PJ^2 / GW)' which contains a dimensionally incorrect energy-squared term; replace this unit in all 33 rows with the correct currency-per-capacity unit 'Mdollar / GW' so investment costs have proper dimensional consistency and output_cost receives correct units.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (13)
docs/source/unit_checking.rst(1 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(2 hunks)temoa/model_checking/unit_checking/entry_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_propagation.py(1 hunks)tests/test_unit_scavenging.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (8)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(140-142)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/entry_checker.py (1)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(140-142)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
temoa/model_checking/unit_checking/screener.py (2)
temoa/model_checking/unit_checking/relations_checker.py (5)
check_cost_tables(353-522)check_efficiency_table(71-156)check_inter_table_relations(169-331)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
tests/test_unit_scavenging.py (1)
temoa/model_checking/unit_checking/unit_propagator.py (2)
UnitPropagator(29-334)_extract_capacity_unit(142-164)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(164-172)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-76)validate_units_expression(39-49)validate_units_format(24-36)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-393)temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
🪛 Ruff (0.14.8)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
temoa/model_checking/unit_checking/entry_checker.py
47-47: Consider moving this statement to an else block
(TRY300)
60-60: Possible SQL injection vector through string-based query construction
(S608)
63-67: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
590-593: Avoid specifying long messages outside the exception class
(TRY003)
596-599: Avoid specifying long messages outside the exception class
(TRY003)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_propagation.py
192-192: Possible SQL injection vector through string-based query construction
(S608)
200-200: Possible SQL injection vector through string-based query construction
(S608)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/unit_propagator.py
179-179: Possible SQL injection vector through string-based query construction
(S608)
tests/test_unit_scavenging.py
21-21: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
26-26: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
temoa/model_checking/unit_checking/table_checker.py
115-115: Avoid specifying long messages outside the exception class
(TRY003)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
🔇 Additional comments (14)
temoa/model_checking/unit_checking/table_checker.py (1)
1-116: LGTM! Unit checking logic is well-structured and previously identified issues have been addressed.The table checker implements a clear validation pipeline:
- Character validation with accurate error messaging
- Format validation (SINGLE_ELEMENT vs RATIO_ELEMENT)
- Registry compliance checking via Pint
- Capacity table time-dimension validation (correctly checks
> -3)- Result assembly with proper unit object handling
All previously flagged issues have been properly resolved:
- Character validation error message now accurately lists allowed characters
- Time exponent comparison uses
float(time_exponent)directly without.real- ValueError uses f-string formatting
- Capacity validation correctly flags energy units (time exponent > -3)
The static analysis hint (TRY003 on line 115) is a false positive—the ValueError message is appropriately concise for this context.
temoa/model_checking/unit_checking/screener.py (1)
244-320: LGTM! Screening orchestration is well-structured and previously identified issues have been addressed.The
screenfunction implements a clear five-stage validation pipeline with proper error handling and reporting:
- DB version check (with early exit on failure)
- Units entries validation across all input tables
- Efficiency table validation (building tech I/O LUT)
- Related tables validation (activity, capacity, commodity)
- Cost tables validation
All previously flagged issues have been properly resolved:
- Unused
verboseflag removed- Helper functions properly extracted
- Unused version parameters removed
- Version variables correctly prefixed with underscores
- Timezone-aware timestamp implemented (line 306)
- Report directory handling simplified
The static analysis hints are false positives:
- TRY003 (line 256): The FileNotFoundError message is appropriately specific
- The timezone-aware timestamp issue (DTZ005) has been addressed with
datetime.UTCtests/test_unit_propagation.py (3)
1-92: LGTM! Unit tests provide good coverage and previously identified issues have been addressed.The unit tests properly validate UnitPropagator functionality:
- Fixture uses
yieldto keep connection alive during tests (line 28-29)- Tests cover all getter methods (flow_out, flow_in, capacity, cost, emission, storage)
- Empty database graceful fallback test uses context manager (line 80)
- Missing commodity/tech tests verify None returns
All previously flagged issues have been properly resolved:
- Connection lifecycle managed correctly with context managers
- Propagator fixture no longer returns object backed by closed connection
99-152: LGTM! Integration test fixture is well-designed with proper safeguards.The
solved_db_with_unitsfixture implements a comprehensive end-to-end test:
- Source database availability check (line 113-114)
- Solver availability check added (lines 116-118) ✓
- Creates isolated test environment with copied database
- Uses
as_posix()for cross-platform path compatibility (line 130)- Runs full model solve via TemoaSequencer
Previously flagged issues have been properly resolved:
- Solver availability check prevents fixture failures in environments without
appsi_highs- Unused
locationvariable correctly prefixed with underscore (line 116)
154-222: LGTM! Integration tests thoroughly validate unit propagation.The integration tests verify units are properly populated in all output tables:
test_units_propagated_to_output_flow_tables: checks output_flow_out, output_curtailment, output_flow_intest_units_propagated_to_capacity_tables: validates output_built_capacity and output_net_capacity with specific unit assertions (GW)test_units_propagated_to_cost_table: confirms cost units include currency dimensionThe SQL injection warnings (S608 on lines 192, 200) are false positives—the
tablevariable comes from a hardcoded list on line 190, not user input.All tests use context managers for proper connection cleanup.
temoa/model_checking/unit_checking/unit_propagator.py (4)
29-103: LGTM! UnitPropagator initialization and lookup building is robust.The class design is well-structured:
- Clear initialization with defensive error handling for all lookups (lines 64-102)
- Multiple lookup tables built (commodity, tech I/O, C2A, capacity, cost, storage)
- Graceful degradation when tables are missing or malformed
- Debug-level logging for failures (doesn't clutter production logs)
The initialization properly handles databases without unit information, ensuring backward compatibility.
104-164: LGTM! Capacity unit extraction logic is well-designed and previously identified issues have been addressed.The capacity lookup building correctly implements a two-source strategy:
- Primary source:
existing_capacitytable (direct definition)- Fallback:
cost_investtable (derived from denominator)The
_extract_capacity_unitmethod (lines 141-164) properly addresses the previous critical issue:
- Uses regex patterns with word boundaries to match standard power units (GW, MW, kW, TW)
- Avoids false matches (e.g., "GWh" won't match "\bGW\b")
- Handles complex denominators like "Mdollar / (PJ^2 / GW)" by searching for known power units
This is a much more robust solution than the previous simple split approach.
166-197: LGTM! Cost unit derivation is robust and previously identified issues have been addressed.The
_derive_common_cost_unitmethod properly extracts currency units from cost tables:
- Uses regex split
r'\s*/\s*\('to handle complex formats (line 186)- Correctly extracts numerator from formats like "kWh/day / (GW)"
- Falls back to simple split for formats without parentheses (line 192)
- Iterates through multiple cost tables to find first valid entry
The previous concern about nested slashes in the numerator has been addressed with the regex-based split.
The SQL injection warning (S608 on line 179) is a false positive—the
tablevariable comes from a hardcoded list on line 176.
199-323: LGTM! Unit getter methods properly preserve unit symbols and previously identified issues have been addressed.All getter methods now use the Pint compact format specifier
f"{unit:~}"to preserve unit symbols:
get_flow_out_units(line 237)get_flow_in_units(line 252)get_emission_units(line 293)get_storage_units(line 321)This addresses the previous critical issue where
str(unit)would return canonical names (e.g., "gigawatt") instead of input symbols (e.g., "GW").The storage commodity lookup (lines 199-222) properly joins multiple tables to identify storage technologies.
All methods return
Nonegracefully when data is unavailable, ensuring backward compatibility.temoa/tutorial_assets/utopia.sql (5)
156-200: LGTM! Commodity and capacity_to_activity units are properly defined.The unit additions are well-structured:
capacity_to_activity: consistently uses "PJ / (GW * year)" (lines 166-176)commodity: physical commodities use "PJ" for energy (lines 187-195, 198-200)- Emission commodities corrected: co2 and nox now use "Mt" instead of "PJ" (lines 196-197) ✓
This addresses the previous major issue where emission units would have propagated incorrectly to output_emission.
396-467: LGTM! Variable cost and demand units are correctly defined.The cost_variable table entries properly use activity-based units:
- All entries consistently use "Mdollar / (PJ)" format (lines 396-446)
- This correctly represents currency per energy (variable O&M per unit output)
The demand table entries include proper energy units:
- All demand entries use "PJ" units (lines 459-467)
531-648: LGTM! Efficiency and emission activity units are correctly defined.Both tables use proper ratio formats:
efficiencytable: consistently uses "PJ / (PJ)" for efficiency ratios (lines 531-594)emission_activitytable: consistently uses "Mt / (PJ)" for emission factors (lines 635-648)These unit formats align with the expected RATIO_ELEMENT format and will be properly validated by the unit checking system.
703-842: LGTM! Capacity and lifetime units are correctly defined.All capacity and lifetime-related tables use appropriate units:
existing_capacity: consistently uses "GW" for power capacity (lines 703-717)lifetime_process: uses "year" for lifetime values (lines 730-734)lifetime_tech: consistently uses "year" for technology lifetimes (lines 757-775)limit_capacity: consistently uses "GW" for capacity limits (lines 831-841)These units are dimensionally correct and will properly propagate to output tables.
1156-1359: LGTM! Output table schema changes properly add units columns.All output tables have been extended with units columns:
output_built_capacity(line 1167)output_cost(line 1186)output_curtailment(line 1211)output_emission(line 1236)output_flow_in(line 1260)output_flow_out(line 1284)output_net_capacity(line 1315)output_retired_capacity(line 1338)output_storage_level(line 1358)These schema changes align with the UnitPropagator design, which will populate these columns during result writing.
Co-authored-by: Jeff-ws <jeff@westernspark.us>
ParticularlyPythonicBS
force-pushed
the
feat/unit_checking
branch
from
ee3d3fb to
7ea06c2
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
requirements.txt (1)
15-18: Verify flexcache and flexparser provenance.A previous review (with web search confirmation) found that Pint 0.25.2 has no required install dependencies and that
flexcache==0.3andflexparser==0.4marked as "via pint" are incorrect. If these packages are actual transitive dependencies, the provenance comments need correction. If they were added in error, they should be removed.Run the following script to verify the actual dependency tree:
#!/bin/bash # Verify pint's actual dependencies echo "=== Checking Pint 0.25.2 dependencies from PyPI ===" curl -s https://pypi.org/pypi/pint/0.25.2/json | jq -r '.info.requires_dist // [] | .[]' echo -e "\n=== Checking if flexcache or flexparser appear ===" curl -s https://pypi.org/pypi/pint/0.25.2/json | jq -r '.info.requires_dist // [] | .[]' | grep -iE "(flexcache|flexparser)" || echo "Not found in pint dependencies" echo -e "\n=== Checking flexcache dependencies ===" curl -s https://pypi.org/pypi/flexcache/0.3/json | jq -r '.info.requires_dist // [] | .[]' echo -e "\n=== Checking flexparser dependencies ===" curl -s https://pypi.org/pypi/flexparser/0.4/json | jq -r '.info.requires_dist // [] | .[]'requirements-dev.txt (1)
30-33: Verify flexcache and flexparser provenance (same issue as requirements.txt).This file has the same provenance issue as
requirements.txtlines 15-18. The packagesflexcache==0.3andflexparser==0.4are marked "via pint", but a previous review confirmed Pint 0.25.2 has no required dependencies. Please verify the actual dependency source.Use the verification script provided for
requirements.txtlines 15-18 to check these dependencies.temoa/tutorial_assets/utopia.sql (2)
255-336: Cost_fixed units still contain dimensionally incorrect PJ² term.All 82 cost_fixed rows use
'Mdollar / (PJ^2 / GW / year)', which is dimensionally incorrect. Fixed O&M costs should be currency per capacity per time (e.g.,Mdollar / GW / yearorMdollar / (GW * year)), not involving energy squared.This issue was previously flagged but remains unaddressed. The incorrect units will propagate to output_cost if unit checking is enabled.
349-381: Cost_invest units still contain dimensionally incorrect PJ² term.All 33 cost_invest rows use
'Mdollar / (PJ^2 / GW)', which is dimensionally incorrect. Investment/capital costs should be currency per capacity (e.g.,Mdollar / GW), not involving energy squared.This issue was previously flagged but remains unaddressed. The unit_propagator's
_extract_capacity_unitmethod will extract 'GW' correctly, but the full cost unit string remains incorrect for validation purposes.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (28)
data_files/my_configs/config_sample.toml(1 hunks)data_files/temoa_schema_v4.sql(15 hunks)docs/source/index.rst(1 hunks)docs/source/unit_checking.rst(1 hunks)pyproject.toml(1 hunks)requirements-dev.txt(3 hunks)requirements.txt(3 hunks)temoa/_internal/table_writer.py(15 hunks)temoa/_internal/temoa_sequencer.py(1 hunks)temoa/cli.py(2 hunks)temoa/core/config.py(3 hunks)temoa/db_schema/temoa_schema_v4.sql(15 hunks)temoa/model_checking/unit_checking/__init__.py(1 hunks)temoa/model_checking/unit_checking/common.py(1 hunks)temoa/model_checking/unit_checking/entry_checker.py(1 hunks)temoa/model_checking/unit_checking/relations_checker.py(1 hunks)temoa/model_checking/unit_checking/screener.py(1 hunks)temoa/model_checking/unit_checking/table_checker.py(1 hunks)temoa/model_checking/unit_checking/temoa_units.txt(1 hunks)temoa/model_checking/unit_checking/unit_propagator.py(1 hunks)temoa/tutorial_assets/config_sample.toml(1 hunks)temoa/tutorial_assets/utopia.sql(21 hunks)tests/conftest.py(2 hunks)tests/test_unit_checking.py(1 hunks)tests/test_unit_checking_integration.py(1 hunks)tests/test_unit_propagation.py(1 hunks)tests/test_unit_scavenging.py(1 hunks)tests/testing_outputs/.gitignore(1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2025-10-27T15:53:41.829Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 177
File: temoa/model_checking/commodity_network.py:26-33
Timestamp: 2025-10-27T15:53:41.829Z
Learning: The Temoa project requires Python 3.12 or above as the minimum supported version, so PEP 695 `type` syntax for type aliases is appropriate and preferred over `TypeAlias`.
Applied to files:
requirements.txtrequirements-dev.txt
📚 Learning: 2025-10-30T14:37:44.260Z
Learnt from: idelder
Repo: TemoaProject/temoa PR: 180
File: temoa/components/limits.py:632-647
Timestamp: 2025-10-30T14:37:44.260Z
Learning: In the `limit_tech_input_split_average_constraint` function in `temoa/components/limits.py`, there was a bug where the `total_inp` summation incorrectly used the fixed variable `i` instead of the loop variable `S_i` when indexing `processOutputsByInput` and calling `get_variable_efficiency`. This pattern should be watched for in similar aggregation constraints: when summing over a loop variable like `S_i`, all references within that loop should use `S_i`, not a fixed outer variable.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during code reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
📚 Learning: 2025-11-03T13:34:35.907Z
Learnt from: ParticularlyPythonicBS
Repo: TemoaProject/temoa PR: 184
File: temoa/data_processing/database_util.py:109-118
Timestamp: 2025-11-03T13:34:35.907Z
Learning: In the Temoa project, SQL injection vulnerabilities in the database utility classes (such as temoa/data_processing/database_util.py) are considered low concern due to the data access patterns and should not be flagged during reviews. Technical debt for SQL injection hardening should be tracked separately as issues rather than blocking PRs.
Applied to files:
temoa/model_checking/unit_checking/relations_checker.py
🧬 Code graph analysis (11)
tests/test_unit_checking_integration.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
temoa/_internal/table_writer.py (2)
temoa/model_checking/unit_checking/unit_propagator.py (8)
UnitPropagator(29-334)has_unit_data(325-334)get_storage_units(304-322)get_emission_units(280-293)get_capacity_units(266-278)get_flow_in_units(239-252)get_flow_out_units(224-237)get_cost_units(295-302)temoa/types/model_types.py (1)
FlowType(275-282)
temoa/model_checking/unit_checking/entry_checker.py (1)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(140-142)
tests/test_unit_checking.py (2)
temoa/model_checking/unit_checking/common.py (1)
UnitsFormat(140-142)temoa/model_checking/unit_checking/entry_checker.py (2)
validate_units_expression(39-49)validate_units_format(24-36)
tests/test_unit_scavenging.py (1)
temoa/model_checking/unit_checking/unit_propagator.py (1)
_extract_capacity_unit(142-164)
temoa/model_checking/unit_checking/table_checker.py (2)
temoa/model_checking/unit_checking/common.py (1)
consolidate_lines(164-172)temoa/model_checking/unit_checking/entry_checker.py (3)
gather_from_table(52-76)validate_units_expression(39-49)validate_units_format(24-36)
temoa/model_checking/unit_checking/screener.py (3)
temoa/model_checking/unit_checking/common.py (1)
RelationType(133-136)temoa/model_checking/unit_checking/relations_checker.py (4)
check_efficiency_table(71-156)check_inter_table_relations(169-331)make_c2a_lut(49-62)make_commodity_lut(33-46)temoa/model_checking/unit_checking/table_checker.py (1)
check_table(30-116)
temoa/_internal/temoa_sequencer.py (3)
temoa/cli.py (1)
check_units(293-393)temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)temoa/_internal/run_actions.py (1)
check_python_version(32-42)
temoa/cli.py (1)
temoa/model_checking/unit_checking/screener.py (1)
screen(244-299)
temoa/core/config.py (1)
temoa/cli.py (1)
check_units(293-393)
temoa/model_checking/unit_checking/unit_propagator.py (1)
temoa/model_checking/unit_checking/relations_checker.py (4)
IOUnits(66-68)check_efficiency_table(71-156)make_c2a_lut(49-62)make_commodity_lut(33-46)
🪛 Ruff (0.14.8)
temoa/model_checking/unit_checking/entry_checker.py
47-47: Consider moving this statement to an else block
(TRY300)
60-60: Possible SQL injection vector through string-based query construction
(S608)
63-67: Use logging.exception instead of logging.error
Replace with exception
(TRY400)
tests/test_unit_checking.py
41-41: Boolean-typed positional argument in function definition
(FBT001)
72-72: Boolean-typed positional argument in function definition
(FBT001)
tests/test_unit_scavenging.py
21-21: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
26-26: Unused noqa directive (non-enabled: N802)
Remove unused noqa directive
(RUF100)
temoa/model_checking/unit_checking/table_checker.py
115-115: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/screener.py
256-256: Avoid specifying long messages outside the exception class
(TRY003)
tests/test_unit_propagation.py
193-193: Possible SQL injection vector through string-based query construction
(S608)
201-201: Possible SQL injection vector through string-based query construction
(S608)
temoa/cli.py
314-314: Boolean default positional argument in function definition
(FBT002)
590-593: Avoid specifying long messages outside the exception class
(TRY003)
596-599: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/relations_checker.py
164-164: Consider moving this statement to an else block
(TRY300)
186-186: Avoid specifying long messages outside the exception class
(TRY003)
207-209: Possible SQL injection vector through string-based query construction
(S608)
213-213: Possible SQL injection vector through string-based query construction
(S608)
221-221: Possible SQL injection vector through string-based query construction
(S608)
223-223: Possible SQL injection vector through string-based query construction
(S608)
225-225: Avoid specifying long messages outside the exception class
(TRY003)
264-264: Avoid specifying long messages outside the exception class
(TRY003)
371-374: Avoid specifying long messages outside the exception class
(TRY003)
376-377: Possible SQL injection vector through string-based query construction
(S608)
temoa/core/config.py
62-62: Boolean-typed positional argument in function definition
(FBT001)
62-62: Boolean default positional argument in function definition
(FBT002)
temoa/model_checking/unit_checking/__init__.py
19-22: Avoid specifying long messages outside the exception class
(TRY003)
temoa/model_checking/unit_checking/unit_propagator.py
179-179: Possible SQL injection vector through string-based query construction
(S608)
tests/conftest.py
111-113: Avoid specifying long messages outside the exception class
(TRY003)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: setup and test (macos-latest, 3.13)
- GitHub Check: setup and test (windows-latest, 3.12)
- GitHub Check: setup and test (windows-latest, 3.13)
- GitHub Check: setup and test (macos-latest, 3.12)
🔇 Additional comments (29)
tests/testing_outputs/.gitignore (1)
1-4: The gitignore pattern change is valid; the subdirectory concern stated is inaccurate.When a directory is ignored, all of its files and subdirectories are also ignored. The pattern
/*matches root-level entries (both files and directories), so subdirectories and their contents would be recursively ignored. The change from specific file-type patterns to a broad root-level ignore is a semantic shift, but it will not fail to ignore nested content within matched root directories.Likely an incorrect or invalid review comment.
pyproject.toml (1)
33-33: LGTM! Pint dependency added appropriately.The addition of
pint>=0.25.2to support the new unit-checking subsystem is correct. The minimum version constraint is appropriate for a library dependency.temoa/tutorial_assets/config_sample.toml (1)
49-52: LGTM! Unit checking enabled appropriately in tutorial config.Enabling
check_units = trueby default in the tutorial configuration is appropriate since the tutorial database is described as fully compliant and serves as a reference implementation. The descriptive comments clearly explain the feature's purpose.requirements.txt (1)
68-71: Verify platformdirs provenance.The comment indicates
platformdirs==4.5.1is "via pint", but the previous review indicated Pint 0.25.2 has no required dependencies. Please verify whether platformdirs is actually a transitive dependency of Pint or if it comes from another source.This can be confirmed by the same verification script above (lines 15-18).
docs/source/unit_checking.rst (1)
1-427: Excellent comprehensive documentation.This documentation provides thorough coverage of the unit-checking subsystem, including:
- Clear activation instructions (config and CLI)
- Detailed format requirements with examples
- Common pitfalls with solutions
- Troubleshooting guidance
- Dimension reference
The structure and examples make this a valuable resource for users adopting the new feature.
docs/source/index.rst (1)
7-7: LGTM! Documentation navigation updated correctly.The addition of
unit_checkingto the toctree properly integrates the new unit-checking documentation into the documentation structure.data_files/my_configs/config_sample.toml (1)
50-53: LGTM! Appropriate default for user config template.Setting
check_units = falseas the default in this user-facing config template is correct. This differs intentionally from the tutorial config (true) because:
- The tutorial database is fully compliant with units populated
- User databases may not yet have units populated
- The feature is opt-in and should not break existing workflows
The comments appropriately guide users to enable it "after units are populated in the database."
temoa/model_checking/unit_checking/temoa_units.txt (1)
1-15: LGTM! Custom unit definitions are well-structured.The custom unit definitions properly extend Pint's registry with Temoa-specific units:
- Entity-based units (
passenger,seat,vehicle) with appropriate dimensions- Currency units (
dollar,euro) using Pint's[currency]dimension- Source commodity unit (
ethos) as an empty base classThe format follows Pint conventions, and the comments provide helpful context about usage and limitations (e.g., currency conversion considerations).
requirements-dev.txt (1)
105-108: Verify platformdirs provenance (same issue as requirements.txt).As with
requirements.txtline 68-71, verify whetherplatformdirs==4.5.1is actually "via pint" or comes from another source.Use the verification script provided for
requirements.txtlines 15-18 to check this dependency.temoa/_internal/table_writer.py (2)
130-150: LGTM! Lazy initialization with proper error handling.The
unit_propagatorproperty correctly implements lazy initialization with graceful fallback. The pattern ensures unit propagation is optional and doesn't break existing functionality when unit data is unavailable.
545-568: LGTM! Flow unit determination logic is correct.The
_get_flow_unitshelper method correctly determines flow units based on flow direction: input flows use input commodity units, while output flows, curtailment, and flex use output commodity units. The implementation follows the expected semantics.temoa/model_checking/unit_checking/entry_checker.py (3)
24-36: LGTM! Format validation is correctly implemented.The function properly validates unit format against regex patterns and returns both validity and parsed groups. The empty string guard is appropriate.
39-49: LGTM! Registry validation is correctly implemented.The function properly validates unit expressions against the Pint registry and handles undefined units gracefully by catching
UndefinedUnitError.
52-76: LGTM! Defensive handling of missing units column.The
gather_from_tablefunction now properly handles the case where a table doesn't have aunitscolumn by catchingOperationalErrorand returning an empty dict. This allows callers to treat schema mismatches as reportable validation failures rather than crashes.The SQL injection concern (S608) is a false positive here since
tablecomes from the trustedtables_with_unitsconstant in the calling code.tests/conftest.py (3)
37-72: LGTM! Clean refactoring to data-driven database setup.The refactored
refresh_databases()approach centralizes the source of truth for tutorial assets and makes the database list more maintainable. The use of tuples to map source directories to output files is clear and the connection management is correct.
74-96: Well-designed helper for test database generation.The
create_unit_test_db_from_sqlfunction properly uses context managers, explicit encoding, and applies modifications cleanly. This pattern enables flexible test database generation from a single SQL source.
98-189: Test database generation strategy is well-structured.The
create_unit_test_dbsfunction systematically generates test databases for various unit-checking scenarios, andpytest_configureproperly handles setup failures with graceful degradation. The error messages provide clear guidance for debugging test setup issues.temoa/cli.py (2)
292-394: check-units command implementation is solid.The command properly validates inputs, creates the output directory, runs the screener, handles both success and failure cases, and exits with appropriate codes. The report summary streaming (lines 366-377) avoids loading the entire file into memory, which is good practice.
550-611: Good shift to SQL-based database generation.Generating the tutorial database from SQL source (instead of copying a pre-built file) ensures users always get the latest schema with unit-compliant data. The connection management is correct and error messages clearly distinguish between config and SQL source issues.
temoa/model_checking/unit_checking/table_checker.py (1)
30-116: Comprehensive table-checking implementation.The
check_tablefunction implements a thorough validation pipeline: blanks → characters → format → registry compliance → domain-specific checks. Error messages consistently include row numbers for easy debugging, and the capacity table time-dimension validation (lines 92-103) correctly identifies energy units in power-unit contexts.temoa/model_checking/unit_checking/screener.py (1)
30-310: Well-architected screening workflow with clear separation of concerns.The screening pipeline is cleanly decomposed into five helper functions that each handle a specific validation stage. The main
screenfunction orchestrates these checks, properly handles early exit on DB version failure, and accumulates results across multiple databases. The timezone-aware timestamp (line 306) ensures consistent report naming across environments.temoa/db_schema/temoa_schema_v4.sql (2)
100-118: Schema expansion supports unit-checking workflow.The addition of nullable
units TEXTcolumns to input tables (capacity_to_activity, commodity, efficiency) provides the necessary metadata storage for the unit-checking subsystem without introducing breaking changes.
703-840: Output table schema properly supports unit propagation.The
units TEXTcolumns added to output tables (output_curtailment, output_net_capacity, output_built_capacity, output_retired_capacity, output_flow_in, output_flow_out, output_storage_level) enable the UnitPropagator to populate derived unit information from input tables, completing the unit-tracking workflow.tests/test_unit_propagation.py (3)
26-30: Proper fixture pattern keeps connection alive for tests.The
propagatorfixture correctly usesyieldwithin a context manager to ensure the SQLite connection remains open while tests execute, addressing the concern from past reviews about returning a propagator backed by a closed connection.
100-152: Comprehensive integration fixture with defensive checks.The
solved_db_with_unitsfixture properly validates both database and solver availability before running a full model solve, ensuring graceful test skipping in incomplete environments. The use ofas_posix()(line 131) prevents Windows path escaping issues in TOML configuration.
155-223: Thorough validation of end-to-end unit propagation.The integration tests verify that units are correctly propagated from input tables to output tables across flow, capacity, and cost domains. The tests check both presence and correctness of unit values, providing comprehensive coverage of the unit propagation workflow.
temoa/model_checking/unit_checking/common.py (1)
1-172: Well-designed shared utilities for unit-checking subsystem.This module provides a clean, centralized source of truth for unit-checking metadata including:
- Table categorizations (activity-based, capacity-based, cost-based, etc.)
- Format definitions (SINGLE_ELEMENT, RATIO_ELEMENT)
- Type definitions (CostTableData, RelationType, UnitsFormat)
- Helper functions (consolidate_lines)
The use of keyword arguments in
cost_based_tables(lines 110-129) makes the boolean flags self-documenting, and the type hints properly useSequencefor flexibility.data_files/temoa_schema_v4.sql (1)
1-1127: Schema changes consistent with db_schema version.The
units TEXTcolumn additions mirror those intemoa/db_schema/temoa_schema_v4.sql, ensuring consistency between the packaged schema and data file schema. The nullable TEXT columns provide metadata storage for the unit-checking and propagation workflows without introducing breaking changes.temoa/model_checking/unit_checking/unit_propagator.py (1)
176-179: Static analysis flags potential SQL injection, but risk is minimal.Line 179 interpolates table names from the
cost_tableslist into an SQL query. While static analysis flags this as S608, the table names are hardcoded literals within this function (['cost_invest', 'cost_fixed', 'cost_variable', 'cost_emission']), making external injection impossible in the current implementation.Based on learnings, SQL injection in Temoa's database utilities is considered low concern. The code is also defensive with
sqlite3.OperationalErrorexception handling.
reworks PR #124 for use with temoa v4 and db v4.
Implements a comprehensive unit checking system for Temoa databases. The system validates unit consistency across input tables and automatically propagates units to output tables during model result writing.
Unit Validation (Pre-solve)
5-step validation pipeline: DB version check → unit entry validation → efficiency table alignment → related tables → cost tables
Unit Propagation (Post-solve)
Units from input tables are automatically written to output tables:
output_flow_outcommodity.unitsfor output commodityoutput_flow_incommodity.unitsfor input commodityoutput_curtailmentcommodity.unitsfor output commodityoutput_built_capacityexisting_capacity.unitsoutput_net_capacityexisting_capacity.unitsoutput_retired_capacityexisting_capacity.unitsoutput_emissioncommodity.unitsfor emission commodityoutput_costoutput_storage_levelUsage:
Tutorial database is 100% unit checking compliant and can serve as a reference implementation
The migration script for v4 will automatically handle these changes
This should not have any breaking changes since:
Known Limitations and quirks
Units are not mathematically enforced: Unit checking is validation only; the Pyomo model does not use units in calculations
Technology output uniformity required: All instances of a technology must produce output in the same units, even with different output commodities
Ratio format required for efficiency/cost tables: Units must use
Numerator / (Denominator)format with parentheses around denominatorGrowth/degrowth tables use
seed_units: These tables have a different column name and are not currently validatedGroup-based constraints not fully validated: Tables using
tech_or_groupwith group names don't have complete unit validation for the group membersLifecycle tables (construction_input, emission_embodied, emission_end_of_life): Validated for format but not for inter-table consistency
Adding an easy way for users to add their own units remains future work, with ideal case of packaging the pint augmentations within the databases themselves
Summary by CodeRabbit
New Features
Database
Documentation
Tests
Chores
✏️ Tip: You can customize this high-level summary in your review settings.