Skip to content

Water Data Unit Conversions 🔢 (addendum to #336) #343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 22 commits into from

Conversation

Wegatriespython
Copy link

@Wegatriespython Wegatriespython commented May 8, 2025

Automatic Unit Conversions for Water Data

Builds on the DSL refactor from PR #336. Adds a unit conversion layer with automatic constants and value scaling. With dimensionality checks.

Conversion logic is modular, can be updated with pint or iam_units. Automatic conversion in the sense when the units for a value are changed, the value and any constants used its calculation are automatically scaled.

How to review

  • Check GDX to see if unit conversion is consistent
  • Questions on use and functionality
  • Ensure that changes/additions are self-documenting

PR checklist

  • Continuous integration checks all ✅
  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
  • Update doc/whatsnew.

Wegatriespython and others added 21 commits May 5, 2025 19:17
Introduces a string-based DSL engine that can
evaluate Python expressions within strings,
plus a full suite of water-data operations. Incl
unit tests for engine and operations.

BREAKING CHANGE: requires Python 3.10+;
minimum-version markers added to functions.
Migrated water demans to use the new DSL engine.
Includes the creation of demand_rules.py. Added
a new test for parity with the legacy code
and updated demands test.
Migrated water supply to use the new DSL engine.
Includes the creation of supply_rules. Added
a new test for parity with the legacy code
and updated supply test.
Similar to previous for infrastructure now.
Helpful utility to wrap citations,
used to replace citations in comments with a
consistent and uniform format.
Similar to previous, uses citation wrapper.
Additional to DSL migration, this breaks file into
two files: water_for_ppl_rules.py and
water_for_ppl_support.py. Cyclomatic complexity
is reduced.
@Wegatriespython Wegatriespython requested a review from adrivinca May 8, 2025 15:13
@Wegatriespython Wegatriespython self-assigned this May 8, 2025
@Wegatriespython Wegatriespython requested a review from awais307 as a code owner May 8, 2025 15:13
@Wegatriespython Wegatriespython added enh New features or functionality water MESSAGEix-Nexus (water) variant safe to test PRs from forks that do not pose security risks labels May 8, 2025
@Wegatriespython Wegatriespython marked this pull request as draft May 8, 2025 15:16
Copy link

codecov bot commented May 8, 2025

Codecov Report

Attention: Patch coverage is 82.31293% with 312 lines in your changes missing coverage. Please review.

Project coverage is 77.7%. Comparing base (ae0cf78) to head (74f6679).
Report is 561 commits behind head on main.

Files with missing lines Patch % Lines
...essage_ix_models/model/water/data/water_for_ppl.py 20.6% 185 Missing ⚠️
message_ix_models/model/water/data/demands.py 83.4% 39 Missing ⚠️
message_ix_models/model/water/data/water_supply.py 86.2% 27 Missing ⚠️
message_ix_models/model/water/dsl_engine.py 87.5% 18 Missing ⚠️
message_ix_models/model/water/rules.py 88.5% 16 Missing ⚠️
...age_ix_models/tests/model/water/test_dsl_engine.py 86.9% 12 Missing ⚠️
message_ix_models/util/citation_wrapper.py 78.5% 6 Missing ⚠️
...ix_models/tests/model/water/test_dsl_engine_pt2.py 97.5% 5 Missing ⚠️
message_ix_models/model/water/utils.py 80.0% 1 Missing ⚠️
...dels/tests/model/water/data/test_infrastructure.py 75.0% 1 Missing ⚠️
... and 2 more
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #343     +/-   ##
=======================================
- Coverage   78.5%   77.7%   -0.9%     
=======================================
  Files        216     227     +11     
  Lines      17409   18655   +1246     
=======================================
+ Hits       13679   14504    +825     
- Misses      3730    4151    +421     
Files with missing lines Coverage Δ
message_ix_models/model/water/data/demand_rules.py 100.0% <100.0%> (ø)
...ssage_ix_models/model/water/data/infrastructure.py 100.0% <100.0%> (ø)
...ix_models/model/water/data/infrastructure_rules.py 100.0% <100.0%> (ø)
message_ix_models/model/water/data/irrigation.py 100.0% <100.0%> (ø)
...age_ix_models/model/water/data/irrigation_rules.py 100.0% <100.0%> (ø)
..._ix_models/model/water/data/water_for_ppl_rules.py 100.0% <100.0%> (ø)
...e_ix_models/model/water/data/water_supply_rules.py 100.0% <100.0%> (ø)
...e_ix_models/tests/model/water/data/test_demands.py 100.0% <100.0%> (ø)
...models/tests/model/water/data/test_water_supply.py 100.0% <100.0%> (ø)
message_ix_models/tests/model/water/rules_test.py 100.0% <100.0%> (ø)
... and 12 more

... and 16 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Wegatriespython Wegatriespython added the ai PR includes some AI generated Code label May 12, 2025
@Wegatriespython Wegatriespython removed the safe to test PRs from forks that do not pose security risks label May 20, 2025
@Wegatriespython
Copy link
Author

superseded by nexusSSP6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ai PR includes some AI generated Code enh New features or functionality water MESSAGEix-Nexus (water) variant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant