Refactor disagg calls into cornerstone_disagg_pipeline.py#453
Merged
jvendries merged 8 commits intoJun 8, 2026
Conversation
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
jvendries
force-pushed
the
move_disagg_code_from_cornerstone_derived_functions
branch
2 times, most recently
from
e412ad3 to
2ad75e6
Compare
Contributor
Author
|
@WesIngwersen there is some duplication in this PR still, for example, between |
jvendries
force-pushed
the
move_disagg_code_from_cornerstone_derived_functions
branch
from
2ad75e6 to
202b7d1
Compare
This was referenced Jun 5, 2026
WesIngwersen
reviewed
Jun 7, 2026
| for code in present: | ||
| x.loc[code] = parent_go * float(shares.loc[code]) | ||
| return x | ||
| return distribute_waste_parent_x_using_v_row_shares(x_cs) |
Member
There was a problem hiding this comment.
Noting this still calling a specific waste disagg function
WesIngwersen
reviewed
Jun 7, 2026
WesIngwersen
reviewed
Jun 7, 2026
| return Udom_cs, Uimp_cs | ||
|
|
||
|
|
||
| def derive_cornerstone_VA_after_waste() -> pd.DataFrame: |
WesIngwersen
reviewed
Jun 7, 2026
| Ytot_orig = load_2017_Ytot_usa() | ||
| Ytot = commodity_corresp() @ Ytot_orig | ||
| Ytot.index.name = 'sector' | ||
| weights = get_waste_disagg_weights() |
WesIngwersen
reviewed
Jun 7, 2026
| return Ytot | ||
|
|
||
|
|
||
| def distribute_waste_parent_x_using_v_row_shares( |
WesIngwersen
reviewed
Jun 7, 2026
WesIngwersen
approved these changes
Jun 7, 2026
WesIngwersen
left a comment
WesIngwersen
left a comment
Member
There was a problem hiding this comment.
Useful and needed refactoring to move disaggregation functionality largely into a separate module. Can use further work to make it more generic but this is good progress and I tested code to generate various models and it does not break it.
added 2 commits
June 8, 2026 12:19
…rts from derived_cornerstone.py
bl-young
reviewed
Jun 8, 2026
Member
|
btw I'm running the integration tests on this branch to make sure they pass |
bl-young
approved these changes
Jun 8, 2026
bl-young
added a commit
that referenced
this pull request
Jun 10, 2026
bl-young
added a commit
that referenced
this pull request
Jun 10, 2026
* refactor(publish): extract shared model getters from excel writer Move cached getters, location suffix helpers, and cache clearing into model_objects and cache_reset so XLSX and other publishers share one path. * feat(publish): emit cornerstone CO2e supply-chain factors CSV Add purchaser-price CO2e tables from N with year rebase via cornerstone industry PI, placeholder Phi/margins, and emission_factors CLI. * move files inside `emission_factors` * fix runtime typing error * fix application of price ratio * use commodity PI instead of industry PI * resolve remaining conflicts with #453
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cc:
Closes:
What changed? Why?
Extracts the cornerstone sector-disaggregation orchestration logic from
derived_cornerstone.pyinto a new dedicated module,cornerstone_disagg_pipeline.py.The new module owns:
cornerstone_sector_disagg_active— single gate replacing the previouselectricity_reallocation_enabledchecks scattered across public routersget_waste_disagg_weights— moved fromderived_cornerstoneelectricity_reallocation_enabled— moved fromderived_cornerstonederive_disagg_io_bundle— replaces_derive_cornerstone_io_after_electricity_reallocationderive_disagg_Ytot_with_trade— replaces_derive_cornerstone_Ytot_with_tradedistribute_waste_parent_x_using_v_row_shares— extracted from_distribute_waste_parent_x_using_v_row_sharesderived_cornerstone.pypublic routers (derive_cornerstone_V,derive_cornerstone_U_with_negatives,derive_cornerstone_VA,derive_cornerstone_Ytot_full_cs_matrix,derive_cornerstone_Aq) now gate oncornerstone_sector_disagg_active()instead ofelectricity_reallocation_enabled(). Baseline (no-disagg) IO helpers are renamed_derive_cornerstone_V_baseline,_derive_cornerstone_U_baseline, and_derive_cornerstone_VA_baselineto make the routing intent explicit. The_normalize_E_for_wastehelper is removed as it was no longer used. Import aliases were removed fromderived_cornerstone.py.inflate_cornerstone_V_with_industry_piis extracted intoinflation_helpers_cornerstone.pyto replace inline inflation logic inderive_cornerstone_V.Monkeypatches in
test_waste_disagg_pipeline_integration.pyare updated to targetdisagg_weights_module.load_disagg_weightsdirectly rather than patching throughderived_cornerstone, reflecting the moved import site.Testing
Existing test suite passes. Cache-clearing lists in
test_electricity_reallocation.pyandtest_waste_disagg_pipeline_integration.pyare updated to reference the relocated cached functions (derive_disagg_io_bundle,derive_disagg_Ytot_with_trade,cornerstone_sector_disagg_active) incornerstone_disagg_pipeline.