Skip to content

feat(publish): integrate Phi purchaser-price adjustment#464

Merged
bl-young merged 2 commits into
mainfrom
by_integrate_phi
Jun 16, 2026
Merged

feat(publish): integrate Phi purchaser-price adjustment#464
bl-young merged 2 commits into
mainfrom
by_integrate_phi

Conversation

@bl-young

Copy link
Copy Markdown
Member

cc:
Closes: #435

What changed? Why?

Margins-based producer-to-purchaser ratios (Phi) from #440 are wired into supply-chain factor export, Excel publish, and USEEIO baseline diagnostics. Phi converts producer-price N to purchaser price so comparisons against USEEIO phoebe workbooks and purchaser-price reporting use consistent valuation.

Transform helpers (derive_PRO_to_PUR_ratio.py)

  • margins_phi_active(cfg) — true when useeio_margins or cornerstone_industry_avg_margins is set (not ceda_margins alone).
  • phi_for_sectors(sector_index) — returns derive_phi_cornerstone_usa() aligned to sectors, or identity 1.0 when margins methodology is inactive.
  • apply_phi_to_ef_vector(ef) — elementwise ef * phi.

Diagnostics and publish both import these helpers from transform; publish does not own Phi derivation logic.

Supply-chain emission factors export

  • CLI flag --purchaser_price / --no-purchaser_price (default on).
  • adjust_publish_matrix() in placeholders.py rebases to dollar_year via commodity CPI, then optionally multiplies columns by Phi.
  • Table unit label reflects purchaser vs producer price.
  • Stub placeholder_phi removed; real Phi replaces it.

Excel publish

  • Cached get_Phi() in model_objects.py returns a single-column DataFrame keyed by model_base_year, or None when margins methodology is inactive (sheet omitted). Multi-year Phi panel deferred — see TODO below.
  • Phi sheet added to workbook writer SheetSpec list.
  • Margins parquet loader caches included in clear_all_publish_caches().

USEEIO baseline diagnostics

Purchaser adjustment applies only when both conditions hold:

  1. diagnostics_baseline_source == 'gcs_useeio_xlsx' (pinned USEEIO Excel baseline).
  2. An active margins flag (useeio_margins or cornerstone_industry_avg_margins).

When eligible, pull_efs_for_diagnostics computes N_new_purchaser and N_old_purchaser via apply_phi_to_ef_vector on the inflated/new producer N vectors (workbook N is producer-price). N_perc_diff compares purchaser vs purchaser. N_and_diffs merge order: construct → inflated merge → purchaser merge. combine_ef_diagnostics reads new N with fallback N_new_purchaserN_new_inflatedN_new, and pinned baseline old N with fallback N_old_purchaserN_old_inflated. config_summary records purchaser_price_adjustment_applied.

D diagnostics are unchanged (Phi applies to N only). Both sides use bedrock derive_phi_cornerstone_usa() Phi at model_base_year.

Follow-ups (not in this branch)

  • Margin supply-chain factor column (placeholder_margin_ef remains zeros).
  • Rho / Tau Excel sheets.
  • Zenodo NAICS purchaser-price parity script (Build supply-chain style CSV #449 broader scope).

TODO (team confirmation pending)

  • Phi time series: Export and Excel emit Phi at model_base_year only; CPI handles --dollar_year. supply-chain-factors / useeior apply Phi[currency_year] (year-specific margins inflation). Follow-up: derive_phi_cornerstone_usa_panel, get_Phi() sector×year panel, phi_for_sectors(..., year=dollar_year) in SEF export — see plan phi-panel-timeseries.

Validation scope (years and comparisons)

Phi year: derive_phi_cornerstone_usa(), get_Phi(), and USEEIO diagnostics purchaser columns use Phi at model_base_year only (2017 for useeio_phoebe_23). get_Phi() emits a single column; the Excel Phi sheet has no other year columns yet.

Pinned workbook comparison (CI, eeio_integration): test_published_phi_matches_useeio_workbook compares bedrock Phi to the phoebe workbook Phi sheet column 2017 only (1% rtol). Other workbook year columns are not tested.

Supply-chain factor CSV — no external published parity in CI:

Check What it does
test_sef_phi_wiring (eeio_integration) Internal wiring: SEF without-margins at --dollar_year 2024 equals N_producer / CPI(2017→2024) × Phi(2017). Not compared to the USEEIO workbook, supply-chain-factors CSV, or Zenodo NAICS publish.
supply-chain-factors / Zenodo Not in scope. Taxonomy and margin-column differences; deferred (#449).

Diagnostics: N_new_purchaser vs N_old_purchaser on the phoebe pin. With default useeio_phoebe_23 (model_base_year 2017), both sides sit at 2017 USD and N_perc_diff is purchaser vs purchaser (~−1% median on local pin runs). Not a comparison to published SEF CSV or Zenodo NAICS factors.

Comparison targets (operator)

Goal Command / artifact Notes
% diff vs pinned phoebe workbook generate_diagnostics with useeio_phoebe_23 + useeio_baseline_pin.json; read N_perc_diff Validated path for model rebuild. Do not override model_base_year unless comparing at that dollar year.
Bedrock SEF closest to Zenodo v1.4.0 (2024 USD) uv run python -m bedrock.publish.emission_factors.cli --config_name useeio_phoebe_23 --dollar_year 2024 Phi(model_base_year) + commodity CPI to 2024; Cornerstone codes, not NAICS. Manual compare to Zenodo v1.4.0.
Diagnostics ≈ SEF without-margins column Only when model_base_year == --dollar_year (2017 on default config) SEF at --dollar_year 2024 is not comparable to default diagnostics N_new_purchaser without an extra CPI step.

combine_ef_diagnostics prefers N_new_purchaser / N_old_purchaser when input Sheets include them (documented in validation/analysis/README.md).

Testing

Unit tests (phi helpers and Excel writer):

uv run pytest bedrock/transform/iot/__tests__/test_phi_helpers.py bedrock/publish/__tests__/test_excel_writer.py -v

Phi wiring and dollar-year rebasing on SEF export:

uv run pytest bedrock/publish/__tests__/test_sef_phi_wiring.py -m eeio_integration -v

Pinned phoebe USEEIO workbook Phi parity:

uv run pytest bedrock/publish/__tests__/test_sef_vs_useeio_baseline.py -m eeio_integration -v

Validation module (diagnostics CLI and baseline loaders):

uv run pytest bedrock/utils/validation/__tests__/ -v

Parity notes

  • Phi @ 2017 matches the phoebe workbook Phi column 2017 within 1% rtol (eeio_integration only).
  • USEEIO diagnostics with purchaser adjustment on both sides: ~−1% median N_perc_diff vs pinned workbook on useeio_phoebe_23 (local pin runs).
  • No automated comparison to supply-chain-factors repo output or Zenodo published SEFs in this PR (Build supply-chain style CSV #449).

bl-young added 2 commits June 12, 2026 10:40
Wire PRO:PUR into SEF export, Excel Phi sheet, and USEEIO baseline
diagnostics (N_new_purchaser / N_old_purchaser).
@bl-young bl-young merged commit a718db2 into main Jun 16, 2026
4 checks passed
@bl-young bl-young deleted the by_integrate_phi branch June 16, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrate PUR:PRO ratios into diagnostics

2 participants