Triage policyengine-us-data backlog for Populace migration

[MaxGhenis]

Context

policyengine-us-data is no longer the live build surface for replacement microdata. Populace supersedes it, so open policyengine-us-data issues/PRs should be triaged into Populace work only when the underlying data-quality, target, export, or release-contract concern is still relevant.

This issue records a pass over the live open PolicyEngine/policyengine-us-data issues and PRs on 2026-06-14, after Populace PRs #44 and #45 merged.

Already covered or mostly covered in Populace

• policyengine-us-data #1156 / PR #1157: microunit tax-unit construction. Populace already delegates US tax units and SPM units to microunit through populace-frame.
• policyengine-us-data #1171 / PR #1171: negative auto_loan_interest from SCF sentinels. Populace already has manifest-declared sentinel handling plus nonnegative source/export gates for auto_loan_interest.
• policyengine-us-data #1105, #1107, #872, #476, #321, UK: household-wealth imputation stage with cash / S&S ISA split #181, social_security stored input contradicts its four components for ~2k records (decomposition gap) #183, Add proximal L1 solver path #184, Add fiscal diagnostics comparison tool #190, Remove count aggregation from requirements example #192: fiscal/SOI/JCT/state-tax target surface. Populace US calibration does not pin federal income tax or AGI; base undershoots income tax ~18% — add SOI fiscal targets + a macro-realism gate #40 / PR Restore US fiscal calibration targets #44 added federal income tax, JCT tax-expenditure reform contracts, state income tax, SOI rows, and macro-realism gates. Keep remaining target refreshes as future target-vintage work, not old-repo work.
• policyengine-us-data Make zero-valued calibration targets intentionally nonzero-weighted #104, #596 and PE-US #8614 follow-up: partnership/S-corp formula aggregate vs leaves. Populace Propagate partnership/S-corp income split (policyengine-us#8614) into the US build #35 / PR Enforce partnership S-corp leaf exports #45 now exports the PE-US leaves and blocks partnership_s_corp_income as a formula-owned compatibility column.
• policyengine-us-data #1165 and PR #1054 concepts: target coverage / target-policy reporting. Populace Use Arch US poverty/nonfiler coverage contract in Populace target gates #29 and Add NIPA and program-admin nonfiler controls to the US target profile #30 already track source-family coverage and hard-target inventory. Copy only the useful fail / warn / diagnostic_only policy concept if Populace needs more granular enforcement.

Copy into existing Populace issues

• Populace Add NIPA and program-admin nonfiler controls to the US target profile #30: program-admin hard targets and nonfiler controls.
  • Copy/source-link policyengine-us-data #1148 / PR #1162 for CHIP enrollment controls.
  • Copy/source-link #1147, #620 / PR #1087, #889, #510 for ACA marketplace/APTC take-up and state targets.
  • Copy/source-link #1136, #557 for Medicaid category/enrollment reconciliation.
  • Copy/source-link #1094, Assign US congressional districts from Ledger facts #188, Add opt-in ACS congressional district age targets #189, [corrected] populace_us_2024 itemized-deduction report retracted (measurement artifact); residual = state-level SOI income targets #122, #1021 for SSI/OASDI/disability support and take-up.
  • Copy/source-link #689, #535, #532 for LIHEAP/TANF/WIC/SNAP/take-up details where they map to source-backed targets.
• Populace Track SPM-specific input gaps: housing, WIC, school meals, child support, workers comp, and expense deductions #32/Add a Populace US poverty/SPM diagnostic gate for resource distribution and negative-income tails #33: SPM/resource diagnostics.
  • Copy/source-link #1113, #736, #733, #735, #739, #740, #732, Treat PE-US aggregate outputs as formula-owned #182, #445, #381 for housing assistance, SPM expenses, child support, CE/MEPS/AHS-style source gaps and validation-only diagnostics.
• Populace Populate remaining US tax-input and reported-observation layers #38: remaining US tax-input and reported-observation layers.
  • Copy/source-link #1126, #1127 / PR #1128, #900 for capital-gains basis / holding period / SOCA-style imputation.
  • Copy/source-link qualified_tuition_expenses essentially unimputed (641 records, $4.5B) — education credits understated ≥40% #253, Run ACA source refresh in fiscal release builder #85, #619, #613 for section 179, Alaska PFD separation, liability pruning, and top-tail AGI support.
  • Copy/source-link #1160, Add calibration warm-start and target-frame checkpoints #227, #525, #526, #527, #529 for tips and Stage-1 source quality failures.
  • Copy/source-link #712, Score legacy flat PE H5s #219, Add direct CHIP enrollment targets to the US fiscal surface #139, Allow staging telemetry repo config from environment #138, UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation #180, Remove aggregation method from Ledger target selectors #193, Score OBBBA reform validation stacked (match JCT) #185 for TIN/SSN/pay-period/pregnancy/widowship support if those remain PE-US input needs.
• Populace Drop raw-source and build-intermediate columns from the published populace-us artifact #27/Fix in_nyc before enforcing no-formula exports #34: export-surface and formula-owned storage.
  • Copy/source-link #444, #829, #348, #349, #814, #441, Add UK SPI support rows #149 where they identify formula-owned exports, entity-level ID propagation, filing-status, or naming issues still relevant to Populace.

New Populace follow-up issues to consider

• PUF real-half / clone support and release gates.
  • Source: policyengine-us-data #1139, #1158 / PR #1159, PR #1170, #555, #560, #590, #595, #541.
  • Populace should decide whether it needs explicit real-half vs clone-half support diagnostics, a PUF-clone weight-share floor/cap, and smoke tests for PUF-imputed charitable/business/tax-detail variables. This should stay spec-driven: country manifests can declare the support contract, but shared runtime code should implement the operation.
• Geography and local-area support.
  • Source: #1143, #1144, #1145, #1146, #482, #454, Materialize Marketplace selected-plan benchmark ratios and metal-level diagnostics #75, #517, #470, #598, PR #1088.
  • Populace should track boundary vintage, district geojson provenance, county/rating-area assignment, local H5/local-area build contracts, and matrix scaling separately from the old staged pipeline.
• Release manifest / publication contract.
  • Source: #724, #799 / PR #1167, Use target-period Medicaid facts in the 2024 US fiscal refresh #91, #600, #641, #616, #310, Tracked publish wrapper to wire release Slack alerts #179, Port UK compact source stages and 2024-25 refresh from uk-data #145, Cache US target reform materialization #201, Fill missing current CD targets from state totals #209, PR #1071, PR #1077.
  • Populace needs the useful release-manifest ideas, compatible model version metadata, diagnostics summaries, and published-run indices, but not the old eCPS Stage 5 implementation shape.
• Wealth/property/asset imputation.
  • Source: #1079, #1080, #1081 / PR #1085, Add coverage-universe targets for uninsured and private coverage #230, Calibrate the undocumented population count as a Ledger fact (weights lane follow-up to #266) #267, Polish follow-ups from the target-aging review (#280) #281, #835, #734.
  • Populace already declares scf_primary_residence_value; still verify whether PE-US-facing primary_residence_value, home equity, vehicle variables, auto-loan eligibility, net-worth decomposition, and Fed Financial Accounts benchmark replacement need dedicated Populace tasks.

Move outside the live Populace repo

• policyengine-us-data #1163 / PR #1164: production eCPS baseline resolver and integrity gate. The lesson is important, but incumbent/eCPS comparison should live in PolicyEngine/populace-benchmarks or another benchmark repo, not in live Populace build code.
• Any remaining eCPS/local-baseline promotion comparisons should be benchmark-repo work only. Populace release gates can expose diagnostics; they should not depend on a stale incumbent artifact.

Do not migrate directly

• Microplex-specific issues such as #1142 and #520.
• L0/national-penalty work such as PR #1166 and PR #685, except for high-level calibration-policy lessons already captured above. Populace should stay on its current calibration stack and spec-driven country content.
• Old staged-pipeline service-boundary PRs/issues (#1118, #1116, #1104, #1100, #1098, #1086, #1083, #1078, #1074, #1073, #1066, #1049 and related PRs #1119, #1117, #1106, #1101, #1099, #1084, #1082, #1076, #1075, #1051, #1068) unless a specific contract concept is missing in Populace.
• Temporary threshold loosening / broken eCPS build patches (#1160 / PR #1161) except for the underlying target or source-quality causes listed above.

Acceptance criteria

• Existing Populace issues above are updated or linked with the relevant source backlog.
• Any missing Populace issues from the follow-up section are opened or explicitly rejected.
• No old policyengine-us-data, Microplex, L0, or eCPS comparison implementation is copied into live Populace without translating it into Populace's spec-driven architecture.
• Benchmark-only eCPS comparison work is tracked outside live Populace.

[MaxGhenis]

Follow-up trackers opened from this audit:

• Port US PUF real-half and clone support diagnostics #47 for PUF real-half / clone support diagnostics and release gates.
• Track US geography and local-area source support #48 for geography, local-area support, boundary vintage, and rating-area/county assignment.
• Verify US wealth, property, and asset imputation backlog #49 for wealth, property, and asset imputation backlog verification.
• Port release manifest and publication diagnostics contracts #50 for release manifest, compatible model version metadata, and publication diagnostics contracts.

Existing tracker cross-links added to #27, #29, #30, #32, #34, and #38.

[MaxGhenis]

2026-06-27 delta while working toward #204:

• New policyengine-us-data #1172 / PR #1173 is old-stack release-orchestration cleanup. It does not need a Populace data migration, but Replace all remaining policyengine-us-data artifacts with Populace #204 should preserve the useful contract: .py owns stack inclusion, while Populace owns data release artifacts and manifests.
• Old PE-US-data PRs still open around eCPS thresholds, local baseline comparison, old Stage 1-5 service boundaries, publication runners, and bundle contracts should not be merged as a path to replacement. Translate only the durable contract into Populace specs/diagnostics/release tooling.
• The remaining content-bearing old issues still map to existing Populace trackers: PUF clone/support (Port US PUF real-half and clone support diagnostics #47), local geography/CDs (Track US geography and local-area source support #48), wealth/property (Verify US wealth, property, and asset imputation backlog #49), release manifest/publication (Port release manifest and publication diagnostics contracts #50/Replace all remaining policyengine-us-data artifacts with Populace #204), fiscal/source target coverage (Add NIPA and program-admin nonfiler controls to the US target profile #30/Populate remaining US tax-input and reported-observation layers #38/Consume Ledger target profiles instead of carrying target values in Populace #160), health/ACA/CHIP (Jointly calibrate ACA take-up to CMS APTC people and SOI PTC returns #168-Add Medicaid and CHIP eligibility-to-enrollment diagnostics #170, Materialize Marketplace selected-plan benchmark ratios and metal-level diagnostics #75/Migrate ACA health-input gates after PE-US take-up contract changes #76/Track health input readiness for upcoming PolicyEngine-US model improvements #98), and formula-constrained leaf imputation (Use formula-constrained leaf imputation for deduction inputs #186).
• Active replacement build: a current-main full CD target-surface rerun is queued with CD rows diagnostic-only for small releases; results should feed Replace all remaining policyengine-us-data artifacts with Populace #204 acceptance criteria before any .py default move.

[MaxGhenis]

Completed 2026-07-02: the full policyengine-us-data backlog (146 issues, 27 PRs) was triaged and closed using this issue's mapping as the authoritative source, the README archive banner merged (policyengine-us-data#1177), and the repository is now archived. Per-item mapping recorded on policyengine-us-data#1175.