Skip to content

Restore Social Security retirement input and remove reported benefit switches#8263

Merged
MaxGhenis merged 9 commits into
mainfrom
codex/remove-ss-retirement-reported
May 12, 2026
Merged

Restore Social Security retirement input and remove reported benefit switches#8263
MaxGhenis merged 9 commits into
mainfrom
codex/remove-ss-retirement-reported

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

@MaxGhenis MaxGhenis commented May 12, 2026

Summary

Restores social_security_retirement as the canonical microsimulation input and removes social_security_retirement_reported plus gov.simulation.reported_social_security_retirement.

Also removes package-side reported-data switches and public reported variables where policyengine-us can calculate or users can override the canonical variable directly:

  • Deletes gov.simulation.reported_snap; snap always uses the modeled SNAP formula, with microsimulation participation controlled by takes_up_snap_if_eligible.
  • Deletes public snap_reported and tanf_reported; WA ECEAP now uses canonical snap, tanf, and ssi variables.
  • Makes is_ssi_disabled depend on modeled disability status instead of ssi_reported receipt.
  • Removes reported WIC, school-meals, SPM tax, state-income-tax, and SALT paths.
  • Removes thin spm_unit_wic, spm_unit_pell_grant, and spm_unit_broadband_subsidy wrappers; callers now aggregate canonical variables with add, and SPM net income includes modeled ACP/EBB directly as broadband subsidies.
  • Renames the remaining data-backed housing and energy SPM subsidy leaves to spm_unit_capped_housing_subsidy_data and spm_unit_energy_subsidy_data.
  • Fixes ACP/EBB categorical eligibility vectorization so one SPM unit's eligibility cannot leak to others.

Lower-level SSA calculation internals remain in place, but are not wired into social_security_retirement. The data contract is: us-data supplies non-formulaic leaf inputs, policyengine-us calculates formulaic variables, and API users override canonical variables directly when needed.

Root cause

PR #8040 made social_security_retirement formula-driven by default and redirected observed data through social_security_retirement_reported. Current data artifacts and downstream microsim usage expect the canonical variable to be social_security_retirement, so the split can drop or ignore retirement Social Security inputs and distort senior SPM resources.

The same audit found several package formulas using reported survey variables as model controls. Reported survey fields can be used by us-data to construct calibrated inputs, but package formulas and user-facing APIs should calculate or override canonical model variables directly.

Validation

  • uv run ruff check ... on touched Python files
  • uv run pytest policyengine_us/tests/policy/contrib/ssa/test_trustees_core_thresholds.py -q
  • uv run pytest policyengine_us/tests/core/test_run_selective_tests.py -q
  • uv run pytest policyengine_us/tests/policy/baseline/gov/fcc/test_vectorization.py -q
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/ssa/social_security --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/ssa/ssi --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/usda/snap --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/hhs/tanf --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/usda/wic --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/usda/school_meals --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/fcc --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/ed/pell_grant/pell_grant_simplified_formula_applies.yaml --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/irs/deductions/itemized/salt_and_real_estate --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/states/tax/income --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/household/income/household/household_state_income_tax.yaml --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/household/household_state_income_tax.yaml --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/household/income/spm_unit --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/household/expense --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/baseline/gov/states/wa/dcyf/eceap --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/contrib/congress/tlaib/boost/boost_middle_class_tax_credit.yaml --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/contrib/congress/tlaib/end_child_poverty_act/integration.yaml --batches 1
  • uv run python policyengine_us/tests/test_batched.py policyengine_us/tests/policy/contrib/congress/tlaib/economic_dignity_for_all_agenda/edaa_end_child_poverty_act.yaml --batches 1
  • registry check confirmed deleted reported/wrapper variables are absent and canonical social_security_retirement, snap, tanf, wic, pell_grant, acp, and ebb remain present
  • git diff --check

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

❌ Patch coverage is 16.66667% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.46%. Comparing base (8cb60e7) to head (f6d4212).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
...able_income/deductions/itemizing/salt_deduction.py 0.00% 7 Missing ⚠️
...ariables/gov/states/tax/income/state_income_tax.py 0.00% 5 Missing ⚠️
policyengine_us/variables/gov/usda/snap/snap.py 0.00% 4 Missing ⚠️
...old/income/household/household_state_income_tax.py 0.00% 3 Missing ⚠️
...to_three_eceap/wa_birth_to_three_eceap_eligible.py 0.00% 2 Missing ⚠️
...engine_us/variables/gov/fcc/ebb/is_ebb_eligible.py 0.00% 1 Missing ⚠️
.../gov/ssa/ssi/eligibility/status/is_ssi_disabled.py 0.00% 1 Missing ⚠️
...income/spm_unit/spm_unit_capped_housing_subsidy.py 0.00% 1 Missing ⚠️
...usehold/income/spm_unit/spm_unit_energy_subsidy.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##              main    #8263       +/-   ##
============================================
- Coverage   100.00%   70.46%   -29.54%     
============================================
  Files            3     4723     +4720     
  Lines           63    68766    +68703     
  Branches         0      340      +340     
============================================
+ Hits            63    48457    +48394     
- Misses           0    20307    +20307     
- Partials         0        2        +2     
Flag Coverage Δ
unittests 70.46% <16.66%> (-29.54%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MaxGhenis MaxGhenis changed the title Restore Social Security retirement as input Restore Social Security retirement input and remove reported benefit switches May 12, 2026
@MaxGhenis MaxGhenis marked this pull request as ready for review May 12, 2026 16:24
@MaxGhenis MaxGhenis merged commit 06a5282 into main May 12, 2026
@MaxGhenis MaxGhenis deleted the codex/remove-ss-retirement-reported branch May 12, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant