Skip to content

Impute unavailable prior-year CPS income#836

Closed
MaxGhenis wants to merge 1 commit intoPolicyEngine:mainfrom
MaxGhenis:codex/fix-prior-year-income-sentinels
Closed

Impute unavailable prior-year CPS income#836
MaxGhenis wants to merge 1 commit intoPolicyEngine:mainfrom
MaxGhenis:codex/fix-prior-year-income-sentinels

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

Summary

  • replace prior-year CPS income sentinel outputs with current-year income fallbacks
  • keep previous_year_income_available false when prior-year values are missing or sentinel-coded
  • add coverage for missing previous-year matches, -1, and -9999 prior-year income sentinels

Rationale

The CPS prior-year join previously encoded unavailable prior-year income as -1 via fillna(-1). In the live Enhanced CPS this made self_employment_income_last_year equal -1 for most person records, which reads as a real dollar amount downstream. This PR keeps numeric income fields numeric and interpretable while preserving the availability flag.

Tests

  • uv run pytest tests/unit/datasets/test_cps_file_handles.py tests/unit/datasets/test_cps_identification.py tests/unit/test_weeks_unemployed.py
  • uv run ruff check policyengine_us_data/datasets/cps/cps.py tests/unit/datasets/test_cps_file_handles.py

@MaxGhenis
Copy link
Copy Markdown
Contributor Author

Closing and reopening from a branch in PolicyEngine/policyengine-us-data because CI blocks fork-based PRs.

@MaxGhenis MaxGhenis closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant