Skip to content

Impute unavailable prior-year CPS income#837

Open
MaxGhenis wants to merge 2 commits intomainfrom
codex/fix-prior-year-income-sentinels
Open

Impute unavailable prior-year CPS income#837
MaxGhenis wants to merge 2 commits intomainfrom
codex/fix-prior-year-income-sentinels

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

Summary

  • replace prior-year CPS income sentinel outputs with current-year income fallbacks
  • keep previous_year_income_available false when prior-year values are missing or sentinel-coded
  • add coverage for missing previous-year matches, -1, and -9999 prior-year income sentinels

Rationale

The CPS prior-year join previously encoded unavailable prior-year income as -1 via fillna(-1). In the live Enhanced CPS this made self_employment_income_last_year equal -1 for most person records, which reads as a real dollar amount downstream. This PR keeps numeric income fields numeric and interpretable while preserving the availability flag.

Tests

  • uv run pytest tests/unit/datasets/test_cps_file_handles.py tests/unit/datasets/test_cps_identification.py tests/unit/test_weeks_unemployed.py
  • uv run ruff check policyengine_us_data/datasets/cps/cps.py tests/unit/datasets/test_cps_file_handles.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant