CensusACS5YearSurvey_SubjectTables_S2702PR

scripts/us_census/acs5yr/subject_tables/s2702pr/README.md

@@ -0,0 +1,59 @@
+# US census ACS5YR S2702PR Subject Table import
+
+This subject table gives the population count and selected characteristics of the uninsured civilian noninstitutionalized population in Puerto Rico, constrained by age, sex, race and Hispanic or Latino origin, nativity and U.S. citizenship status, disability status, residence 1 year ago, educational attainment, employment status, work experience, class of worker, industry, occupation, earnings, household income, and ratio of income to poverty level.
+
+Years: 2013-2024
+
+Geo : All geographic levels in Puerto Rico from state to census tracts.
+
+## Download Step
+
+Run `census_api_data_downloader.py` from `scripts/us_census/api_utils` to download the survey data:
+
+```bash
+cd scripts/us_census/api_utils
+
+python3 census_api_data_downloader.py \
+  --dataset=acs/acs5/subject \
+  --table_id=S2702PR \
+  --start_year=2013 \
+  --end_year=2024 \
+  --all_summaries \
+  --output_path=/tmp/census_download \
+  --api_key=<YOUR_API_KEY>
+```
+
+Copy only the generated `.zip` file into `input_data/`:
+
+```bash
+mkdir -p scripts/us_census/acs5yr/subject_tables/s2702pr/input_data
+cp /tmp/census_download/acs/acs5/subject/S2702PR/S2702PR.zip scripts/us_census/acs5yr/subject_tables/s2702pr/input_data/S2702PR.zip
+```
+
+## Process Step
+
+Run the subject table processor from `scripts/us_census/acs5yr/subject_tables`:
+
+```bash
+cd scripts/us_census/acs5yr/subject_tables
+
+python3 process.py \
+  --option=all \
+  --table_prefix=s2702pr \
+  --has_percent=True \
+  --debug=False \
+  --spec_path=s2702pr/s2702pr_spec.json \
+  --input_path=s2702pr/input_data/S2702PR.zip \
+  --output_dir=s2702pr
+```
+
+- `--option=all`: Runs both column map generation (`column_map.json`) and observation processing (`s2702pr_cleaned.csv`, `s2702pr_output.mcf`, `s2702pr_output.tmcf`, `s2702pr_summary.json`).
+- `--has_percent=True`: Converts percentage values in the dataset to counts using the `denominators` mapping in `s2702pr_spec.json`.
+
+## Notes
+
+1. For Puerto Rico, Municipio is treated as the equivalent of County, and Puerto Rico itself as a state.
+2. Percentage values for uninsured population breakdowns are converted into corresponding counts using the `denominators` configuration in `s2702pr_spec.json`.
+3. There has been a change in the age brackets from [2017](https://www.census.gov/programs-surveys/acs/technical-documentation/table-and-geography-changes/2017/5-year.html) onwards (`Under 18 years` / `18 to 64 years` in 2013–2016 vs. `Under 19 years` / `19 to 64 years` from 2017 onwards).
+4. Estimate and Margin of Error columns for household income between `$50,000 to $74,999` (2019–2024) are ignored in `ignoreColumns` to maintain consistency with historical years (2013–2018) and avoid collisions with the `$50,000 to $74,999` earnings bracket.
+5. Margin of Error columns for median measurements (`Median age (years)`, `Median earnings (dollars)`, and `Median household income of householders`) are ignored across all years.

scripts/us_census/acs5yr/subject_tables/s2702pr/testdata/s2702pr_cleaned.csv

@@ -0,0 +1,100 @@
+Year,Place,StatVar,Quantity,Unit
+2013,geoId/72,dcid:Count_Person_NoHealthInsurance,655394.0,
+2013,geoId/72,dcid:MarginOfError_Count_Person_NoHealthInsurance,9050.0,
+2013,geoId/72,dcid:Count_Person_18OrLessYears_NoHealthInsurance,57674.672,
+2013,geoId/72,dcid:Median_Age_Person_NoHealthInsurance,33.1,Years
+2013,geoId/72,dcid:Count_Person_Male_NoHealthInsurance,342771.062,
+2013,geoId/72,dcid:Count_Person_NoHealthInsurance_OneRace,645563.09,
+2013,geoId/72,dcid:Count_Person_USCitizenBornInTheUnitedStates_NoHealthInsurance_Native,578712.902,
+2013,geoId/72,dcid:Count_Person_NoHealthInsurance_WithDisability,79302.674,
+2013,geoId/72,dcid:Count_Person_NoHealthInsurance_NoDisability,576091.326,
+2013,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold,653836.0,
+2013,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_SameHouse1YearAgo,505415.228,
+2013,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInTheUS,145151.592,
+2013,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInSameCounty,89575.532,
+2013,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInDifferentCountySameState,30076.456,
+2013,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseAbroad,3269.18,
+2013,geoId/72,dcid:Count_Person_25OrMoreYears_NoHealthInsurance,472813.0,
+2013,geoId/72,dcid:Count_Person_25OrMoreYears_EducationalAttainmentLessThanHighSchoolGraduate_NoHealthInsurance,134278.892,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance,608556.0,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_InLaborForce_NoHealthInsurance,433900.428,
+2013,geoId/72,dcid:Count_Person_16To64Years_NoHealthInsurance,605886.0,
+2013,geoId/72,dcid:Count_Person_16To64Years_NoHealthInsurance_FullTimeYearRoundWorker,172677.51,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_Worker,312982.0,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_PrivatelyOwnedForProfitEstablishment_NoHealthInsurance_PaidWorker_Worker,253828.402,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_NAICSAgricultureForestryFishingHuntingMiningQuarryingOilGasExtraction_Worker,8450.514,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_SOCManagementBusinessScienceArtsOccupation_Worker,37870.822,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_WithEarnings,406265.0,
+2013,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_IncomeOf50000To74999USDollar_WithEarnings,13813.01,
+2013,geoId/72,dcid:Median_Income_Person_16OrMoreYears_NoHealthInsurance_WithEarnings,14446.0,USDollar
+2013,geoId/72,dcid:Count_Household_NoHealthInsurance,648719.0,
+2013,geoId/72,dcid:Count_Household_NoHealthInsurance_IncomeOf25000OrLessUSDollar,260136.319,
+2013,geoId/72,dcid:Median_Income_Household_NoHealthInsurance,22151.0,USDollar
+2013,geoId/72,dcid:Count_Person_NoHealthInsurance_PovertyStatusDetermined,651864.0,
+2013,geoId/72,dcid:Count_Person_NoHealthInsurance_0.5OrLessRatioToPovertyLine,112772.472,
+2017,geoId/72,dcid:Count_Person_NoHealthInsurance,512029.0,
+2017,geoId/72,dcid:MarginOfError_Count_Person_NoHealthInsurance,7826.0,
+2017,geoId/72,dcid:Count_Person_19OrLessYears_NoHealthInsurance,41474.349,
+2017,geoId/72,dcid:Median_Age_Person_NoHealthInsurance,34.4,Years
+2017,geoId/72,dcid:Count_Person_Male_NoHealthInsurance,270351.312,
+2017,geoId/72,dcid:Count_Person_NoHealthInsurance_OneRace,503836.536,
+2017,geoId/72,dcid:Count_Person_USCitizenBornInTheUnitedStates_NoHealthInsurance_Native,451097.549,
+2017,geoId/72,dcid:Count_Person_NoHealthInsurance_WithDisability,60419.422,
+2017,geoId/72,dcid:Count_Person_NoHealthInsurance_NoDisability,451609.578,
+2017,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold,510847.0,
+2017,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_SameHouse1YearAgo,405101.671,
+2017,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInTheUS,102680.247,
+2017,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInSameCounty,64877.569,
+2017,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInDifferentCountySameState,21966.421,
+2017,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseAbroad,3065.082,
+2017,geoId/72,dcid:Count_Person_25OrMoreYears_NoHealthInsurance,385966.0,
+2017,geoId/72,dcid:Count_Person_25OrMoreYears_EducationalAttainmentLessThanHighSchoolGraduate_NoHealthInsurance,105368.718,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance,482773.0,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_InLaborForce_NoHealthInsurance,324423.456,
+2017,geoId/72,dcid:Count_Person_16To64Years_NoHealthInsurance,479387.0,
+2017,geoId/72,dcid:Count_Person_16To64Years_NoHealthInsurance_FullTimeYearRoundWorker,148609.97,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_Worker,256677.0,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_PrivatelyOwnedForProfitEstablishment_NoHealthInsurance_PaidWorker_Worker,210731.817,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_NAICSAgricultureForestryFishingHuntingMiningQuarryingOilGasExtraction_Worker,6160.248,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_SOCManagementBusinessScienceArtsOccupation_Worker,32084.625,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_WithEarnings,312865.0,
+2017,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_IncomeOf50000To74999USDollar_WithEarnings,14078.925,
+2017,geoId/72,dcid:Median_Income_Person_16OrMoreYears_NoHealthInsurance_WithEarnings,15858.0,USDollar
+2017,geoId/72,dcid:Count_Household_NoHealthInsurance,505756.0,
+2017,geoId/72,dcid:Count_Household_NoHealthInsurance_IncomeOf25000OrLessUSDollar,193198.792,
+2017,geoId/72,dcid:Median_Income_Household_NoHealthInsurance,23017.0,USDollar
+2017,geoId/72,dcid:Count_Person_NoHealthInsurance_PovertyStatusDetermined,508384.0,
+2017,geoId/72,dcid:Count_Person_NoHealthInsurance_0.5OrLessRatioToPovertyLine,93034.272,
+2024,geoId/72,dcid:Count_Person_NoHealthInsurance,457826.0,
+2024,geoId/72,dcid:MarginOfError_Count_Person_NoHealthInsurance,8546.0,
+2024,geoId/72,dcid:Count_Person_19OrLessYears_NoHealthInsurance,37083.906,
+2024,geoId/72,dcid:Median_Age_Person_NoHealthInsurance,34.9,Years
+2024,geoId/72,dcid:Count_Person_Male_NoHealthInsurance,244021.258,
+2024,geoId/72,dcid:Count_Person_NoHealthInsurance_OneRace,449127.306,
+2024,geoId/72,dcid:Count_Person_USCitizenBornInTheUnitedStates_NoHealthInsurance_Native,402886.88,
+2024,geoId/72,dcid:Count_Person_NoHealthInsurance_WithDisability,56770.424,
+2024,geoId/72,dcid:Count_Person_NoHealthInsurance_NoDisability,401055.576,
+2024,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold,456341.0,
+2024,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_SameHouse1YearAgo,364160.118,
+2024,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInTheUS,88530.154,
+2024,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInSameCounty,53848.238,
+2024,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseInDifferentCountySameState,19622.663,
+2024,geoId/72,dcid:Count_Person_1OrMoreYears_NoHealthInsurance_ResidesInHousehold_DifferentHouseAbroad,3650.728,
+2024,geoId/72,dcid:Count_Person_25OrMoreYears_NoHealthInsurance,351017.0,
+2024,geoId/72,dcid:Count_Person_25OrMoreYears_EducationalAttainmentLessThanHighSchoolGraduate_NoHealthInsurance,95125.607,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance,431865.0,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_InLaborForce_NoHealthInsurance,286758.36,
+2024,geoId/72,dcid:Count_Person_16To64Years_NoHealthInsurance,428973.0,
+2024,geoId/72,dcid:Count_Person_16To64Years_NoHealthInsurance_FullTimeYearRoundWorker,141132.117,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_Worker,236691.0,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_PrivatelyOwnedForProfitEstablishment_NoHealthInsurance_PaidWorker_Worker,193139.856,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_NAICSAgricultureForestryFishingHuntingMiningQuarryingOilGasExtraction_Worker,4733.82,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_SOCManagementBusinessScienceArtsOccupation_Worker,31716.594,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_WithEarnings,282684.0,
+2024,geoId/72,dcid:Count_Person_16OrMoreYears_NoHealthInsurance_IncomeOf50000To74999USDollar_WithEarnings,15264.936,
+2024,geoId/72,dcid:Median_Income_Person_16OrMoreYears_NoHealthInsurance_WithEarnings,17126.0,USDollar
+2024,geoId/72,dcid:Count_Household_NoHealthInsurance,452061.0,
+2024,geoId/72,dcid:Count_Household_NoHealthInsurance_IncomeOf25000OrLessUSDollar,154604.862,
+2024,geoId/72,dcid:Median_Income_Household_NoHealthInsurance,25620.0,USDollar
+2024,geoId/72,dcid:Count_Person_NoHealthInsurance_PovertyStatusDetermined,454201.0,
+2024,geoId/72,dcid:Count_Person_NoHealthInsurance_0.5OrLessRatioToPovertyLine,79485.175,