Skip to content

oecd_regional_education |Increasing deletion threshold - #2209

Open
Ashwani725 wants to merge 12 commits into
datacommonsorg:masterfrom
Ashwani725:oecd_regional_education
Open

Ashwani725 wants to merge 12 commits into
datacommonsorg:masterfrom
Ashwani725:oecd_regional_education

Conversation

@Ashwani725

@Ashwani725 Ashwani725 commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

The PR is raised to Allow up to 5% deleted records in OECD regional education validation.

Root Cause

OECD transitioned to dataflow 2.5 (OECD.CFE.EDS:DSD_REG_EDU@DF_ATTAIN(2.5)), introducing Eurostat NUTS 2024 regional restructuring (retiring obsolete territorial units such as PT16, PT17, NL31, and NL33) and pruning historical zero-value estimates. This resulted in 9,531 legitimate record deletions (~3.90% of total observations).

##Test Result:
Test Cloud batch
Test GCS Bucket

@google-cla

google-cla Bot commented Sep 7, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the OECD regional education import manifest to include a node MCF file and a validation configuration file, and introduces a new validation configuration file to check the percentage of deleted records. The reviewer noted a mismatch in the validation configuration where the rule description specifies a 5% threshold, but the parameter is set to 4.

Comment thread statvar_imports/oecd/regional_education/validation_config.json Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants