[emmanuelgjr]

Summary

Items 10–11 of the data-validation review.

Anonymization beyond regex

• gitleaks (v8.30.1; the release tarball's SHA-256 is pinned in the workflow) runs on every PR over datasets/ and data_validation/, and fails on a finding. The current data has none: the datasets' synthetic secrets use placeholders, which gitleaks' real-credential rules don't match.
• qc_tools/ner_pii_scan.py: named-entity recognition with Microsoft Presidio and spaCy en_core_web_sm, run on the datasets whose rules require anonymization. It looks for person names, plus organisation names in incident_dataset, whose README forbids them.
  • Unfiltered, the model flagged ~400 acronyms ("PII", "EU", "AI") as organisations. A hit now counts only at ≥85% confidence and when it looks like a proper name (two or more capitalised words).
  • On current data that leaves one false positive ("Model Bulletin", a regulatory document), now in a reviewed allowlist (reference_data/ner_allowlist.txt). It still catches the fixture's planted "John Smith".
  • It reports warnings only, in the CI job summary. Its dependencies live in requirements-ner.txt so the core checks keep a single dependency.

Standard identifiers

• SPDX licences (validators/metadata_check.py): every license/*_license field must resolve to exactly one SPDX licence: an ID, an expression (MIT OR Apache-2.0, Apache-2.0 WITH LLVM-exception) or an SPDX full name, optionally followed by its short form. A mismatched pair such as "CC BY 4.0 International (CC BY-SA 4.0)" fails. All 898 licence strings in the data resolve.
• BCP 47 language tags: every language/languages/*_language field must be a well-formed tag whose subtags are in the IANA registry. *_label fields are skipped because they copy a source's own label (e.g. tr-en) verbatim.
• NIST AI 100-2e2025: reference_data/nist_ai_100_2.csv holds all 30 NISTAML.* identifiers. That's the document's Taxonomy Index plus NISTAML.016/.017, which appear only in §3.4.1. New optional nist_aml_mapping field in the exploit, vulnerability and agent-trace schemas, and crossref_validator rejects unknown NISTAML IDs.
• update_reference_data.py, and so the monthly refresh, now also regenerates the SPDX table (3.29.0, 826 licences and exceptions) and the IANA language table (2026-09-17, 8,802 subtags).

Verification

• 178 tests pass. The NER filter logic is tested with a stub analyzer, so the main test job doesn't need spaCy.
• run_all_checks.py: 0 errors. gitleaks: no leaks. NER: no findings after the allowlist. actionlint is clean.

🤖 Generated with Claude Code