diff --git a/.github/workflows/datasets-validate-all.yml b/.github/workflows/datasets-validate-all.yml index ab6dca7..acfcdc9 100644 --- a/.github/workflows/datasets-validate-all.yml +++ b/.github/workflows/datasets-validate-all.yml @@ -1,8 +1,11 @@ name: datasets-validate-all -# Runs data_validation/run_all_checks.py, which runs every dataset's own -# validate.py, so the PR template's "validation scripts pass" box is checked -# by CI for every dataset rather than only the ones with their own workflow. +# Runs data_validation/run_all_checks.py: every dataset's own validate.py +# (including ones in sub-collections), then the shared checks in +# data_validation/validators/ on every data file. Then the data_validation +# test suite, and the cross-dataset consistency check. The coverage report +# and anomaly list go to the job summary for reviewers; they never fail +# the job. on: push: branches: [main] @@ -31,11 +34,27 @@ jobs: with: python-version: '3.11' - - name: Install jsonschema and pytest - run: python -m pip install --quiet jsonschema pytest + - name: Install requirements + run: python -m pip install --quiet -r data_validation/requirements.txt - - name: Run every dataset validator - run: python data_validation/run_all_checks.py + - name: Run every dataset validator and the shared checks + run: python data_validation/run_all_checks.py --verbose - - name: Runner tests - run: python -m pytest data_validation/tests/test_run_all_checks.py -v + - name: Cross-dataset consistency + run: python data_validation/qc_tools/consistency_check.py + + - name: data_validation tests + run: python -m pytest data_validation/tests -v + + - name: Coverage report and anomalies (job summary) + if: always() + run: | + python data_validation/qc_tools/bias_report.py >> "$GITHUB_STEP_SUMMARY" + { + echo + echo '## Anomalies' + echo + echo '```' + python data_validation/qc_tools/anomaly_detector.py + echo '```' + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.gitignore b/.gitignore index db1fd0c..17b8307 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,8 @@ node_modules/ __pycache__/ *.pyc .env +venv/ +.venv/ # DSGAI scanner generated artifacts DSGAI-scan.json diff --git a/data_validation/README.md b/data_validation/README.md index 7e39da6..a8283e7 100644 --- a/data_validation/README.md +++ b/data_validation/README.md @@ -6,134 +6,132 @@ Part of the [OWASP GenAI Data Security Initiative](../README.md) · Workstream 5 ## Overview -This directory contains the data validation and quality control (QC) framework for all data contributed to the OWASP GenAI Data Security Initiative. Every dataset in the [`/datasets`](../datasets) directory — whether community-contributed vulnerability reports, incident records, framework mappings, or adversarial test cases — passes through this validation pipeline before being accepted. +This directory holds the validation and quality control (QC) tooling for the data in [`/datasets`](../datasets). Every pull request that changes `datasets/` or `data_validation/` runs it in CI (`.github/workflows/datasets-validate-all.yml`), and contributors can run the same thing locally with one command: -The framework serves two purposes: ensuring the integrity and accuracy of research data that informs OWASP deliverables (Top 10 for LLM Applications, DSGAI risk taxonomy, framework mappings), and providing reusable validation tooling that the community can adapt for their own GenAI data security workflows. - ---- - -## What Gets Validated +```bash +python data_validation/run_all_checks.py +``` -| Dataset | Key Validation Checks | -|---|---| -| [`vulnerability_dataset`](../datasets/vulnerability_dataset) | DSGAI mapping accuracy, CVE/CWE reference verification, severity consistency, deduplication | -| [`exploit_dataset`](../datasets/exploit_dataset) | Source verification, MITRE ATLAS alignment, prerequisite completeness, responsible disclosure check | -| [`riskassessment_dataset`](../datasets/riskassessment_dataset) | Anonymization verification, DSGAI coverage, framework control alignment, internal consistency | -| [`incident_dataset`](../datasets/incident_dataset) | Anonymization completeness, DSGAI mapping, source attribution, deduplication against existing entries | -| [`promptinj_dataextraction_testcases`](../datasets/promptinj_dataextraction_testcases) | DSGAI mapping, expected behavior definitions, severity classification, no undisclosed zero-days | -| [`rag_dataset`](../datasets/rag_dataset) | Synthetic data verification (no real PII/PHI), payload labeling, adversarial content safety check | -| [`crossframework_mapping_dataset`](../datasets/crossframework_mapping_dataset) | Control ID verification against source framework, relationship type accuracy, schema compliance | -| [`agentdataflow_toolexchange_traces`](../datasets/agentdataflow_toolexchange_traces) | Sanitization completeness (no real credentials/PII), structural integrity, DSGAI annotation accuracy | +The tooling has two jobs: keep the research data behind OWASP deliverables (the DSGAI risk taxonomy, framework mappings) accurate, and give the community validation code it can adapt for its own GenAI data security work. --- ## Validation Pipeline -All contributed data goes through a two-stage validation process. - -### Stage 1 — Automated Validation +### Stage 1 — Automated checks -Automated scripts in this directory perform structural and consistency checks on contributed data. These run on every pull request that modifies the `/datasets` directory. +`run_all_checks.py` runs two layers. -**Schema validation** — Verifies that contributed files conform to the expected schema for their dataset (field names, types, required fields, allowed values). Each dataset defines its schema in its own README. +**1. Dataset validators.** Each dataset that has a `validate.py` (at its root, or in a sub-collection folder) is run in its own directory. These are the authoritative checks for their data: schema conformance against the dataset's own `schema.json`, plus rules a schema cannot express. Datasets without one are listed as `NO VALIDATOR` so the gap is visible. -**DSGAI mapping verification** — Checks that referenced DSGAI IDs (DSGAI01–DSGAI21) exist and that the mapping is plausible given the content. Flags entries that reference non-existent IDs or where the described risk clearly misaligns with the mapped DSGAI entry. - -**Cross-reference verification** — For entries that cite CVEs, CWEs, MITRE ATLAS technique IDs, or framework control IDs, validates that the referenced identifiers exist and are formatted correctly. - -**Deduplication** — Compares new entries against existing dataset records to identify potential duplicates based on content similarity, source URLs, and key field matching. - -**Anonymization checks** — For datasets requiring anonymization (incidents, risk assessments, agent traces), scans for patterns that may indicate residual PII, credentials, internal hostnames, or organization-identifying information. - -**Completeness checks** — Verifies that all required fields are populated and that optional fields, when present, contain valid data. +| Dataset | Own validator | +|---|---| +| [`agentdataflow_toolexchange_traces`](../datasets/agentdataflow_toolexchange_traces) | `validate.py`: schema, span graph, provenance evidence, secret and IP scan | +| [`exploit_dataset`](../datasets/exploit_dataset) | `validate.py`: schema, DSGAI IDs, ATLAS ID consistency | +| [`riskassessment_dataset`](../datasets/riskassessment_dataset) | `validate.py`: schema, DSGAI IDs | +| [`vulnerability_dataset`](../datasets/vulnerability_dataset) | `validate.py`: schema, DSGAI IDs | +| [`promptinj_dataextraction_testcases`](../datasets/promptinj_dataextraction_testcases) | `tr_turkish_conversation_prompt_injection_pairs/validate.py` for that collection only | +| [`rag_dataset`](../datasets/rag_dataset), [`incident_dataset`](../datasets/incident_dataset) | `validate.py`: schema (from `data_validation/schemas/`), DSGAI IDs, ID matches file name | +| [`crossframework_mapping_dataset`](../datasets/crossframework_mapping_dataset) | none; superseded by the [GenAI Crosswalk](https://github.com/GenAI-Security-Project/crosswalk) | -### Stage 2 — Peer Review +**2. Shared checks** (`validators/`), on every data file in every dataset: -After automated checks pass, contributions are reviewed by cybersecurity and AI security experts from the initiative's contributor community. Peer review focuses on areas that automated tooling cannot reliably assess. +| Check | ERROR (fails the run) | WARN (for the reviewer) | +|---|---|---| +| `schema_validator.py` | Schema violations, for files no dataset validator covers | File with no schema to check against | +| `dsgai_mapping_check.py` | DSGAI ID not in the taxonomy; a mapping value that is not a DSGAI ID | Same ID twice in one mapping; more than 6 mappings | +| `crossref_validator.py` | Malformed CVE/GHSA/CWE/ATLAS ID; CVE year in the future; unknown CWE or ATLAS ID | Deprecated CWE; retired ATLAS ID; malformed ID in free text (often a placeholder) | +| `anonymization_scanner.py` | — | Email addresses, keys and tokens, private or (in a network context) public IPs, internal hostnames, home paths, SSNs | +| `dedup_checker.py` | Two records in one dataset with the same ID | Records with identical or ≥ 80% similar text | -**Accuracy and plausibility** — Does the described vulnerability, incident, or technique reflect a real and correctly characterized GenAI data security issue? +Anonymization and similarity hits are warnings because a pattern cannot tell a real secret from a synthetic one; the prompt-injection datasets contain fake credentials on purpose. Use obviously fake values to keep a file quiet: `example.com` and other RFC 2606 domains, the RFC 5737 documentation addresses (`192.0.2.0/24`, `198.51.100.0/24`, `203.0.113.0/24`), or placeholders such as `` or `[API_KEY]`. -**DSGAI alignment quality** — Is the DSGAI mapping the best fit, or would a different entry be more appropriate? Are secondary mappings needed? +CVE and GHSA IDs are format-checked only; there is no offline list to confirm they exist. Framework control IDs (ISO, NIST, CIS) are not checked: see [`reference_data/SOURCES.md`](reference_data/SOURCES.md). -**Severity calibration** — Is the assigned severity consistent with similar entries in the dataset and with the DSGAI mitigation tier structure? +### Stage 2 — Peer review -**Bias assessment** — Does the contribution introduce systematic bias into the dataset (e.g., overrepresentation of a specific vendor, framework, or attack class)? +After the automated checks pass, contributions are reviewed by cybersecurity and AI security practitioners from the initiative's contributor community. Peer review covers what tooling cannot reliably judge: -**Responsible disclosure** — For exploit and vulnerability entries, confirms that the technique is publicly documented and that submission does not constitute premature disclosure. +- **Accuracy and plausibility.** Does the entry describe a real, correctly characterized GenAI data security issue? +- **DSGAI alignment.** Is the mapping the best fit? Are secondary mappings needed? The tooling only confirms the IDs exist. +- **Severity calibration.** Is the severity consistent with similar entries? +- **Bias.** Does the contribution skew the dataset toward one vendor, framework or attack class? The QC reports below help here. +- **Responsible disclosure.** For exploits and vulnerabilities, is the technique already public? +- **Anonymization.** Are the scanner's warnings real? Is anything identifying that no pattern would catch? --- -## Scripts +## Quality control tools -### Validation Scripts +`qc_tools/` produces reports for reviewers. CI writes the coverage report and the anomaly list to the job summary of every run. -| Script | Purpose | +| Script | What it reports | |---|---| -| `schema_validator.py` | Validates contributed files against dataset-specific JSON schemas | -| `dsgai_mapping_check.py` | Verifies DSGAI ID references and flags mapping misalignments | -| `crossref_validator.py` | Checks CVE, CWE, MITRE ATLAS, and framework control ID references | -| `dedup_checker.py` | Identifies potential duplicate entries across datasets | -| `anonymization_scanner.py` | Scans for residual PII, credentials, and identifying information | -| `completeness_check.py` | Reports missing required fields and invalid optional field values | - -### Quality Control Tools - -| Script | Purpose | -|---|---| -| `anomaly_detector.py` | Identifies statistical outliers in severity distributions, DSGAI mapping concentrations, and submission patterns | -| `bias_report.py` | Generates coverage reports showing dataset distribution across DSGAI entries, industries, attack vectors, and frameworks to surface gaps and overrepresentation | -| `consistency_check.py` | Cross-validates related entries across datasets (e.g., an exploit entry should have a corresponding vulnerability entry if both describe the same issue) | +| `bias_report.py` | Markdown coverage report: records per dataset, a DSGAI × dataset matrix, DSGAI entries nothing maps to, and the spread of severity and each dataset's categorical fields | +| `anomaly_detector.py` | A DSGAI entry most of a dataset maps to, records with far more mappings than usual, and one severity or category value dominating a dataset (datasets of 10+ records) | +| `consistency_check.py` | **Fails** on internal references (`DSGAI-VULN-…`, `DSGAI-EXP-…`, `DSGAI-RA-…`) that point at no existing entry; notes CVEs cited by exploits that have no vulnerability entry | --- -## Adapting the Scripts - -These scripts are designed to be reusable. Organizations can adapt them for validating their own internal GenAI security data. Common adaptations include: +## Running the checks -**Parameter tuning** — Adjusting similarity thresholds for deduplication, modifying anonymization detection patterns for your data types, or tuning anomaly detection sensitivity. +Python 3.10 or higher. -**Custom schema definitions** — Adding dataset-specific schemas for internal data formats beyond what the initiative uses. +```bash +pip install -r data_validation/requirements.txt -**Additional cross-references** — Extending the cross-reference validator to check against internal control frameworks, proprietary vulnerability databases, or organization-specific risk taxonomies. +python data_validation/run_all_checks.py # everything +python data_validation/run_all_checks.py --dataset datasets/exploit_dataset +python data_validation/run_all_checks.py --verbose # every validator's output and every warning +python data_validation/run_all_checks.py --strict # warnings fail too -**CI/CD integration** — Wiring the validation scripts into your own pull request or data ingestion pipelines. +# One check, on chosen files or datasets (each script's docstring has details) +python data_validation/validators/schema_validator.py --file datasets/rag_dataset/entries/RAG-0001.json +python data_validation/validators/dedup_checker.py --file my_new_entry.json --dataset datasets/exploit_dataset ---- +# Reports +python data_validation/qc_tools/bias_report.py --output coverage.md +python data_validation/qc_tools/anomaly_detector.py +python data_validation/qc_tools/consistency_check.py -## Getting Started +# Tests for the tooling itself +python -m pytest data_validation/tests +``` -### Prerequisites +`run_all_checks.py` exits 0 when everything passed, 1 on any failure, and 2 when there was nothing to check. -- Python 3.8 or higher -- Dependencies listed in `requirements.txt` +--- -### Setup +## Layout -```bash -cd data_validation -pip install -r requirements.txt +``` +data_validation/ +├── run_all_checks.py One command for everything (CI runs this) +├── validators/ Shared checks; each also runs on its own +├── qc_tools/ Reports for reviewers +├── schemas/ Schemas for datasets without their own schema.json +│ (the rest are pointers to the dataset-local schemas) +├── reference_data/ DSGAI, MITRE ATLAS and CWE lookup tables; see SOURCES.md +└── tests/ Tests and deliberately valid/invalid fixtures ``` -### Running Validation +Dataset-local schemas (`datasets//schema.json`) are authoritative. `schemas/exploit`, `vulnerability`, `riskassessment` and `agentdataflow_trace` are kept only as `$ref` pointers to them; `schemas/incident`, `rag`, `crossframework_mapping` and `promptinj_testcase` are the only schemas for their data. -```bash -# Validate a single contributed file -python schema_validator.py --file ../datasets/vulnerability_dataset/new_entry.json +The ATLAS and CWE tables are generated from MITRE's releases by `reference_data/update_reference_data.py`; don't edit them by hand. + +--- -# Run every dataset's own validate.py (or one dataset with --dataset) -python run_all_checks.py -python run_all_checks.py --dataset ../datasets/exploit_dataset/ +## Adapting the scripts -# Generate a bias and coverage report for all datasets -python bias_report.py --datasets ../datasets/ -``` +The scripts are plain Python with one runtime dependency (`jsonschema`) and are meant to be reused: -Refer to the docstring at the top of each script for detailed usage, configuration options, and output format. +- **Tune the heuristics.** `DEFAULT_THRESHOLD` in `dedup_checker.py`, `PII_PATTERNS` in `anonymization_scanner.py`, `CONCENTRATION` and `SKEW` in `anomaly_detector.py`. +- **Add a schema.** Give a new dataset a `schema.json` and have its entries point at it with `"$schema"`; `schema_validator.py` finds it. +- **Add a reference table.** Put the list in `reference_data/`, record its source in `SOURCES.md`, and check against it in `crossref_validator.py`. +- **Wire it into your pipeline.** `run_all_checks.py` exit codes are CI-friendly, and each validator's check functions (`validate_file`, `check_data`, `check_records`) can be imported and return a list of findings. --- ## Contributing -Improvements to the validation framework are welcome — new checks, better detection patterns, performance improvements, and additional schema definitions. Submit a pull request or discuss ideas in `#team-genai-data-security-initiative` on the [OWASP Slack workspace](https://owasp.slack.com). - -When contributing new validation scripts, include unit tests and update this README with the script's purpose and usage. +Improvements are welcome: new checks, better patterns, fewer false positives, a `validate.py` for a sub-collection that has none. Include tests in `tests/`, and update this README with what the check does and which findings are errors versus warnings. Discuss ideas in `#team-genai-data-security-initiative` on the [OWASP Slack workspace](https://owasp.slack.com). diff --git a/data_validation/SETUP.md b/data_validation/SETUP.md index e3fa3a6..a6e47a7 100644 --- a/data_validation/SETUP.md +++ b/data_validation/SETUP.md @@ -17,9 +17,9 @@ The data validation pipeline is a set of Python scripts that check contributed d ## Prerequisites -### Python 3.8+ +### Python 3.10+ -The scripts require Python 3.8 or higher. Most modern systems have Python pre-installed. +The scripts require Python 3.10 or higher. Most modern systems have Python pre-installed. **Check if you have it:** @@ -27,7 +27,7 @@ The scripts require Python 3.8 or higher. Most modern systems have Python pre-in python3 --version ``` -If you see `Python 3.8.x` or higher, you're good. If not: +If you see `Python 3.10.x` or higher, you're good. If not: | Platform | How to install | |---|---| @@ -95,7 +95,7 @@ You'll know it's active when your terminal prompt changes to show `(venv)` at th pip install -r requirements.txt ``` -> **What this does:** Reads the `requirements.txt` file and installs all the Python libraries the validation scripts need. This includes things like `jsonschema` for schema validation, `pandas` for data manipulation, and any other dependencies. +> **What this does:** Reads the `requirements.txt` file and installs all the Python libraries the validation scripts need. That is `jsonschema` (schema validation) and `pytest` (the tests), plus `pyyaml`, which only the reference-data refresh script uses. **If you get a permissions error:** Make sure your virtual environment is activated (Step 2). If you're not using a virtual environment, add `--user` to the command: `pip install --user -r requirements.txt` @@ -109,16 +109,16 @@ Run the built-in test suite to confirm everything is working: python -m pytest tests/ -v ``` -> **What this does:** Runs all unit tests for the validation scripts. You should see a series of green `PASSED` results. If any test fails, something went wrong in setup — check the error message and revisit the steps above. +> **What this does:** Runs the unit tests for the validation scripts. You should see a series of green `PASSED` results. If any test fails, something went wrong in setup — check the error message and revisit the steps above. -**Expected output (approximate):** +**Expected output (abridged):** ``` -tests/test_schema_validator.py::test_valid_vulnerability PASSED -tests/test_schema_validator.py::test_invalid_vulnerability PASSED -tests/test_dsgai_mapping_check.py::test_valid_mapping PASSED tests/test_anonymization_scanner.py::test_clean_entry PASSED tests/test_anonymization_scanner.py::test_pii_detected PASSED +tests/test_crossref_validator.py::test_unknown_cwe_and_atlas PASSED +tests/test_dsgai_mapping_check.py::test_invalid_dsgai_id PASSED +tests/test_schema_validator.py::test_valid_vulnerability PASSED ... ``` @@ -126,31 +126,36 @@ tests/test_anonymization_scanner.py::test_pii_detected PASSED ## Step 5 — Run Your First Validation -### Validate a single file +### Check a single file ```bash -python validators/schema_validator.py --file ../datasets/vulnerability_dataset/example_entry.json +python validators/schema_validator.py --file ../datasets/rag_dataset/entries/RAG-0001.json +python validators/dsgai_mapping_check.py --file ../datasets/rag_dataset/entries/RAG-0001.json ``` -### Run all checks on an entire dataset +Each script in `validators/` takes `--file` (one or more files) or `--dataset` (a folder), and checks every dataset when given neither. + +### Run all checks ```bash python run_all_checks.py # every dataset python run_all_checks.py --dataset ../datasets/exploit_dataset/ # one dataset ``` -Each dataset is checked by its own `validate.py`; datasets without one are listed as NO VALIDATOR. +This runs each dataset's own `validate.py`, then the shared checks in `validators/` on every data file. It is the same command CI runs on your pull request. Add `--verbose` to see every warning. ### Generate a coverage and bias report ```bash -python qc_tools/bias_report.py --datasets ../datasets/ +python qc_tools/bias_report.py --output coverage.md ``` > **What the output means:** -> - ✅ **PASS** — The check passed, no issues found -> - ⚠️ **WARN** — Something looks unusual but isn't blocking (e.g., a rarely used DSGAI mapping) -> - ❌ **FAIL** — A required check failed. The output will tell you which field, which file, and what's wrong +> - ✅ **PASS** — No issues found +> - ⚠️ **WARN** — Something a reviewer should look at, but not blocking (e.g., text that looks like an email address, or two very similar entries) +> - ❌ **FAIL** — A check failed. The lines under it name the file, the field, and what's wrong +> - **ERROR** — A dataset validator could not run (a missing dependency, or it timed out) +> - **NO VALIDATOR** — The dataset has no `validate.py` of its own yet; the shared checks still cover it --- @@ -164,7 +169,7 @@ Python might be installed as `python` instead of `python3` on your system (commo python --version ``` -If that shows 3.8+, use `python` everywhere this guide says `python3`. +If that shows 3.10+, use `python` everywhere this guide says `python3`. ### "No module named 'jsonschema'" (or any other module) @@ -189,7 +194,9 @@ This is expected — it means your data has an issue, not the tooling. Read the - **Missing required field** — Check the dataset's README for the expected schema - **Invalid DSGAI mapping** — Make sure the DSGAI ID exists (DSGAI01 through DSGAI21) -- **Anonymization failure** — Your entry may contain patterns that look like email addresses, IP addresses, or API keys. Replace them with synthetic values +- **Unknown identifier** — A CWE or MITRE ATLAS ID that doesn't exist, or a malformed CVE ID such as `CVE-24-1234` + +Anonymization findings are warnings, not failures, but reviewers will ask about them. If your entry contains something that looks like an email address, IP address or API key, use an obviously fake value instead: an `example.com` address, a documentation IP such as `192.0.2.10`, or a placeholder such as ``. --- @@ -202,21 +209,20 @@ GenAI-Data-Security-Initiative/ ├── README.md ← Project overview — start here ├── CONTRIBUTING.md ← How to contribute to any workstream ├── datasets/ ← The actual data (what gets validated) -│ ├── vulnerability_dataset/ +│ ├── _shared/dsgai_taxonomy.json ← The DSGAI IDs every check uses +│ ├── vulnerability_dataset/ ← schema.json, validate.py, entries/ │ ├── exploit_dataset/ -│ ├── incident_dataset/ │ └── ... ├── data_validation/ ← You are here │ ├── README.md ← What the validation framework does │ ├── SETUP.md ← This file │ ├── requirements.txt │ ├── run_all_checks.py -│ ├── schemas/ ← Defines what valid data looks like -│ ├── validators/ ← Automated checks (run on every PR) +│ ├── schemas/ ← Schemas for datasets without their own schema.json +│ ├── validators/ ← Shared automated checks (run on every PR) │ ├── qc_tools/ ← Reports for human reviewers -│ ├── reference_data/ ← Lookup tables (DSGAI IDs, CWEs, etc.) +│ ├── reference_data/ ← Lookup tables (DSGAI, MITRE ATLAS, CWE) │ └── tests/ ← Tests for the validators themselves -├── mappings/ ← Framework mapping files └── literature/ ← Reference materials ``` @@ -230,10 +236,10 @@ Now that your environment is set up, here's what you can do: Read the README in the specific dataset folder you're interested in (e.g., `datasets/incident_dataset/README.md`). It describes the expected format and contribution guidelines. Create your entry, run the validators locally, and submit a pull request. **If you want to improve the validators:** -Look at the `validators/` and `qc_tools/` directories. Each script has a docstring at the top explaining what it does. Pick an issue from the repo or propose a new check. Add unit tests in `tests/` for any new logic. +Look at the `validators/` and `qc_tools/` directories. Each script has a docstring at the top explaining what it checks and which findings are errors versus warnings. A useful first contribution is a `validate.py` for a promptinj sub-collection that has none yet (`tr_altaysec_turkish_llm_injection`). Add unit tests in `tests/` for any new logic. -**If you want to add a new framework mapping:** -Add a reference data file in `reference_data/framework_control_ids/` and update `crossref_validator.py` to support it. +**If you want to check a new kind of identifier:** +Add the lookup table to `reference_data/`, record where it came from in `reference_data/SOURCES.md`, and check against it in `crossref_validator.py`. **If you have questions:** Join `#team-genai-data-security-initiative` on the [OWASP Slack workspace](https://owasp.slack.com) ([join here](https://owasp.org/slack/invite) if you're new). No question is too basic. @@ -245,7 +251,7 @@ Join `#team-genai-data-security-initiative` on the [OWASP Slack workspace](https These tools are designed to be used in educational settings. If you're running a workshop, training session, or university course on AI security: - The `tests/fixtures/` directory contains deliberately valid and invalid sample entries — useful for hands-on exercises -- `bias_report.py` generates visual coverage reports that make good discussion starters for data quality conversations +- `bias_report.py` generates a coverage report (which DSGAI risks the data covers, and where it is thin) that makes a good discussion starter for data quality conversations - The validation pipeline itself demonstrates applied data governance concepts from the DSGAI risk taxonomy (DSGAI05: Data Integrity & Validation Failures, DSGAI07: Data Governance, Lifecycle & Classification) - Students can contribute real data back to the initiative — a practical way to engage with open-source security research diff --git a/data_validation/qc_tools/anomaly_detector.py b/data_validation/qc_tools/anomaly_detector.py index 8824cd8..29d73c8 100644 --- a/data_validation/qc_tools/anomaly_detector.py +++ b/data_validation/qc_tools/anomaly_detector.py @@ -1,17 +1,93 @@ """ -Identifies statistical outliers in severity distributions, DSGAI mapping -concentrations, and submission patterns. +Flags statistical outliers for a reviewer to look at: + + * a DSGAI entry that more than CONCENTRATION of a dataset's records map to + * a record that maps to far more DSGAI entries than its dataset's typical + record (more than mean + 2 standard deviations, more than twice the mean, + and more than 3) + * a severity level, or a value of a categorical field, that covers more + than SKEW of a dataset's records + +Datasets with fewer than MIN_RECORDS records are skipped: percentages of a +handful of records say nothing. Findings are WARN only; the exit code is 0 +unless the arguments are wrong. Usage: + python anomaly_detector.py python anomaly_detector.py --datasets ../../datasets/ """ +from __future__ import annotations + import argparse +import statistics +import sys +from collections import Counter +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from qc_tools.bias_report import CATEGORY_FIELDS, severity_of # noqa: E402 +from validators._common import ( # noqa: E402 + DATASETS_ROOT, WARN, Finding, ensure_utf8_stdout, load_dataset_records, print_findings, record_id, +) +from validators.dsgai_mapping_check import record_dsgai_ids # noqa: E402 + +CHECK = "anomaly" +MIN_RECORDS = 10 +CONCENTRATION = 0.6 +SKEW = 0.85 + -def main(): +def detect(datasets_root: Path = DATASETS_ROOT) -> list[Finding]: + findings: list[Finding] = [] + for name, items in load_dataset_records(datasets_root).items(): + if len(items) < MIN_RECORDS: + continue + ds_path = datasets_root / name + total = len(items) + mappings = [record_dsgai_ids(r) for _p, r in items] + + for did, n in Counter(i for m in mappings for i in m).most_common(): + if n / total > CONCENTRATION: + findings.append(Finding(WARN, CHECK, ds_path, "", + f"{n / total:.0%} of {total} records map to {did}; check it is not a default mapping")) + + sizes = [len(m) for m in mappings] + if len(set(sizes)) > 1: + mean = statistics.mean(sizes) + limit = max(mean + 2 * statistics.stdev(sizes), 2 * mean, 3) + for (path, rec), size in zip(items, sizes): + if size > limit: + findings.append(Finding(WARN, CHECK, path, "", + f"{record_id(rec) or path.name} maps to {size} DSGAI entries " + f"(dataset mean {mean:.1f})")) + + fields = {"severity": Counter(s for _p, r in items if (s := severity_of(r)))} + for field in CATEGORY_FIELDS: + fields[field] = Counter(r[field] for _p, r in items if isinstance(r.get(field), str)) + for field, counts in fields.items(): + if not counts or len(counts) == 1 and field != "severity": + continue # a field with one value everywhere is a constant, not a skew + value, n = counts.most_common(1)[0] + if n / total > SKEW: + findings.append(Finding(WARN, CHECK, ds_path, "", + f"{field} is {value!r} in {n / total:.0%} of {total} records")) + return findings + + +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() parser = argparse.ArgumentParser(description="Detect anomalies in dataset distributions") - parser.add_argument("--datasets", required=True, help="Path to root datasets directory") - args = parser.parse_args() - print("Anomaly detector stub - contributions welcome!") + parser.add_argument("--datasets", default=str(DATASETS_ROOT), help="Root datasets directory") + args = parser.parse_args(argv) + root = Path(args.datasets) + if not root.is_dir(): + print(f"Directory not found: {args.datasets}") + return 2 + print_findings(detect(root.resolve()), root.resolve().parent) + return 0 + if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/qc_tools/bias_report.py b/data_validation/qc_tools/bias_report.py index a245735..407ce00 100644 --- a/data_validation/qc_tools/bias_report.py +++ b/data_validation/qc_tools/bias_report.py @@ -1,18 +1,107 @@ """ -Generates coverage reports showing dataset distribution across DSGAI entries, -industries, attack vectors, and frameworks to surface gaps and overrepresentation. +Generates a Markdown coverage report: how records spread across DSGAI +entries, severity, and each dataset's main categorical fields. It shows gaps +(DSGAI entries no record maps to) and overrepresentation; it does not judge +whether either is a problem. Usage: + python bias_report.py # print to stdout + python bias_report.py --output bias_report.md python bias_report.py --datasets ../../datasets/ """ +from __future__ import annotations + import argparse +import json +import sys +from collections import Counter +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import DATASETS_ROOT, ensure_utf8_stdout, load_dataset_records # noqa: E402 +from validators.dsgai_mapping_check import record_dsgai_ids # noqa: E402 + +# Top-level fields reported per dataset when present. +CATEGORY_FIELDS = ( + "category", "affected_component", "target_component", "industry_sector", + "deployment_pattern", "entry_type", "disclosure_status", "disposition", +) + + +def taxonomy(datasets_root: Path) -> list[tuple[str, str]]: + data = json.loads((datasets_root / "_shared" / "dsgai_taxonomy.json").read_text(encoding="utf-8")) + return [(e["id"], e["name"]) for e in data["entries"]] + + +def severity_of(record: dict) -> str | None: + sev = record.get("severity") + if isinstance(sev, dict): + sev = sev.get("qualitative") + return sev if isinstance(sev, str) else None + + +def _pct(n: int, total: int) -> str: + return f"{n} ({n / total:.0%})" if total else "0" + + +def build_report(datasets_root: Path = DATASETS_ROOT) -> str: + records = load_dataset_records(datasets_root) + tax = taxonomy(datasets_root) + names = [n for n, recs in records.items() if recs] + lines = ["# Dataset coverage report", ""] + lines += ["| Dataset | Records |", "|---|---|"] + lines += [f"| `{n}` | {len(recs)} |" for n, recs in records.items()] + + coverage = {n: Counter(i for _p, r in records[n] for i in record_dsgai_ids(r)) for n in names} + lines += ["", "## DSGAI coverage", "", + "Records that map to each DSGAI entry. A record can map to several.", ""] + lines.append("| DSGAI | Name | " + " | ".join(f"`{n}`" for n in names) + " | Total |") + lines.append("|---|---|" + "---|" * (len(names) + 1)) + for did, dname in tax: + counts = [coverage[n][did] for n in names] + lines.append(f"| {did} | {dname} | " + " | ".join(str(c) for c in counts) + f" | {sum(counts)} |") + uncovered = [did for did, _ in tax if not any(coverage[n][did] for n in names)] + lines += ["", f"DSGAI entries no record maps to: {', '.join(uncovered) if uncovered else 'none'}."] + + for n in names: + recs = [r for _p, r in records[n]] + total = len(recs) + sections = [] + sev = Counter(s for r in recs if (s := severity_of(r))) + if sev: + sections.append(("severity", sev)) + for field in CATEGORY_FIELDS: + c = Counter(r[field] for r in recs if isinstance(r.get(field), str)) + if c: + sections.append((field, c)) + if not sections: + continue + lines += ["", f"## `{n}`"] + for field, c in sections: + lines += ["", f"**{field}**: " + ", ".join(f"{k} {_pct(v, total)}" for k, v in c.most_common())] + return "\n".join(lines).rstrip() + "\n" + + +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Generate a dataset coverage and bias report") + parser.add_argument("--datasets", default=str(DATASETS_ROOT), help="Root datasets directory") + parser.add_argument("--output", help="Write the report here instead of stdout") + args = parser.parse_args(argv) + root = Path(args.datasets) + if not root.is_dir(): + print(f"Directory not found: {args.datasets}") + return 2 + report = build_report(root.resolve()) + if args.output: + Path(args.output).write_text(report, encoding="utf-8") + print(f"Wrote {args.output}") + else: + print(report, end="") + return 0 -def main(): - parser = argparse.ArgumentParser(description="Generate dataset coverage and bias reports") - parser.add_argument("--datasets", required=True, help="Path to root datasets directory") - parser.add_argument("--output", default="bias_report.md", help="Output file path") - args = parser.parse_args() - print("Bias report generator stub - contributions welcome!") if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/qc_tools/consistency_check.py b/data_validation/qc_tools/consistency_check.py index 59b0f89..5879607 100644 --- a/data_validation/qc_tools/consistency_check.py +++ b/data_validation/qc_tools/consistency_check.py @@ -1,17 +1,81 @@ """ -Cross-validates related entries across datasets. For example, an exploit entry -should reference an existing vulnerability entry if both describe the same issue. +Cross-validates references between datasets. + +ERROR: a record cites one of the initiative's own IDs (DSGAI-VULN-..., + DSGAI-EXP-..., DSGAI-RA-...) that no record in the matching dataset + has. These are internal links, so a dangling one is always a mistake: + a typo, or a renamed or removed entry. + +Also printed, as a note rather than a finding: how many CVE IDs cited by +exploit entries have no vulnerability entry of their own. The vulnerability +dataset is not meant to hold every CVE, so this is a list of candidates for +new entries, not an error. Usage: + python consistency_check.py python consistency_check.py --datasets ../../datasets/ """ +from __future__ import annotations + import argparse +import re +import sys +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import ( # noqa: E402 + DATASETS_ROOT, ERROR, Finding, ensure_utf8_stdout, load_dataset_records, + print_findings, record_id, walk_strings, +) + +CHECK = "consistency" +INTERNAL_ID_RE = re.compile(r"DSGAI-(VULN|EXP|RA)-[A-Za-z0-9._-]+") +TARGET_DATASET = {"VULN": "vulnerability_dataset", "EXP": "exploit_dataset", "RA": "riskassessment_dataset"} +CVE_RE = re.compile(r"CVE-\d{4}-\d{4,}") + + +def check(datasets_root: Path = DATASETS_ROOT) -> tuple[list[Finding], list[str]]: + """(findings, CVE IDs cited by exploits that have no vulnerability entry).""" + records = load_dataset_records(datasets_root) + ids = {name: {record_id(r) for _p, r in items} for name, items in records.items()} + findings: list[Finding] = [] + for name, items in records.items(): + for path, rec in items: + own = record_id(rec) + for pointer, _key, value in walk_strings(rec): + m = INTERNAL_ID_RE.fullmatch(value.strip()) + if not m or value.strip() == own: + continue + target = TARGET_DATASET[m.group(1)] + if value.strip() not in ids.get(target, set()): + findings.append(Finding(ERROR, CHECK, path, pointer, + f"{value.strip()} is not an entry in {target}")) + vuln_ids = ids.get("vulnerability_dataset", set()) + missing = sorted({ + v.strip() + for _p, rec in records.get("exploit_dataset", []) + for _ptr, key, v in walk_strings(rec) + if key == "related_vulnerabilities" and CVE_RE.fullmatch(v.strip()) and v.strip() not in vuln_ids + }) + return findings, missing + + +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Check cross-dataset references") + parser.add_argument("--datasets", default=str(DATASETS_ROOT), help="Root datasets directory") + args = parser.parse_args(argv) + root = Path(args.datasets) + if not root.is_dir(): + print(f"Directory not found: {args.datasets}") + return 2 + findings, missing = check(root.resolve()) + if missing: + print(f"Note: {len(missing)} CVE(s) cited by exploit entries have no vulnerability entry: {', '.join(missing)}") + return print_findings(findings, root.resolve().parent) -def main(): - parser = argparse.ArgumentParser(description="Check cross-dataset consistency") - parser.add_argument("--datasets", required=True, help="Path to root datasets directory") - args = parser.parse_args() - print("Consistency checker stub - contributions welcome!") if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/reference_data/SOURCES.md b/data_validation/reference_data/SOURCES.md new file mode 100644 index 0000000..d095de6 --- /dev/null +++ b/data_validation/reference_data/SOURCES.md @@ -0,0 +1,15 @@ +# Reference data sources + +Lookup tables the validators check identifiers against. + +| Table | File | Version | Source | +|---|---|---|---| +| DSGAI | `dsgai_entries.json` | 1.0 | OWASP GenAI Data Security Risks and Mitigations 2026 (kept identical to `datasets/_shared/dsgai_taxonomy.json`; a test enforces it) | +| MITRE ATLAS | `mitre_atlas_techniques.csv` | 2026.09 | latest release listed in `https://raw.githubusercontent.com/mitre-atlas/atlas-data/main/dist/manifest.yaml` | +| CWE | `cwe_ids.csv` | 4.20 | `https://cwe.mitre.org/data/xml/cwec_latest.xml.zip` (weaknesses, categories and views) | + +Regenerate the ATLAS and CWE tables with `python update_reference_data.py`; it fills in the Version column. Do not edit those two CSVs by hand. + +`datasets/_shared/dsgai_taxonomy.json` is the DSGAI list the validators use. `dsgai_entries.json` is kept because the Turkish prompt-injection validator reads it. + +Framework control lists (ISO/IEC 27001, ISO/IEC 42001, CIS Controls, NIST) are not included. Complete lists are needed for a check to be useful, and the ISO and CIS texts are not freely redistributable, so framework control IDs are left to reviewers. diff --git a/data_validation/reference_data/cwe_ids.csv b/data_validation/reference_data/cwe_ids.csv index 35d786c..68f8622 100644 --- a/data_validation/reference_data/cwe_ids.csv +++ b/data_validation/reference_data/cwe_ids.csv @@ -1,12 +1,1451 @@ -cwe_id,name,description -CWE-20,Improper Input Validation,Product does not validate or incorrectly validates input -CWE-79,Cross-site Scripting (XSS),Improper neutralization of input during web page generation -CWE-89,SQL Injection,Improper neutralization of special elements used in SQL commands -CWE-200,Exposure of Sensitive Information,Intentional or unintentional disclosure of information -CWE-285,Improper Authorization,Software does not perform authorization checks -CWE-311,Missing Encryption of Sensitive Data,Failure to encrypt sensitive data -CWE-327,Use of Broken Crypto Algorithm,Use of a risky cryptographic algorithm -CWE-352,Cross-Site Request Forgery,Web application does not verify request was intentionally provided -CWE-522,Insufficiently Protected Credentials,Transmitting or storing credentials in an unsafe manner -CWE-732,Incorrect Permission Assignment,Permissions assigned to resource are incorrect -CWE-918,Server-Side Request Forgery,Web application fetches a remote resource without validating the URL +cwe_id,name,type,status +CWE-1,DEPRECATED: Location,category,Deprecated +CWE-2,7PK - Environment,category,Draft +CWE-3,DEPRECATED: Technology-specific Environment Issues,category,Deprecated +CWE-4,DEPRECATED: J2EE Environment Issues,category,Deprecated +CWE-5,J2EE Misconfiguration: Data Transmission Without Encryption,weakness,Draft +CWE-6,J2EE Misconfiguration: Insufficient Session-ID Length,weakness,Incomplete +CWE-7,J2EE Misconfiguration: Missing Custom Error Page,weakness,Incomplete +CWE-8,J2EE Misconfiguration: Entity Bean Declared Remote,weakness,Incomplete +CWE-9,J2EE Misconfiguration: Weak Access Permissions for EJB Methods,weakness,Draft +CWE-10,DEPRECATED: ASP.NET Environment Issues,category,Deprecated +CWE-11,ASP.NET Misconfiguration: Creating Debug Binary,weakness,Draft +CWE-12,ASP.NET Misconfiguration: Missing Custom Error Page,weakness,Draft +CWE-13,ASP.NET Misconfiguration: Password in Configuration File,weakness,Draft +CWE-14,Compiler Removal of Code to Clear Buffers,weakness,Draft +CWE-15,External Control of System or Configuration Setting,weakness,Incomplete +CWE-16,Configuration,category,Obsolete +CWE-17,DEPRECATED: Code,category,Deprecated +CWE-18,DEPRECATED: Source Code,category,Deprecated +CWE-19,Data Processing Errors,category,Draft +CWE-20,Improper Input Validation,weakness,Stable +CWE-21,DEPRECATED: Pathname Traversal and Equivalence Errors,category,Deprecated +CWE-22,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),weakness,Stable +CWE-23,Relative Path Traversal,weakness,Draft +CWE-24,Path Traversal: '../filedir',weakness,Incomplete +CWE-25,Path Traversal: '/../filedir',weakness,Incomplete +CWE-26,Path Traversal: '/dir/../filename',weakness,Draft +CWE-27,Path Traversal: 'dir/../../filename',weakness,Draft +CWE-28,Path Traversal: '..\filedir',weakness,Incomplete +CWE-29,Path Traversal: '\..\filename',weakness,Incomplete +CWE-30,Path Traversal: '\dir\..\filename',weakness,Draft +CWE-31,Path Traversal: 'dir\..\..\filename',weakness,Draft +CWE-32,Path Traversal: '...' (Triple Dot),weakness,Incomplete +CWE-33,Path Traversal: '....' (Multiple Dot),weakness,Incomplete +CWE-34,Path Traversal: '....//',weakness,Incomplete +CWE-35,Path Traversal: '.../...//',weakness,Incomplete +CWE-36,Absolute Path Traversal,weakness,Draft +CWE-37,Path Traversal: '/absolute/pathname/here',weakness,Draft +CWE-38,Path Traversal: '\absolute\pathname\here',weakness,Draft +CWE-39,Path Traversal: 'C:dirname',weakness,Draft +CWE-40,Path Traversal: '\\UNC\share\name\' (Windows UNC Share),weakness,Draft +CWE-41,Improper Resolution of Path Equivalence,weakness,Incomplete +CWE-42,Path Equivalence: 'filename.' (Trailing Dot),weakness,Incomplete +CWE-43,Path Equivalence: 'filename....' (Multiple Trailing Dot),weakness,Incomplete +CWE-44,Path Equivalence: 'file.name' (Internal Dot),weakness,Incomplete +CWE-45,Path Equivalence: 'file...name' (Multiple Internal Dot),weakness,Incomplete +CWE-46,Path Equivalence: 'filename ' (Trailing Space),weakness,Incomplete +CWE-47,Path Equivalence: ' filename' (Leading Space),weakness,Incomplete +CWE-48,Path Equivalence: 'file name' (Internal Whitespace),weakness,Incomplete +CWE-49,Path Equivalence: 'filename/' (Trailing Slash),weakness,Incomplete +CWE-50,Path Equivalence: '//multiple/leading/slash',weakness,Incomplete +CWE-51,Path Equivalence: '/multiple//internal/slash',weakness,Incomplete +CWE-52,Path Equivalence: '/multiple/trailing/slash//',weakness,Incomplete +CWE-53,Path Equivalence: '\multiple\\internal\backslash',weakness,Incomplete +CWE-54,Path Equivalence: 'filedir\' (Trailing Backslash),weakness,Incomplete +CWE-55,Path Equivalence: '/./' (Single Dot Directory),weakness,Incomplete +CWE-56,Path Equivalence: 'filedir*' (Wildcard),weakness,Incomplete +CWE-57,Path Equivalence: 'fakedir/../realdir/filename',weakness,Incomplete +CWE-58,Path Equivalence: Windows 8.3 Filename,weakness,Incomplete +CWE-59,Improper Link Resolution Before File Access ('Link Following'),weakness,Draft +CWE-60,DEPRECATED: UNIX Path Link Problems,category,Deprecated +CWE-61,UNIX Symbolic Link (Symlink) Following,weakness,Incomplete +CWE-62,UNIX Hard Link,weakness,Incomplete +CWE-63,DEPRECATED: Windows Path Link Problems,category,Deprecated +CWE-64,Windows Shortcut Following (.LNK),weakness,Incomplete +CWE-65,Windows Hard Link,weakness,Incomplete +CWE-66,Improper Handling of File Names that Identify Virtual Resources,weakness,Draft +CWE-67,Improper Handling of Windows Device Names,weakness,Incomplete +CWE-68,DEPRECATED: Windows Virtual File Problems,category,Deprecated +CWE-69,Improper Handling of Windows ::DATA Alternate Data Stream,weakness,Incomplete +CWE-70,DEPRECATED: Mac Virtual File Problems,category,Deprecated +CWE-71,DEPRECATED: Apple '.DS_Store',weakness,Deprecated +CWE-72,Improper Handling of Apple HFS+ Alternate Data Stream Path,weakness,Incomplete +CWE-73,External Control of File Name or Path,weakness,Draft +CWE-74,Improper Neutralization of Special Elements in Output Used by a Downstream Component ('Injection'),weakness,Incomplete +CWE-75,Failure to Sanitize Special Elements into a Different Plane (Special Element Injection),weakness,Draft +CWE-76,Improper Neutralization of Equivalent Special Elements,weakness,Draft +CWE-77,Improper Neutralization of Special Elements used in a Command ('Command Injection'),weakness,Draft +CWE-78,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),weakness,Stable +CWE-79,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),weakness,Stable +CWE-80,Improper Neutralization of Script-Related HTML Tags in a Web Page (Basic XSS),weakness,Incomplete +CWE-81,Improper Neutralization of Script in an Error Message Web Page,weakness,Incomplete +CWE-82,Improper Neutralization of Script in Attributes of IMG Tags in a Web Page,weakness,Incomplete +CWE-83,Improper Neutralization of Script in Attributes in a Web Page,weakness,Draft +CWE-84,Improper Neutralization of Encoded URI Schemes in a Web Page,weakness,Draft +CWE-85,Doubled Character XSS Manipulations,weakness,Draft +CWE-86,Improper Neutralization of Invalid Characters in Identifiers in Web Pages,weakness,Draft +CWE-87,Improper Neutralization of Alternate XSS Syntax,weakness,Draft +CWE-88,Improper Neutralization of Argument Delimiters in a Command ('Argument Injection'),weakness,Draft +CWE-89,Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection'),weakness,Stable +CWE-90,Improper Neutralization of Special Elements used in an LDAP Query ('LDAP Injection'),weakness,Draft +CWE-91,XML Injection (aka Blind XPath Injection),weakness,Draft +CWE-92,DEPRECATED: Improper Sanitization of Custom Special Characters,weakness,Deprecated +CWE-93,Improper Neutralization of CRLF Sequences ('CRLF Injection'),weakness,Draft +CWE-94,Improper Control of Generation of Code ('Code Injection'),weakness,Draft +CWE-95,Improper Neutralization of Directives in Dynamically Evaluated Code ('Eval Injection'),weakness,Incomplete +CWE-96,Improper Neutralization of Directives in Statically Saved Code ('Static Code Injection'),weakness,Draft +CWE-97,Improper Neutralization of Server-Side Includes (SSI) Within a Web Page,weakness,Draft +CWE-98,Improper Control of Filename for Include/Require Statement in PHP Program ('PHP Remote File Inclusion'),weakness,Draft +CWE-99,Improper Control of Resource Identifiers ('Resource Injection'),weakness,Draft +CWE-100,DEPRECATED: Technology-Specific Input Validation Problems,category,Deprecated +CWE-101,DEPRECATED: Struts Validation Problems,category,Deprecated +CWE-102,Struts: Duplicate Validation Forms,weakness,Incomplete +CWE-103,Struts: Incomplete validate() Method Definition,weakness,Draft +CWE-104,Struts: Form Bean Does Not Extend Validation Class,weakness,Draft +CWE-105,Struts: Form Field Without Validator,weakness,Draft +CWE-106,Struts: Plug-in Framework not in Use,weakness,Draft +CWE-107,Struts: Unused Validation Form,weakness,Draft +CWE-108,Struts: Unvalidated Action Form,weakness,Incomplete +CWE-109,Struts: Validator Turned Off,weakness,Draft +CWE-110,Struts: Validator Without Form Field,weakness,Draft +CWE-111,Direct Use of Unsafe JNI,weakness,Draft +CWE-112,Missing XML Validation,weakness,Draft +CWE-113,Improper Neutralization of CRLF Sequences in HTTP Headers ('HTTP Request/Response Splitting'),weakness,Incomplete +CWE-114,Process Control,weakness,Incomplete +CWE-115,Misinterpretation of Input,weakness,Incomplete +CWE-116,Improper Encoding or Escaping of Output,weakness,Draft +CWE-117,Improper Output Neutralization for Logs,weakness,Draft +CWE-118,Incorrect Access of Indexable Resource ('Range Error'),weakness,Incomplete +CWE-119,Improper Restriction of Operations within the Bounds of a Memory Buffer,weakness,Stable +CWE-120,Buffer Copy without Checking Size of Input ('Classic Buffer Overflow'),weakness,Incomplete +CWE-121,Stack-based Buffer Overflow,weakness,Draft +CWE-122,Heap-based Buffer Overflow,weakness,Draft +CWE-123,Write-what-where Condition,weakness,Draft +CWE-124,Buffer Underwrite ('Buffer Underflow'),weakness,Incomplete +CWE-125,Out-of-bounds Read,weakness,Draft +CWE-126,Buffer Over-read,weakness,Draft +CWE-127,Buffer Under-read,weakness,Draft +CWE-128,Wrap-around Error,weakness,Incomplete +CWE-129,Improper Validation of Array Index,weakness,Draft +CWE-130,Improper Handling of Length Parameter Inconsistency,weakness,Incomplete +CWE-131,Incorrect Calculation of Buffer Size,weakness,Draft +CWE-132,DEPRECATED: Miscalculated Null Termination,weakness,Deprecated +CWE-133,String Errors,category,Draft +CWE-134,Use of Externally-Controlled Format String,weakness,Draft +CWE-135,Incorrect Calculation of Multi-Byte String Length,weakness,Draft +CWE-136,Type Errors,category,Draft +CWE-137,Data Neutralization Issues,category,Draft +CWE-138,Improper Neutralization of Special Elements,weakness,Draft +CWE-139,DEPRECATED: General Special Element Problems,category,Deprecated +CWE-140,Improper Neutralization of Delimiters,weakness,Draft +CWE-141,Improper Neutralization of Parameter/Argument Delimiters,weakness,Draft +CWE-142,Improper Neutralization of Value Delimiters,weakness,Draft +CWE-143,Improper Neutralization of Record Delimiters,weakness,Draft +CWE-144,Improper Neutralization of Line Delimiters,weakness,Draft +CWE-145,Improper Neutralization of Section Delimiters,weakness,Incomplete +CWE-146,Improper Neutralization of Expression/Command Delimiters,weakness,Incomplete +CWE-147,Improper Neutralization of Input Terminators,weakness,Draft +CWE-148,Improper Neutralization of Input Leaders,weakness,Draft +CWE-149,Improper Neutralization of Quoting Syntax,weakness,Draft +CWE-150,"Improper Neutralization of Escape, Meta, or Control Sequences",weakness,Incomplete +CWE-151,Improper Neutralization of Comment Delimiters,weakness,Draft +CWE-152,Improper Neutralization of Macro Symbols,weakness,Draft +CWE-153,Improper Neutralization of Substitution Characters,weakness,Draft +CWE-154,Improper Neutralization of Variable Name Delimiters,weakness,Incomplete +CWE-155,Improper Neutralization of Wildcards or Matching Symbols,weakness,Draft +CWE-156,Improper Neutralization of Whitespace,weakness,Draft +CWE-157,Failure to Sanitize Paired Delimiters,weakness,Draft +CWE-158,Improper Neutralization of Null Byte or NUL Character,weakness,Incomplete +CWE-159,Improper Handling of Invalid Use of Special Elements,weakness,Draft +CWE-160,Improper Neutralization of Leading Special Elements,weakness,Incomplete +CWE-161,Improper Neutralization of Multiple Leading Special Elements,weakness,Incomplete +CWE-162,Improper Neutralization of Trailing Special Elements,weakness,Incomplete +CWE-163,Improper Neutralization of Multiple Trailing Special Elements,weakness,Incomplete +CWE-164,Improper Neutralization of Internal Special Elements,weakness,Incomplete +CWE-165,Improper Neutralization of Multiple Internal Special Elements,weakness,Incomplete +CWE-166,Improper Handling of Missing Special Element,weakness,Draft +CWE-167,Improper Handling of Additional Special Element,weakness,Draft +CWE-168,Improper Handling of Inconsistent Special Elements,weakness,Draft +CWE-169,DEPRECATED: Technology-Specific Special Elements,category,Deprecated +CWE-170,Improper Null Termination,weakness,Incomplete +CWE-171,"DEPRECATED: Cleansing, Canonicalization, and Comparison Errors",category,Deprecated +CWE-172,Encoding Error,weakness,Draft +CWE-173,Improper Handling of Alternate Encoding,weakness,Draft +CWE-174,Double Decoding of the Same Data,weakness,Draft +CWE-175,Improper Handling of Mixed Encoding,weakness,Draft +CWE-176,Improper Handling of Unicode Encoding,weakness,Draft +CWE-177,Improper Handling of URL Encoding (Hex Encoding),weakness,Draft +CWE-178,Improper Handling of Case Sensitivity,weakness,Incomplete +CWE-179,Incorrect Behavior Order: Early Validation,weakness,Incomplete +CWE-180,Incorrect Behavior Order: Validate Before Canonicalize,weakness,Draft +CWE-181,Incorrect Behavior Order: Validate Before Filter,weakness,Draft +CWE-182,Collapse of Data into Unsafe Value,weakness,Draft +CWE-183,Permissive List of Allowed Inputs,weakness,Draft +CWE-184,Incomplete List of Disallowed Inputs,weakness,Draft +CWE-185,Incorrect Regular Expression,weakness,Draft +CWE-186,Overly Restrictive Regular Expression,weakness,Draft +CWE-187,Partial String Comparison,weakness,Incomplete +CWE-188,Reliance on Data/Memory Layout,weakness,Draft +CWE-189,Numeric Errors,category,Draft +CWE-190,Integer Overflow or Wraparound,weakness,Stable +CWE-191,Integer Underflow (Wrap or Wraparound),weakness,Draft +CWE-192,Integer Coercion Error,weakness,Incomplete +CWE-193,Off-by-one Error,weakness,Draft +CWE-194,Unexpected Sign Extension,weakness,Incomplete +CWE-195,Signed to Unsigned Conversion Error,weakness,Draft +CWE-196,Unsigned to Signed Conversion Error,weakness,Draft +CWE-197,Numeric Truncation Error,weakness,Incomplete +CWE-198,Use of Incorrect Byte Ordering,weakness,Draft +CWE-199,Information Management Errors,category,Draft +CWE-200,Exposure of Sensitive Information to an Unauthorized Actor,weakness,Draft +CWE-201,Insertion of Sensitive Information Into Sent Data,weakness,Draft +CWE-202,Exposure of Sensitive Information Through Data Queries,weakness,Draft +CWE-203,Observable Discrepancy,weakness,Incomplete +CWE-204,Observable Response Discrepancy,weakness,Incomplete +CWE-205,Observable Behavioral Discrepancy,weakness,Incomplete +CWE-206,Observable Internal Behavioral Discrepancy,weakness,Incomplete +CWE-207,Observable Behavioral Discrepancy With Equivalent Products,weakness,Draft +CWE-208,Observable Timing Discrepancy,weakness,Incomplete +CWE-209,Generation of Error Message Containing Sensitive Information,weakness,Draft +CWE-210,Self-generated Error Message Containing Sensitive Information,weakness,Draft +CWE-211,Externally-Generated Error Message Containing Sensitive Information,weakness,Incomplete +CWE-212,Improper Removal of Sensitive Information Before Storage or Transfer,weakness,Incomplete +CWE-213,Exposure of Sensitive Information Due to Incompatible Policies,weakness,Draft +CWE-214,Invocation of Process Using Visible Sensitive Information,weakness,Incomplete +CWE-215,Insertion of Sensitive Information Into Debugging Code,weakness,Draft +CWE-216,DEPRECATED: Containment Errors (Container Errors),weakness,Deprecated +CWE-217,DEPRECATED: Failure to Protect Stored Data from Modification,weakness,Deprecated +CWE-218,DEPRECATED: Failure to provide confidentiality for stored data,weakness,Deprecated +CWE-219,Storage of File with Sensitive Data Under Web Root,weakness,Draft +CWE-220,Storage of File With Sensitive Data Under FTP Root,weakness,Draft +CWE-221,Information Loss or Omission,weakness,Incomplete +CWE-222,Truncation of Security-relevant Information,weakness,Draft +CWE-223,Omission of Security-relevant Information,weakness,Draft +CWE-224,Obscured Security-relevant Information by Alternate Name,weakness,Incomplete +CWE-225,DEPRECATED: General Information Management Problems,weakness,Deprecated +CWE-226,Sensitive Information in Resource Not Removed Before Reuse,weakness,Draft +CWE-227,7PK - API Abuse,category,Draft +CWE-228,Improper Handling of Syntactically Invalid Structure,weakness,Incomplete +CWE-229,Improper Handling of Values,weakness,Incomplete +CWE-230,Improper Handling of Missing Values,weakness,Draft +CWE-231,Improper Handling of Extra Values,weakness,Draft +CWE-232,Improper Handling of Undefined Values,weakness,Draft +CWE-233,Improper Handling of Parameters,weakness,Incomplete +CWE-234,Failure to Handle Missing Parameter,weakness,Incomplete +CWE-235,Improper Handling of Extra Parameters,weakness,Draft +CWE-236,Improper Handling of Undefined Parameters,weakness,Draft +CWE-237,Improper Handling of Structural Elements,weakness,Incomplete +CWE-238,Improper Handling of Incomplete Structural Elements,weakness,Draft +CWE-239,Failure to Handle Incomplete Element,weakness,Draft +CWE-240,Improper Handling of Inconsistent Structural Elements,weakness,Draft +CWE-241,Improper Handling of Unexpected Data Type,weakness,Draft +CWE-242,Use of Inherently Dangerous Function,weakness,Draft +CWE-243,Creation of chroot Jail Without Changing Working Directory,weakness,Draft +CWE-244,Improper Clearing of Heap Memory Before Release ('Heap Inspection'),weakness,Draft +CWE-245,J2EE Bad Practices: Direct Management of Connections,weakness,Draft +CWE-246,J2EE Bad Practices: Direct Use of Sockets,weakness,Draft +CWE-247,DEPRECATED: Reliance on DNS Lookups in a Security Decision,weakness,Deprecated +CWE-248,Uncaught Exception,weakness,Draft +CWE-249,DEPRECATED: Often Misused: Path Manipulation,weakness,Deprecated +CWE-250,Execution with Unnecessary Privileges,weakness,Draft +CWE-251,Often Misused: String Management,category,Incomplete +CWE-252,Unchecked Return Value,weakness,Draft +CWE-253,Incorrect Check of Function Return Value,weakness,Incomplete +CWE-254,7PK - Security Features,category,Incomplete +CWE-255,Credentials Management Errors,category,Draft +CWE-256,Plaintext Storage of a Password,weakness,Incomplete +CWE-257,Storing Passwords in a Recoverable Format,weakness,Incomplete +CWE-258,Empty Password in Configuration File,weakness,Incomplete +CWE-259,Use of Hard-coded Password,weakness,Draft +CWE-260,Password in Configuration File,weakness,Incomplete +CWE-261,Weak Encoding for Password,weakness,Incomplete +CWE-262,Not Using Password Aging,weakness,Draft +CWE-263,Password Aging with Long Expiration,weakness,Draft +CWE-264,"Permissions, Privileges, and Access Controls",category,Obsolete +CWE-265,Privilege Issues,category,Incomplete +CWE-266,Incorrect Privilege Assignment,weakness,Draft +CWE-267,Privilege Defined With Unsafe Actions,weakness,Incomplete +CWE-268,Privilege Chaining,weakness,Draft +CWE-269,Improper Privilege Management,weakness,Draft +CWE-270,Privilege Context Switching Error,weakness,Draft +CWE-271,Privilege Dropping / Lowering Errors,weakness,Incomplete +CWE-272,Least Privilege Violation,weakness,Incomplete +CWE-273,Improper Check for Dropped Privileges,weakness,Incomplete +CWE-274,Improper Handling of Insufficient Privileges,weakness,Draft +CWE-275,Permission Issues,category,Draft +CWE-276,Incorrect Default Permissions,weakness,Draft +CWE-277,Insecure Inherited Permissions,weakness,Draft +CWE-278,Insecure Preserved Inherited Permissions,weakness,Incomplete +CWE-279,Incorrect Execution-Assigned Permissions,weakness,Draft +CWE-280,Improper Handling of Insufficient Permissions or Privileges,weakness,Draft +CWE-281,Improper Preservation of Permissions,weakness,Draft +CWE-282,Improper Ownership Management,weakness,Draft +CWE-283,Unverified Ownership,weakness,Draft +CWE-284,Improper Access Control,weakness,Incomplete +CWE-285,Improper Authorization,weakness,Draft +CWE-286,Incorrect User Management,weakness,Incomplete +CWE-287,Improper Authentication,weakness,Draft +CWE-288,Authentication Bypass Using an Alternate Path or Channel,weakness,Incomplete +CWE-289,Authentication Bypass by Alternate Name,weakness,Incomplete +CWE-290,Authentication Bypass by Spoofing,weakness,Incomplete +CWE-291,Reliance on IP Address for Authentication,weakness,Incomplete +CWE-292,DEPRECATED: Trusting Self-reported DNS Name,weakness,Deprecated +CWE-293,Using Referer Field for Authentication,weakness,Draft +CWE-294,Authentication Bypass by Capture-replay,weakness,Incomplete +CWE-295,Improper Certificate Validation,weakness,Draft +CWE-296,Improper Following of a Certificate's Chain of Trust,weakness,Draft +CWE-297,Improper Validation of Certificate with Host Mismatch,weakness,Incomplete +CWE-298,Improper Validation of Certificate Expiration,weakness,Draft +CWE-299,Improper Check for Certificate Revocation,weakness,Draft +CWE-300,Channel Accessible by Non-Endpoint,weakness,Draft +CWE-301,Reflection Attack in an Authentication Protocol,weakness,Draft +CWE-302,Authentication Bypass by Assumed-Immutable Data,weakness,Incomplete +CWE-303,Incorrect Implementation of Authentication Algorithm,weakness,Draft +CWE-304,Missing Critical Step in Authentication,weakness,Draft +CWE-305,Authentication Bypass by Primary Weakness,weakness,Draft +CWE-306,Missing Authentication for Critical Function,weakness,Draft +CWE-307,Improper Restriction of Excessive Authentication Attempts,weakness,Draft +CWE-308,Use of Single-factor Authentication,weakness,Draft +CWE-309,Use of Password System for Primary Authentication,weakness,Draft +CWE-310,Cryptographic Issues,category,Draft +CWE-311,Missing Encryption of Sensitive Data,weakness,Draft +CWE-312,Cleartext Storage of Sensitive Information,weakness,Draft +CWE-313,Cleartext Storage in a File or on Disk,weakness,Draft +CWE-314,Cleartext Storage in the Registry,weakness,Draft +CWE-315,Cleartext Storage of Sensitive Information in a Cookie,weakness,Draft +CWE-316,Cleartext Storage of Sensitive Information in Memory,weakness,Draft +CWE-317,Cleartext Storage of Sensitive Information in GUI,weakness,Draft +CWE-318,Cleartext Storage of Sensitive Information in Executable,weakness,Draft +CWE-319,Cleartext Transmission of Sensitive Information,weakness,Draft +CWE-320,Key Management Errors,category,Obsolete +CWE-321,Use of Hard-coded Cryptographic Key,weakness,Draft +CWE-322,Key Exchange without Entity Authentication,weakness,Draft +CWE-323,"Reusing a Nonce, Key Pair in Encryption",weakness,Incomplete +CWE-324,Use of a Key Past its Expiration Date,weakness,Draft +CWE-325,Missing Cryptographic Step,weakness,Draft +CWE-326,Inadequate Encryption Strength,weakness,Draft +CWE-327,Use of a Broken or Risky Cryptographic Algorithm,weakness,Draft +CWE-328,Use of Weak Hash,weakness,Draft +CWE-329,Generation of Predictable IV with CBC Mode,weakness,Draft +CWE-330,Use of Insufficiently Random Values,weakness,Stable +CWE-331,Insufficient Entropy,weakness,Draft +CWE-332,Insufficient Entropy in PRNG,weakness,Draft +CWE-333,Improper Handling of Insufficient Entropy in TRNG,weakness,Draft +CWE-334,Small Space of Random Values,weakness,Draft +CWE-335,Incorrect Usage of Seeds in Pseudo-Random Number Generator (PRNG),weakness,Draft +CWE-336,Same Seed in Pseudo-Random Number Generator (PRNG),weakness,Draft +CWE-337,Predictable Seed in Pseudo-Random Number Generator (PRNG),weakness,Draft +CWE-338,Use of Cryptographically Weak Pseudo-Random Number Generator (PRNG),weakness,Draft +CWE-339,Small Seed Space in PRNG,weakness,Draft +CWE-340,Generation of Predictable Numbers or Identifiers,weakness,Incomplete +CWE-341,Predictable from Observable State,weakness,Draft +CWE-342,Predictable Exact Value from Previous Values,weakness,Draft +CWE-343,Predictable Value Range from Previous Values,weakness,Draft +CWE-344,Use of Invariant Value in Dynamically Changing Context,weakness,Draft +CWE-345,Insufficient Verification of Data Authenticity,weakness,Draft +CWE-346,Origin Validation Error,weakness,Draft +CWE-347,Improper Verification of Cryptographic Signature,weakness,Draft +CWE-348,Use of Less Trusted Source,weakness,Draft +CWE-349,Acceptance of Extraneous Untrusted Data With Trusted Data,weakness,Draft +CWE-350,Reliance on Reverse DNS Resolution for a Security-Critical Action,weakness,Draft +CWE-351,Insufficient Type Distinction,weakness,Draft +CWE-352,Cross-Site Request Forgery (CSRF),weakness,Stable +CWE-353,Missing Support for Integrity Check,weakness,Draft +CWE-354,Improper Validation of Integrity Check Value,weakness,Draft +CWE-355,User Interface Security Issues,category,Draft +CWE-356,Product UI does not Warn User of Unsafe Actions,weakness,Incomplete +CWE-357,Insufficient UI Warning of Dangerous Operations,weakness,Draft +CWE-358,Improperly Implemented Security Check for Standard,weakness,Draft +CWE-359,Exposure of Private Personal Information to an Unauthorized Actor,weakness,Incomplete +CWE-360,Trust of System Event Data,weakness,Incomplete +CWE-361,7PK - Time and State,category,Incomplete +CWE-362,Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition'),weakness,Draft +CWE-363,Race Condition Enabling Link Following,weakness,Draft +CWE-364,Signal Handler Race Condition,weakness,Incomplete +CWE-365,DEPRECATED: Race Condition in Switch,weakness,Deprecated +CWE-366,Race Condition within a Thread,weakness,Draft +CWE-367,Time-of-check Time-of-use (TOCTOU) Race Condition,weakness,Incomplete +CWE-368,Context Switching Race Condition,weakness,Draft +CWE-369,Divide By Zero,weakness,Draft +CWE-370,Missing Check for Certificate Revocation after Initial Check,weakness,Draft +CWE-371,State Issues,category,Draft +CWE-372,Incomplete Internal State Distinction,weakness,Draft +CWE-373,DEPRECATED: State Synchronization Error,weakness,Deprecated +CWE-374,Passing Mutable Objects to an Untrusted Method,weakness,Draft +CWE-375,Returning a Mutable Object to an Untrusted Caller,weakness,Draft +CWE-376,DEPRECATED: Temporary File Issues,category,Deprecated +CWE-377,Insecure Temporary File,weakness,Incomplete +CWE-378,Creation of Temporary File With Insecure Permissions,weakness,Draft +CWE-379,Creation of Temporary File in Directory with Insecure Permissions,weakness,Incomplete +CWE-380,DEPRECATED: Technology-Specific Time and State Issues,category,Deprecated +CWE-381,DEPRECATED: J2EE Time and State Issues,category,Deprecated +CWE-382,J2EE Bad Practices: Use of System.exit(),weakness,Draft +CWE-383,J2EE Bad Practices: Direct Use of Threads,weakness,Draft +CWE-384,Session Fixation,weakness,Incomplete +CWE-385,Covert Timing Channel,weakness,Incomplete +CWE-386,Symbolic Name not Mapping to Correct Object,weakness,Draft +CWE-387,Signal Errors,category,Incomplete +CWE-388,7PK - Errors,category,Draft +CWE-389,"Error Conditions, Return Values, Status Codes",category,Incomplete +CWE-390,Detection of Error Condition Without Action,weakness,Draft +CWE-391,Unchecked Error Condition,weakness,Incomplete +CWE-392,Missing Report of Error Condition,weakness,Draft +CWE-393,Return of Wrong Status Code,weakness,Draft +CWE-394,Unexpected Status Code or Return Value,weakness,Draft +CWE-395,Use of NullPointerException Catch to Detect NULL Pointer Dereference,weakness,Draft +CWE-396,Declaration of Catch for Generic Exception,weakness,Draft +CWE-397,Declaration of Throws for Generic Exception,weakness,Draft +CWE-398,7PK - Code Quality,category,Draft +CWE-399,Resource Management Errors,category,Draft +CWE-400,Uncontrolled Resource Consumption,weakness,Draft +CWE-401,Missing Release of Memory after Effective Lifetime,weakness,Draft +CWE-402,Transmission of Private Resources into a New Sphere ('Resource Leak'),weakness,Draft +CWE-403,Exposure of File Descriptor to Unintended Control Sphere ('File Descriptor Leak'),weakness,Draft +CWE-404,Improper Resource Shutdown or Release,weakness,Draft +CWE-405,Asymmetric Resource Consumption (Amplification),weakness,Incomplete +CWE-406,Insufficient Control of Network Message Volume (Network Amplification),weakness,Incomplete +CWE-407,Inefficient Algorithmic Complexity,weakness,Incomplete +CWE-408,Incorrect Behavior Order: Early Amplification,weakness,Draft +CWE-409,Improper Handling of Highly Compressed Data (Data Amplification),weakness,Incomplete +CWE-410,Insufficient Resource Pool,weakness,Incomplete +CWE-411,Resource Locking Problems,category,Draft +CWE-412,Unrestricted Externally Accessible Lock,weakness,Incomplete +CWE-413,Improper Resource Locking,weakness,Draft +CWE-414,Missing Lock Check,weakness,Draft +CWE-415,Double Free,weakness,Draft +CWE-416,Use After Free,weakness,Stable +CWE-417,Communication Channel Errors,category,Draft +CWE-418,DEPRECATED: Channel Errors,category,Deprecated +CWE-419,Unprotected Primary Channel,weakness,Draft +CWE-420,Unprotected Alternate Channel,weakness,Draft +CWE-421,Race Condition During Access to Alternate Channel,weakness,Draft +CWE-422,Unprotected Windows Messaging Channel ('Shatter'),weakness,Draft +CWE-423,DEPRECATED: Proxied Trusted Channel,weakness,Deprecated +CWE-424,Improper Protection of Alternate Path,weakness,Draft +CWE-425,Direct Request ('Forced Browsing'),weakness,Incomplete +CWE-426,Untrusted Search Path,weakness,Stable +CWE-427,Uncontrolled Search Path Element,weakness,Draft +CWE-428,Unquoted Search Path or Element,weakness,Draft +CWE-429,Handler Errors,category,Draft +CWE-430,Deployment of Wrong Handler,weakness,Incomplete +CWE-431,Missing Handler,weakness,Draft +CWE-432,Dangerous Signal Handler not Disabled During Sensitive Operations,weakness,Draft +CWE-433,Unparsed Raw Web Content Delivery,weakness,Incomplete +CWE-434,Unrestricted Upload of File with Dangerous Type,weakness,Draft +CWE-435,Improper Interaction Between Multiple Correctly-Behaving Entities,weakness,Draft +CWE-436,Interpretation Conflict,weakness,Incomplete +CWE-437,Incomplete Model of Endpoint Features,weakness,Incomplete +CWE-438,Behavioral Problems,category,Draft +CWE-439,Behavioral Change in New Version or Environment,weakness,Draft +CWE-440,Expected Behavior Violation,weakness,Draft +CWE-441,Unintended Proxy or Intermediary ('Confused Deputy'),weakness,Draft +CWE-442,DEPRECATED: Web Problems,category,Deprecated +CWE-443,DEPRECATED: HTTP response splitting,weakness,Deprecated +CWE-444,Inconsistent Interpretation of HTTP Requests ('HTTP Request/Response Smuggling'),weakness,Incomplete +CWE-445,DEPRECATED: User Interface Errors,category,Deprecated +CWE-446,UI Discrepancy for Security Feature,weakness,Incomplete +CWE-447,Unimplemented or Unsupported Feature in UI,weakness,Draft +CWE-448,Obsolete Feature in UI,weakness,Draft +CWE-449,The UI Performs the Wrong Action,weakness,Incomplete +CWE-450,Multiple Interpretations of UI Input,weakness,Draft +CWE-451,User Interface (UI) Misrepresentation of Critical Information,weakness,Draft +CWE-452,Initialization and Cleanup Errors,category,Draft +CWE-453,Insecure Default Variable Initialization,weakness,Draft +CWE-454,External Initialization of Trusted Variables or Data Stores,weakness,Draft +CWE-455,Non-exit on Failed Initialization,weakness,Draft +CWE-456,Missing Initialization of a Variable,weakness,Draft +CWE-457,Use of Uninitialized Variable,weakness,Draft +CWE-458,DEPRECATED: Incorrect Initialization,weakness,Deprecated +CWE-459,Incomplete Cleanup,weakness,Draft +CWE-460,Improper Cleanup on Thrown Exception,weakness,Draft +CWE-461,DEPRECATED: Data Structure Issues,category,Deprecated +CWE-462,Duplicate Key in Associative List (Alist),weakness,Incomplete +CWE-463,Deletion of Data Structure Sentinel,weakness,Incomplete +CWE-464,Addition of Data Structure Sentinel,weakness,Incomplete +CWE-465,Pointer Issues,category,Draft +CWE-466,Return of Pointer Value Outside of Expected Range,weakness,Draft +CWE-467,Use of sizeof() on a Pointer Type,weakness,Draft +CWE-468,Incorrect Pointer Scaling,weakness,Incomplete +CWE-469,Use of Pointer Subtraction to Determine Size,weakness,Draft +CWE-470,Use of Externally-Controlled Input to Select Classes or Code ('Unsafe Reflection'),weakness,Draft +CWE-471,Modification of Assumed-Immutable Data (MAID),weakness,Draft +CWE-472,External Control of Assumed-Immutable Web Parameter,weakness,Draft +CWE-473,PHP External Variable Modification,weakness,Draft +CWE-474,Use of Function with Inconsistent Implementations,weakness,Draft +CWE-475,Undefined Behavior for Input to API,weakness,Incomplete +CWE-476,NULL Pointer Dereference,weakness,Stable +CWE-477,Use of Obsolete Function,weakness,Draft +CWE-478,Missing Default Case in Multiple Condition Expression,weakness,Draft +CWE-479,Signal Handler Use of a Non-reentrant Function,weakness,Draft +CWE-480,Use of Incorrect Operator,weakness,Draft +CWE-481,Assigning instead of Comparing,weakness,Draft +CWE-482,Comparing instead of Assigning,weakness,Draft +CWE-483,Incorrect Block Delimitation,weakness,Draft +CWE-484,Omitted Break Statement in Switch,weakness,Draft +CWE-485,7PK - Encapsulation,category,Draft +CWE-486,Comparison of Classes by Name,weakness,Draft +CWE-487,Reliance on Package-level Scope,weakness,Incomplete +CWE-488,Exposure of Data Element to Wrong Session,weakness,Draft +CWE-489,Active Debug Code,weakness,Draft +CWE-490,DEPRECATED: Mobile Code Issues,category,Deprecated +CWE-491,Public cloneable() Method Without Final ('Object Hijack'),weakness,Draft +CWE-492,Use of Inner Class Containing Sensitive Data,weakness,Draft +CWE-493,Critical Public Variable Without Final Modifier,weakness,Draft +CWE-494,Download of Code Without Integrity Check,weakness,Draft +CWE-495,Private Data Structure Returned From A Public Method,weakness,Draft +CWE-496,Public Data Assigned to Private Array-Typed Field,weakness,Incomplete +CWE-497,Exposure of Sensitive System Information to an Unauthorized Control Sphere,weakness,Incomplete +CWE-498,Cloneable Class Containing Sensitive Information,weakness,Draft +CWE-499,Serializable Class Containing Sensitive Data,weakness,Draft +CWE-500,Public Static Field Not Marked Final,weakness,Draft +CWE-501,Trust Boundary Violation,weakness,Draft +CWE-502,Deserialization of Untrusted Data,weakness,Draft +CWE-503,DEPRECATED: Byte/Object Code,category,Deprecated +CWE-504,DEPRECATED: Motivation/Intent,category,Deprecated +CWE-505,DEPRECATED: Intentionally Introduced Weakness,category,Deprecated +CWE-506,Embedded Malicious Code,weakness,Incomplete +CWE-507,Trojan Horse,weakness,Incomplete +CWE-508,Non-Replicating Malicious Code,weakness,Incomplete +CWE-509,Replicating Malicious Code (Virus or Worm),weakness,Incomplete +CWE-510,Trapdoor,weakness,Incomplete +CWE-511,Logic/Time Bomb,weakness,Incomplete +CWE-512,Spyware,weakness,Incomplete +CWE-513,DEPRECATED: Intentionally Introduced Nonmalicious Weakness,category,Deprecated +CWE-514,Covert Channel,weakness,Incomplete +CWE-515,Covert Storage Channel,weakness,Incomplete +CWE-516,DEPRECATED: Covert Timing Channel,weakness,Deprecated +CWE-517,"DEPRECATED: Other Intentional, Nonmalicious Weakness",category,Deprecated +CWE-518,DEPRECATED: Inadvertently Introduced Weakness,category,Deprecated +CWE-519,DEPRECATED: .NET Environment Issues,category,Deprecated +CWE-520,.NET Misconfiguration: Use of Impersonation,weakness,Incomplete +CWE-521,Weak Password Requirements,weakness,Draft +CWE-522,Insufficiently Protected Credentials,weakness,Incomplete +CWE-523,Unprotected Transport of Credentials,weakness,Incomplete +CWE-524,Use of Cache Containing Sensitive Information,weakness,Incomplete +CWE-525,Use of Web Browser Cache Containing Sensitive Information,weakness,Incomplete +CWE-526,Cleartext Storage of Sensitive Information in an Environment Variable,weakness,Incomplete +CWE-527,Exposure of Version-Control Repository to an Unauthorized Control Sphere,weakness,Incomplete +CWE-528,Exposure of Core Dump File to an Unauthorized Control Sphere,weakness,Draft +CWE-529,Exposure of Access Control List Files to an Unauthorized Control Sphere,weakness,Incomplete +CWE-530,Exposure of Backup File to an Unauthorized Control Sphere,weakness,Incomplete +CWE-531,Inclusion of Sensitive Information in Test Code,weakness,Incomplete +CWE-532,Insertion of Sensitive Information into Log File,weakness,Incomplete +CWE-533,DEPRECATED: Information Exposure Through Server Log Files,weakness,Deprecated +CWE-534,DEPRECATED: Information Exposure Through Debug Log Files,weakness,Deprecated +CWE-535,Exposure of Information Through Shell Error Message,weakness,Incomplete +CWE-536,Servlet Runtime Error Message Containing Sensitive Information,weakness,Incomplete +CWE-537,Java Runtime Error Message Containing Sensitive Information,weakness,Incomplete +CWE-538,Insertion of Sensitive Information into Externally-Accessible File or Directory,weakness,Draft +CWE-539,Use of Persistent Cookies Containing Sensitive Information,weakness,Incomplete +CWE-540,Inclusion of Sensitive Information in Source Code,weakness,Incomplete +CWE-541,Inclusion of Sensitive Information in an Include File,weakness,Incomplete +CWE-542,DEPRECATED: Information Exposure Through Cleanup Log Files,weakness,Deprecated +CWE-543,Use of Singleton Pattern Without Synchronization in a Multithreaded Context,weakness,Incomplete +CWE-544,Missing Standardized Error Handling Mechanism,weakness,Draft +CWE-545,DEPRECATED: Use of Dynamic Class Loading,weakness,Deprecated +CWE-546,Suspicious Comment,weakness,Draft +CWE-547,"Use of Hard-coded, Security-relevant Constants",weakness,Draft +CWE-548,Exposure of Information Through Directory Listing,weakness,Draft +CWE-549,Missing Password Field Masking,weakness,Draft +CWE-550,Server-generated Error Message Containing Sensitive Information,weakness,Incomplete +CWE-551,Incorrect Behavior Order: Authorization Before Parsing and Canonicalization,weakness,Incomplete +CWE-552,Files or Directories Accessible to External Parties,weakness,Draft +CWE-553,Command Shell in Externally Accessible Directory,weakness,Incomplete +CWE-554,ASP.NET Misconfiguration: Not Using Input Validation Framework,weakness,Draft +CWE-555,J2EE Misconfiguration: Plaintext Password in Configuration File,weakness,Draft +CWE-556,ASP.NET Misconfiguration: Use of Identity Impersonation,weakness,Incomplete +CWE-557,Concurrency Issues,category,Draft +CWE-558,Use of getlogin() in Multithreaded Application,weakness,Draft +CWE-559,DEPRECATED: Often Misused: Arguments and Parameters,category,Deprecated +CWE-560,Use of umask() with chmod-style Argument,weakness,Draft +CWE-561,Dead Code,weakness,Draft +CWE-562,Return of Stack Variable Address,weakness,Draft +CWE-563,Assignment to Variable without Use,weakness,Draft +CWE-564,SQL Injection: Hibernate,weakness,Incomplete +CWE-565,Reliance on Cookies without Validation and Integrity Checking,weakness,Incomplete +CWE-566,Authorization Bypass Through User-Controlled SQL Primary Key,weakness,Incomplete +CWE-567,Unsynchronized Access to Shared Data in a Multithreaded Context,weakness,Draft +CWE-568,finalize() Method Without super.finalize(),weakness,Draft +CWE-569,Expression Issues,category,Draft +CWE-570,Expression is Always False,weakness,Draft +CWE-571,Expression is Always True,weakness,Draft +CWE-572,Call to Thread run() instead of start(),weakness,Draft +CWE-573,Improper Following of Specification by Caller,weakness,Draft +CWE-574,EJB Bad Practices: Use of Synchronization Primitives,weakness,Draft +CWE-575,EJB Bad Practices: Use of AWT Swing,weakness,Draft +CWE-576,EJB Bad Practices: Use of Java I/O,weakness,Draft +CWE-577,EJB Bad Practices: Use of Sockets,weakness,Draft +CWE-578,EJB Bad Practices: Use of Class Loader,weakness,Draft +CWE-579,J2EE Bad Practices: Non-serializable Object Stored in Session,weakness,Draft +CWE-580,clone() Method Without super.clone(),weakness,Draft +CWE-581,Object Model Violation: Just One of Equals and Hashcode Defined,weakness,Draft +CWE-582,"Array Declared Public, Final, and Static",weakness,Draft +CWE-583,finalize() Method Declared Public,weakness,Incomplete +CWE-584,Return Inside Finally Block,weakness,Draft +CWE-585,Empty Synchronized Block,weakness,Draft +CWE-586,Explicit Call to Finalize(),weakness,Draft +CWE-587,Assignment of a Fixed Address to a Pointer,weakness,Draft +CWE-588,Attempt to Access Child of a Non-structure Pointer,weakness,Incomplete +CWE-589,Call to Non-ubiquitous API,weakness,Incomplete +CWE-590,Free of Memory not on the Heap,weakness,Incomplete +CWE-591,Sensitive Data Storage in Improperly Locked Memory,weakness,Draft +CWE-592,DEPRECATED: Authentication Bypass Issues,weakness,Deprecated +CWE-593,Authentication Bypass: OpenSSL CTX Object Modified after SSL Objects are Created,weakness,Draft +CWE-594,J2EE Framework: Saving Unserializable Objects to Disk,weakness,Incomplete +CWE-595,Comparison of Object References Instead of Object Contents,weakness,Incomplete +CWE-596,DEPRECATED: Incorrect Semantic Object Comparison,weakness,Deprecated +CWE-597,Use of Wrong Operator in String Comparison,weakness,Draft +CWE-598,Use of HTTP Request With Sensitive Query String,weakness,Draft +CWE-599,Missing Validation of OpenSSL Certificate,weakness,Incomplete +CWE-600,Uncaught Exception in Servlet,weakness,Draft +CWE-601,URL Redirection to Untrusted Site ('Open Redirect'),weakness,Draft +CWE-602,Client-Side Enforcement of Server-Side Security,weakness,Draft +CWE-603,Use of Client-Side Authentication,weakness,Draft +CWE-604,Deprecated Entries,view,Draft +CWE-605,Multiple Binds to the Same Port,weakness,Draft +CWE-606,Unchecked Input for Loop Condition,weakness,Draft +CWE-607,Public Static Final Field References Mutable Object,weakness,Draft +CWE-608,Struts: Non-private Field in ActionForm Class,weakness,Draft +CWE-609,Double-Checked Locking,weakness,Draft +CWE-610,Externally Controlled Reference to a Resource in Another Sphere,weakness,Draft +CWE-611,Improper Restriction of XML External Entity Reference,weakness,Draft +CWE-612,Improper Authorization of Index Containing Sensitive Information,weakness,Draft +CWE-613,Insufficient Session Expiration,weakness,Incomplete +CWE-614,Sensitive Cookie in HTTPS Session Without 'Secure' Attribute,weakness,Draft +CWE-615,Inclusion of Sensitive Information in Source Code Comments,weakness,Incomplete +CWE-616,Incomplete Identification of Uploaded File Variables (PHP),weakness,Incomplete +CWE-617,Reachable Assertion,weakness,Draft +CWE-618,Exposed Unsafe ActiveX Method,weakness,Incomplete +CWE-619,Dangling Database Cursor ('Cursor Injection'),weakness,Incomplete +CWE-620,Unverified Password Change,weakness,Draft +CWE-621,Variable Extraction Error,weakness,Incomplete +CWE-622,Improper Validation of Function Hook Arguments,weakness,Draft +CWE-623,Unsafe ActiveX Control Marked Safe For Scripting,weakness,Draft +CWE-624,Executable Regular Expression Error,weakness,Incomplete +CWE-625,Permissive Regular Expression,weakness,Draft +CWE-626,Null Byte Interaction Error (Poison Null Byte),weakness,Draft +CWE-627,Dynamic Variable Evaluation,weakness,Incomplete +CWE-628,Function Call with Incorrectly Specified Arguments,weakness,Draft +CWE-629,Weaknesses in OWASP Top Ten (2007),view,Obsolete +CWE-630,DEPRECATED: Weaknesses Examined by SAMATE,view,Deprecated +CWE-631,DEPRECATED: Resource-specific Weaknesses,view,Deprecated +CWE-632,DEPRECATED: Weaknesses that Affect Files or Directories,category,Deprecated +CWE-633,DEPRECATED: Weaknesses that Affect Memory,category,Deprecated +CWE-634,DEPRECATED: Weaknesses that Affect System Processes,category,Deprecated +CWE-635,Weaknesses Originally Used by NVD from 2008 to 2016,view,Obsolete +CWE-636,Not Failing Securely ('Failing Open'),weakness,Draft +CWE-637,Unnecessary Complexity in Protection Mechanism (Not Using 'Economy of Mechanism'),weakness,Draft +CWE-638,Not Using Complete Mediation,weakness,Draft +CWE-639,Authorization Bypass Through User-Controlled Key,weakness,Incomplete +CWE-640,Weak Password Recovery Mechanism for Forgotten Password,weakness,Incomplete +CWE-641,Improper Restriction of Names for Files and Other Resources,weakness,Incomplete +CWE-642,External Control of Critical State Data,weakness,Draft +CWE-643,Improper Neutralization of Data within XPath Expressions ('XPath Injection'),weakness,Incomplete +CWE-644,Improper Neutralization of HTTP Headers for Scripting Syntax,weakness,Incomplete +CWE-645,Overly Restrictive Account Lockout Mechanism,weakness,Incomplete +CWE-646,Reliance on File Name or Extension of Externally-Supplied File,weakness,Incomplete +CWE-647,Use of Non-Canonical URL Paths for Authorization Decisions,weakness,Incomplete +CWE-648,Incorrect Use of Privileged APIs,weakness,Incomplete +CWE-649,Reliance on Obfuscation or Encryption of Security-Relevant Inputs without Integrity Checking,weakness,Incomplete +CWE-650,Trusting HTTP Permission Methods on the Server Side,weakness,Incomplete +CWE-651,Exposure of WSDL File Containing Sensitive Information,weakness,Incomplete +CWE-652,Improper Neutralization of Data within XQuery Expressions ('XQuery Injection'),weakness,Incomplete +CWE-653,Improper Isolation or Compartmentalization,weakness,Draft +CWE-654,Reliance on a Single Factor in a Security Decision,weakness,Draft +CWE-655,Insufficient Psychological Acceptability,weakness,Draft +CWE-656,Reliance on Security Through Obscurity,weakness,Draft +CWE-657,Violation of Secure Design Principles,weakness,Draft +CWE-658,Weaknesses in Software Written in C,view,Draft +CWE-659,Weaknesses in Software Written in C++,view,Draft +CWE-660,Weaknesses in Software Written in Java,view,Draft +CWE-661,Weaknesses in Software Written in PHP,view,Draft +CWE-662,Improper Synchronization,weakness,Draft +CWE-663,Use of a Non-reentrant Function in a Concurrent Context,weakness,Draft +CWE-664,Improper Control of a Resource Through its Lifetime,weakness,Draft +CWE-665,Improper Initialization,weakness,Draft +CWE-666,Operation on Resource in Wrong Phase of Lifetime,weakness,Draft +CWE-667,Improper Locking,weakness,Draft +CWE-668,Exposure of Resource to Wrong Sphere,weakness,Draft +CWE-669,Incorrect Resource Transfer Between Spheres,weakness,Draft +CWE-670,Always-Incorrect Control Flow Implementation,weakness,Draft +CWE-671,Lack of Administrator Control over Security,weakness,Draft +CWE-672,Operation on a Resource after Expiration or Release,weakness,Draft +CWE-673,External Influence of Sphere Definition,weakness,Draft +CWE-674,Uncontrolled Recursion,weakness,Draft +CWE-675,Multiple Operations on Resource in Single-Operation Context,weakness,Draft +CWE-676,Use of Potentially Dangerous Function,weakness,Draft +CWE-677,Weakness Base Elements,view,Draft +CWE-678,Composites,view,Draft +CWE-679,DEPRECATED: Chain Elements,view,Deprecated +CWE-680,Integer Overflow to Buffer Overflow,weakness,Draft +CWE-681,Incorrect Conversion between Numeric Types,weakness,Draft +CWE-682,Incorrect Calculation,weakness,Draft +CWE-683,Function Call With Incorrect Order of Arguments,weakness,Draft +CWE-684,Incorrect Provision of Specified Functionality,weakness,Draft +CWE-685,Function Call With Incorrect Number of Arguments,weakness,Draft +CWE-686,Function Call With Incorrect Argument Type,weakness,Draft +CWE-687,Function Call With Incorrectly Specified Argument Value,weakness,Draft +CWE-688,Function Call With Incorrect Variable or Reference as Argument,weakness,Draft +CWE-689,Permission Race Condition During Resource Copy,weakness,Draft +CWE-690,Unchecked Return Value to NULL Pointer Dereference,weakness,Draft +CWE-691,Insufficient Control Flow Management,weakness,Draft +CWE-692,Incomplete Denylist to Cross-Site Scripting,weakness,Draft +CWE-693,Protection Mechanism Failure,weakness,Draft +CWE-694,Use of Multiple Resources with Duplicate Identifier,weakness,Incomplete +CWE-695,Use of Low-Level Functionality,weakness,Incomplete +CWE-696,Incorrect Behavior Order,weakness,Incomplete +CWE-697,Incorrect Comparison,weakness,Incomplete +CWE-698,Execution After Redirect (EAR),weakness,Incomplete +CWE-699,Software Development,view,Draft +CWE-700,Seven Pernicious Kingdoms,view,Incomplete +CWE-701,Weaknesses Introduced During Design,view,Incomplete +CWE-702,Weaknesses Introduced During Implementation,view,Incomplete +CWE-703,Improper Check or Handling of Exceptional Conditions,weakness,Incomplete +CWE-704,Incorrect Type Conversion or Cast,weakness,Incomplete +CWE-705,Incorrect Control Flow Scoping,weakness,Incomplete +CWE-706,Use of Incorrectly-Resolved Name or Reference,weakness,Incomplete +CWE-707,Improper Neutralization,weakness,Incomplete +CWE-708,Incorrect Ownership Assignment,weakness,Incomplete +CWE-709,Named Chains,view,Incomplete +CWE-710,Improper Adherence to Coding Standards,weakness,Incomplete +CWE-711,Weaknesses in OWASP Top Ten (2004),view,Obsolete +CWE-712,OWASP Top Ten 2007 Category A1 - Cross Site Scripting (XSS),category,Obsolete +CWE-713,OWASP Top Ten 2007 Category A2 - Injection Flaws,category,Obsolete +CWE-714,OWASP Top Ten 2007 Category A3 - Malicious File Execution,category,Obsolete +CWE-715,OWASP Top Ten 2007 Category A4 - Insecure Direct Object Reference,category,Obsolete +CWE-716,OWASP Top Ten 2007 Category A5 - Cross Site Request Forgery (CSRF),category,Obsolete +CWE-717,OWASP Top Ten 2007 Category A6 - Information Leakage and Improper Error Handling,category,Obsolete +CWE-718,OWASP Top Ten 2007 Category A7 - Broken Authentication and Session Management,category,Obsolete +CWE-719,OWASP Top Ten 2007 Category A8 - Insecure Cryptographic Storage,category,Obsolete +CWE-720,OWASP Top Ten 2007 Category A9 - Insecure Communications,category,Obsolete +CWE-721,OWASP Top Ten 2007 Category A10 - Failure to Restrict URL Access,category,Obsolete +CWE-722,OWASP Top Ten 2004 Category A1 - Unvalidated Input,category,Obsolete +CWE-723,OWASP Top Ten 2004 Category A2 - Broken Access Control,category,Obsolete +CWE-724,OWASP Top Ten 2004 Category A3 - Broken Authentication and Session Management,category,Obsolete +CWE-725,OWASP Top Ten 2004 Category A4 - Cross-Site Scripting (XSS) Flaws,category,Obsolete +CWE-726,OWASP Top Ten 2004 Category A5 - Buffer Overflows,category,Obsolete +CWE-727,OWASP Top Ten 2004 Category A6 - Injection Flaws,category,Obsolete +CWE-728,OWASP Top Ten 2004 Category A7 - Improper Error Handling,category,Obsolete +CWE-729,OWASP Top Ten 2004 Category A8 - Insecure Storage,category,Obsolete +CWE-730,OWASP Top Ten 2004 Category A9 - Denial of Service,category,Obsolete +CWE-731,OWASP Top Ten 2004 Category A10 - Insecure Configuration Management,category,Obsolete +CWE-732,Incorrect Permission Assignment for Critical Resource,weakness,Draft +CWE-733,Compiler Optimization Removal or Modification of Security-critical Code,weakness,Incomplete +CWE-734,Weaknesses Addressed by the CERT C Secure Coding Standard (2008),view,Obsolete +CWE-735,CERT C Secure Coding Standard (2008) Chapter 2 - Preprocessor (PRE),category,Obsolete +CWE-736,CERT C Secure Coding Standard (2008) Chapter 3 - Declarations and Initialization (DCL),category,Obsolete +CWE-737,CERT C Secure Coding Standard (2008) Chapter 4 - Expressions (EXP),category,Obsolete +CWE-738,CERT C Secure Coding Standard (2008) Chapter 5 - Integers (INT),category,Obsolete +CWE-739,CERT C Secure Coding Standard (2008) Chapter 6 - Floating Point (FLP),category,Obsolete +CWE-740,CERT C Secure Coding Standard (2008) Chapter 7 - Arrays (ARR),category,Obsolete +CWE-741,CERT C Secure Coding Standard (2008) Chapter 8 - Characters and Strings (STR),category,Obsolete +CWE-742,CERT C Secure Coding Standard (2008) Chapter 9 - Memory Management (MEM),category,Obsolete +CWE-743,CERT C Secure Coding Standard (2008) Chapter 10 - Input Output (FIO),category,Obsolete +CWE-744,CERT C Secure Coding Standard (2008) Chapter 11 - Environment (ENV),category,Obsolete +CWE-745,CERT C Secure Coding Standard (2008) Chapter 12 - Signals (SIG),category,Obsolete +CWE-746,CERT C Secure Coding Standard (2008) Chapter 13 - Error Handling (ERR),category,Obsolete +CWE-747,CERT C Secure Coding Standard (2008) Chapter 14 - Miscellaneous (MSC),category,Obsolete +CWE-748,CERT C Secure Coding Standard (2008) Appendix - POSIX (POS),category,Obsolete +CWE-749,Exposed Dangerous Method or Function,weakness,Incomplete +CWE-750,Weaknesses in the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors,view,Obsolete +CWE-751,2009 Top 25 - Insecure Interaction Between Components,category,Obsolete +CWE-752,2009 Top 25 - Risky Resource Management,category,Obsolete +CWE-753,2009 Top 25 - Porous Defenses,category,Obsolete +CWE-754,Improper Check for Unusual or Exceptional Conditions,weakness,Incomplete +CWE-755,Improper Handling of Exceptional Conditions,weakness,Incomplete +CWE-756,Missing Custom Error Page,weakness,Incomplete +CWE-757,Selection of Less-Secure Algorithm During Negotiation ('Algorithm Downgrade'),weakness,Incomplete +CWE-758,"Reliance on Undefined, Unspecified, or Implementation-Defined Behavior",weakness,Incomplete +CWE-759,Use of a One-Way Hash without a Salt,weakness,Incomplete +CWE-760,Use of a One-Way Hash with a Predictable Salt,weakness,Incomplete +CWE-761,Free of Pointer not at Start of Buffer,weakness,Incomplete +CWE-762,Mismatched Memory Management Routines,weakness,Incomplete +CWE-763,Release of Invalid Pointer or Reference,weakness,Incomplete +CWE-764,Multiple Locks of a Critical Resource,weakness,Incomplete +CWE-765,Multiple Unlocks of a Critical Resource,weakness,Incomplete +CWE-766,Critical Data Element Declared Public,weakness,Incomplete +CWE-767,Access to Critical Private Variable via Public Method,weakness,Incomplete +CWE-768,Incorrect Short Circuit Evaluation,weakness,Incomplete +CWE-769,DEPRECATED: Uncontrolled File Descriptor Consumption,weakness,Deprecated +CWE-770,Allocation of Resources Without Limits or Throttling,weakness,Incomplete +CWE-771,Missing Reference to Active Allocated Resource,weakness,Incomplete +CWE-772,Missing Release of Resource after Effective Lifetime,weakness,Draft +CWE-773,Missing Reference to Active File Descriptor or Handle,weakness,Incomplete +CWE-774,Allocation of File Descriptors or Handles Without Limits or Throttling,weakness,Incomplete +CWE-775,Missing Release of File Descriptor or Handle after Effective Lifetime,weakness,Incomplete +CWE-776,Improper Restriction of Recursive Entity References in DTDs ('XML Entity Expansion'),weakness,Draft +CWE-777,Regular Expression without Anchors,weakness,Incomplete +CWE-778,Insufficient Logging,weakness,Draft +CWE-779,Logging of Excessive Data,weakness,Draft +CWE-780,Use of RSA Algorithm without OAEP,weakness,Incomplete +CWE-781,Improper Address Validation in IOCTL with METHOD_NEITHER I/O Control Code,weakness,Draft +CWE-782,Exposed IOCTL with Insufficient Access Control,weakness,Draft +CWE-783,Operator Precedence Logic Error,weakness,Draft +CWE-784,Reliance on Cookies without Validation and Integrity Checking in a Security Decision,weakness,Draft +CWE-785,Use of Path Manipulation Function without Maximum-sized Buffer,weakness,Incomplete +CWE-786,Access of Memory Location Before Start of Buffer,weakness,Incomplete +CWE-787,Out-of-bounds Write,weakness,Draft +CWE-788,Access of Memory Location After End of Buffer,weakness,Incomplete +CWE-789,Memory Allocation with Excessive Size Value,weakness,Draft +CWE-790,Improper Filtering of Special Elements,weakness,Incomplete +CWE-791,Incomplete Filtering of Special Elements,weakness,Incomplete +CWE-792,Incomplete Filtering of One or More Instances of Special Elements,weakness,Incomplete +CWE-793,Only Filtering One Instance of a Special Element,weakness,Incomplete +CWE-794,Incomplete Filtering of Multiple Instances of Special Elements,weakness,Incomplete +CWE-795,Only Filtering Special Elements at a Specified Location,weakness,Incomplete +CWE-796,Only Filtering Special Elements Relative to a Marker,weakness,Incomplete +CWE-797,Only Filtering Special Elements at an Absolute Position,weakness,Incomplete +CWE-798,Use of Hard-coded Credentials,weakness,Draft +CWE-799,Improper Control of Interaction Frequency,weakness,Incomplete +CWE-800,Weaknesses in the 2010 CWE/SANS Top 25 Most Dangerous Programming Errors,view,Obsolete +CWE-801,2010 Top 25 - Insecure Interaction Between Components,category,Obsolete +CWE-802,2010 Top 25 - Risky Resource Management,category,Obsolete +CWE-803,2010 Top 25 - Porous Defenses,category,Obsolete +CWE-804,Guessable CAPTCHA,weakness,Incomplete +CWE-805,Buffer Access with Incorrect Length Value,weakness,Incomplete +CWE-806,Buffer Access Using Size of Source Buffer,weakness,Incomplete +CWE-807,Reliance on Untrusted Inputs in a Security Decision,weakness,Incomplete +CWE-808,2010 Top 25 - Weaknesses On the Cusp,category,Obsolete +CWE-809,Weaknesses in OWASP Top Ten (2010),view,Obsolete +CWE-810,OWASP Top Ten 2010 Category A1 - Injection,category,Obsolete +CWE-811,OWASP Top Ten 2010 Category A2 - Cross-Site Scripting (XSS),category,Obsolete +CWE-812,OWASP Top Ten 2010 Category A3 - Broken Authentication and Session Management,category,Obsolete +CWE-813,OWASP Top Ten 2010 Category A4 - Insecure Direct Object References,category,Obsolete +CWE-814,OWASP Top Ten 2010 Category A5 - Cross-Site Request Forgery(CSRF),category,Obsolete +CWE-815,OWASP Top Ten 2010 Category A6 - Security Misconfiguration,category,Obsolete +CWE-816,OWASP Top Ten 2010 Category A7 - Insecure Cryptographic Storage,category,Obsolete +CWE-817,OWASP Top Ten 2010 Category A8 - Failure to Restrict URL Access,category,Obsolete +CWE-818,OWASP Top Ten 2010 Category A9 - Insufficient Transport Layer Protection,category,Obsolete +CWE-819,OWASP Top Ten 2010 Category A10 - Unvalidated Redirects and Forwards,category,Obsolete +CWE-820,Missing Synchronization,weakness,Incomplete +CWE-821,Incorrect Synchronization,weakness,Incomplete +CWE-822,Untrusted Pointer Dereference,weakness,Incomplete +CWE-823,Use of Out-of-range Pointer Offset,weakness,Incomplete +CWE-824,Access of Uninitialized Pointer,weakness,Incomplete +CWE-825,Expired Pointer Dereference,weakness,Incomplete +CWE-826,Premature Release of Resource During Expected Lifetime,weakness,Incomplete +CWE-827,Improper Control of Document Type Definition,weakness,Incomplete +CWE-828,Signal Handler with Functionality that is not Asynchronous-Safe,weakness,Incomplete +CWE-829,Inclusion of Functionality from Untrusted Control Sphere,weakness,Incomplete +CWE-830,Inclusion of Web Functionality from an Untrusted Source,weakness,Incomplete +CWE-831,Signal Handler Function Associated with Multiple Signals,weakness,Incomplete +CWE-832,Unlock of a Resource that is not Locked,weakness,Incomplete +CWE-833,Deadlock,weakness,Incomplete +CWE-834,Excessive Iteration,weakness,Incomplete +CWE-835,Loop with Unreachable Exit Condition ('Infinite Loop'),weakness,Incomplete +CWE-836,Use of Password Hash Instead of Password for Authentication,weakness,Incomplete +CWE-837,"Improper Enforcement of a Single, Unique Action",weakness,Incomplete +CWE-838,Inappropriate Encoding for Output Context,weakness,Incomplete +CWE-839,Numeric Range Comparison Without Minimum Check,weakness,Incomplete +CWE-840,Business Logic Errors,category,Incomplete +CWE-841,Improper Enforcement of Behavioral Workflow,weakness,Incomplete +CWE-842,Placement of User into Incorrect Group,weakness,Incomplete +CWE-843,Access of Resource Using Incompatible Type ('Type Confusion'),weakness,Incomplete +CWE-844,Weaknesses Addressed by The CERT Oracle Secure Coding Standard for Java (2011),view,Obsolete +CWE-845,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 2 - Input Validation and Data Sanitization (IDS),category,Obsolete +CWE-846,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 3 - Declarations and Initialization (DCL),category,Obsolete +CWE-847,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 4 - Expressions (EXP),category,Obsolete +CWE-848,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 5 - Numeric Types and Operations (NUM),category,Obsolete +CWE-849,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 6 - Object Orientation (OBJ),category,Obsolete +CWE-850,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 7 - Methods (MET),category,Obsolete +CWE-851,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 8 - Exceptional Behavior (ERR),category,Obsolete +CWE-852,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 9 - Visibility and Atomicity (VNA),category,Obsolete +CWE-853,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 10 - Locking (LCK),category,Obsolete +CWE-854,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 11 - Thread APIs (THI),category,Obsolete +CWE-855,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 12 - Thread Pools (TPS),category,Obsolete +CWE-856,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 13 - Thread-Safety Miscellaneous (TSM),category,Obsolete +CWE-857,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 14 - Input Output (FIO),category,Obsolete +CWE-858,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 15 - Serialization (SER),category,Obsolete +CWE-859,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 16 - Platform Security (SEC),category,Obsolete +CWE-860,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 17 - Runtime Environment (ENV),category,Obsolete +CWE-861,The CERT Oracle Secure Coding Standard for Java (2011) Chapter 18 - Miscellaneous (MSC),category,Obsolete +CWE-862,Missing Authorization,weakness,Incomplete +CWE-863,Incorrect Authorization,weakness,Incomplete +CWE-864,2011 Top 25 - Insecure Interaction Between Components,category,Obsolete +CWE-865,2011 Top 25 - Risky Resource Management,category,Obsolete +CWE-866,2011 Top 25 - Porous Defenses,category,Obsolete +CWE-867,2011 Top 25 - Weaknesses On the Cusp,category,Obsolete +CWE-868,Weaknesses Addressed by the SEI CERT C++ Coding Standard (2016 Version),view,Obsolete +CWE-869,CERT C++ Secure Coding Section 01 - Preprocessor (PRE),category,Incomplete +CWE-870,CERT C++ Secure Coding Section 02 - Declarations and Initialization (DCL),category,Incomplete +CWE-871,CERT C++ Secure Coding Section 03 - Expressions (EXP),category,Incomplete +CWE-872,CERT C++ Secure Coding Section 04 - Integers (INT),category,Incomplete +CWE-873,CERT C++ Secure Coding Section 05 - Floating Point Arithmetic (FLP),category,Incomplete +CWE-874,CERT C++ Secure Coding Section 06 - Arrays and the STL (ARR),category,Incomplete +CWE-875,CERT C++ Secure Coding Section 07 - Characters and Strings (STR),category,Incomplete +CWE-876,CERT C++ Secure Coding Section 08 - Memory Management (MEM),category,Incomplete +CWE-877,CERT C++ Secure Coding Section 09 - Input Output (FIO),category,Incomplete +CWE-878,CERT C++ Secure Coding Section 10 - Environment (ENV),category,Incomplete +CWE-879,CERT C++ Secure Coding Section 11 - Signals (SIG),category,Incomplete +CWE-880,CERT C++ Secure Coding Section 12 - Exceptions and Error Handling (ERR),category,Incomplete +CWE-881,CERT C++ Secure Coding Section 13 - Object Oriented Programming (OOP),category,Incomplete +CWE-882,CERT C++ Secure Coding Section 14 - Concurrency (CON),category,Incomplete +CWE-883,CERT C++ Secure Coding Section 49 - Miscellaneous (MSC),category,Incomplete +CWE-884,CWE Cross-section,view,Incomplete +CWE-885,SFP Primary Cluster: Risky Values,category,Incomplete +CWE-886,SFP Primary Cluster: Unused entities,category,Incomplete +CWE-887,SFP Primary Cluster: API,category,Incomplete +CWE-888,Software Fault Pattern (SFP) Clusters,view,Incomplete +CWE-889,SFP Primary Cluster: Exception Management,category,Incomplete +CWE-890,SFP Primary Cluster: Memory Access,category,Incomplete +CWE-891,SFP Primary Cluster: Memory Management,category,Incomplete +CWE-892,SFP Primary Cluster: Resource Management,category,Incomplete +CWE-893,SFP Primary Cluster: Path Resolution,category,Incomplete +CWE-894,SFP Primary Cluster: Synchronization,category,Incomplete +CWE-895,SFP Primary Cluster: Information Leak,category,Incomplete +CWE-896,SFP Primary Cluster: Tainted Input,category,Incomplete +CWE-897,SFP Primary Cluster: Entry Points,category,Incomplete +CWE-898,SFP Primary Cluster: Authentication,category,Incomplete +CWE-899,SFP Primary Cluster: Access Control,category,Incomplete +CWE-900,Weaknesses in the 2011 CWE/SANS Top 25 Most Dangerous Software Errors,view,Obsolete +CWE-901,SFP Primary Cluster: Privilege,category,Incomplete +CWE-902,SFP Primary Cluster: Channel,category,Incomplete +CWE-903,SFP Primary Cluster: Cryptography,category,Incomplete +CWE-904,SFP Primary Cluster: Malware,category,Incomplete +CWE-905,SFP Primary Cluster: Predictability,category,Incomplete +CWE-906,SFP Primary Cluster: UI,category,Incomplete +CWE-907,SFP Primary Cluster: Other,category,Incomplete +CWE-908,Use of Uninitialized Resource,weakness,Incomplete +CWE-909,Missing Initialization of Resource,weakness,Incomplete +CWE-910,Use of Expired File Descriptor,weakness,Incomplete +CWE-911,Improper Update of Reference Count,weakness,Incomplete +CWE-912,Hidden Functionality,weakness,Incomplete +CWE-913,Improper Control of Dynamically-Managed Code Resources,weakness,Incomplete +CWE-914,Improper Control of Dynamically-Identified Variables,weakness,Incomplete +CWE-915,Improperly Controlled Modification of Dynamically-Determined Object Attributes,weakness,Incomplete +CWE-916,Use of Password Hash With Insufficient Computational Effort,weakness,Incomplete +CWE-917,Improper Neutralization of Special Elements used in an Expression Language Statement ('Expression Language Injection'),weakness,Incomplete +CWE-918,Server-Side Request Forgery (SSRF),weakness,Incomplete +CWE-919,Weaknesses in Mobile Applications,view,Incomplete +CWE-920,Improper Restriction of Power Consumption,weakness,Incomplete +CWE-921,Storage of Sensitive Data in a Mechanism without Access Control,weakness,Incomplete +CWE-922,Insecure Storage of Sensitive Information,weakness,Incomplete +CWE-923,Improper Restriction of Communication Channel to Intended Endpoints,weakness,Incomplete +CWE-924,Improper Enforcement of Message Integrity During Transmission in a Communication Channel,weakness,Incomplete +CWE-925,Improper Verification of Intent by Broadcast Receiver,weakness,Incomplete +CWE-926,Improper Export of Android Application Components,weakness,Incomplete +CWE-927,Use of Implicit Intent for Sensitive Communication,weakness,Incomplete +CWE-928,Weaknesses in OWASP Top Ten (2013),view,Obsolete +CWE-929,OWASP Top Ten 2013 Category A1 - Injection,category,Obsolete +CWE-930,OWASP Top Ten 2013 Category A2 - Broken Authentication and Session Management,category,Obsolete +CWE-931,OWASP Top Ten 2013 Category A3 - Cross-Site Scripting (XSS),category,Obsolete +CWE-932,OWASP Top Ten 2013 Category A4 - Insecure Direct Object References,category,Obsolete +CWE-933,OWASP Top Ten 2013 Category A5 - Security Misconfiguration,category,Obsolete +CWE-934,OWASP Top Ten 2013 Category A6 - Sensitive Data Exposure,category,Obsolete +CWE-935,OWASP Top Ten 2013 Category A7 - Missing Function Level Access Control,category,Obsolete +CWE-936,OWASP Top Ten 2013 Category A8 - Cross-Site Request Forgery (CSRF),category,Obsolete +CWE-937,OWASP Top Ten 2013 Category A9 - Using Components with Known Vulnerabilities,category,Obsolete +CWE-938,OWASP Top Ten 2013 Category A10 - Unvalidated Redirects and Forwards,category,Obsolete +CWE-939,Improper Authorization in Handler for Custom URL Scheme,weakness,Incomplete +CWE-940,Improper Verification of Source of a Communication Channel,weakness,Incomplete +CWE-941,Incorrectly Specified Destination in a Communication Channel,weakness,Incomplete +CWE-942,Permissive Cross-domain Security Policy with Untrusted Domains,weakness,Incomplete +CWE-943,Improper Neutralization of Special Elements in Data Query Logic,weakness,Incomplete +CWE-944,SFP Secondary Cluster: Access Management,category,Incomplete +CWE-945,SFP Secondary Cluster: Insecure Resource Access,category,Incomplete +CWE-946,SFP Secondary Cluster: Insecure Resource Permissions,category,Incomplete +CWE-947,SFP Secondary Cluster: Authentication Bypass,category,Incomplete +CWE-948,SFP Secondary Cluster: Digital Certificate,category,Incomplete +CWE-949,SFP Secondary Cluster: Faulty Endpoint Authentication,category,Incomplete +CWE-950,SFP Secondary Cluster: Hardcoded Sensitive Data,category,Incomplete +CWE-951,SFP Secondary Cluster: Insecure Authentication Policy,category,Incomplete +CWE-952,SFP Secondary Cluster: Missing Authentication,category,Incomplete +CWE-953,SFP Secondary Cluster: Missing Endpoint Authentication,category,Incomplete +CWE-954,SFP Secondary Cluster: Multiple Binds to the Same Port,category,Incomplete +CWE-955,SFP Secondary Cluster: Unrestricted Authentication,category,Incomplete +CWE-956,SFP Secondary Cluster: Channel Attack,category,Incomplete +CWE-957,SFP Secondary Cluster: Protocol Error,category,Incomplete +CWE-958,SFP Secondary Cluster: Broken Cryptography,category,Incomplete +CWE-959,SFP Secondary Cluster: Weak Cryptography,category,Incomplete +CWE-960,SFP Secondary Cluster: Ambiguous Exception Type,category,Incomplete +CWE-961,SFP Secondary Cluster: Incorrect Exception Behavior,category,Incomplete +CWE-962,SFP Secondary Cluster: Unchecked Status Condition,category,Incomplete +CWE-963,SFP Secondary Cluster: Exposed Data,category,Incomplete +CWE-964,SFP Secondary Cluster: Exposure Temporary File,category,Incomplete +CWE-965,SFP Secondary Cluster: Insecure Session Management,category,Incomplete +CWE-966,SFP Secondary Cluster: Other Exposures,category,Incomplete +CWE-967,SFP Secondary Cluster: State Disclosure,category,Incomplete +CWE-968,SFP Secondary Cluster: Covert Channel,category,Incomplete +CWE-969,SFP Secondary Cluster: Faulty Memory Release,category,Incomplete +CWE-970,SFP Secondary Cluster: Faulty Buffer Access,category,Incomplete +CWE-971,SFP Secondary Cluster: Faulty Pointer Use,category,Incomplete +CWE-972,SFP Secondary Cluster: Faulty String Expansion,category,Incomplete +CWE-973,SFP Secondary Cluster: Improper NULL Termination,category,Incomplete +CWE-974,SFP Secondary Cluster: Incorrect Buffer Length Computation,category,Incomplete +CWE-975,SFP Secondary Cluster: Architecture,category,Incomplete +CWE-976,SFP Secondary Cluster: Compiler,category,Incomplete +CWE-977,SFP Secondary Cluster: Design,category,Incomplete +CWE-978,SFP Secondary Cluster: Implementation,category,Incomplete +CWE-979,SFP Secondary Cluster: Failed Chroot Jail,category,Incomplete +CWE-980,SFP Secondary Cluster: Link in Resource Name Resolution,category,Incomplete +CWE-981,SFP Secondary Cluster: Path Traversal,category,Incomplete +CWE-982,SFP Secondary Cluster: Failure to Release Resource,category,Incomplete +CWE-983,SFP Secondary Cluster: Faulty Resource Use,category,Incomplete +CWE-984,SFP Secondary Cluster: Life Cycle,category,Incomplete +CWE-985,SFP Secondary Cluster: Unrestricted Consumption,category,Incomplete +CWE-986,SFP Secondary Cluster: Missing Lock,category,Incomplete +CWE-987,SFP Secondary Cluster: Multiple Locks/Unlocks,category,Incomplete +CWE-988,SFP Secondary Cluster: Race Condition Window,category,Incomplete +CWE-989,SFP Secondary Cluster: Unrestricted Lock,category,Incomplete +CWE-990,SFP Secondary Cluster: Tainted Input to Command,category,Incomplete +CWE-991,SFP Secondary Cluster: Tainted Input to Environment,category,Incomplete +CWE-992,SFP Secondary Cluster: Faulty Input Transformation,category,Incomplete +CWE-993,SFP Secondary Cluster: Incorrect Input Handling,category,Incomplete +CWE-994,SFP Secondary Cluster: Tainted Input to Variable,category,Incomplete +CWE-995,SFP Secondary Cluster: Feature,category,Incomplete +CWE-996,SFP Secondary Cluster: Security,category,Incomplete +CWE-997,SFP Secondary Cluster: Information Loss,category,Incomplete +CWE-998,SFP Secondary Cluster: Glitch in Computation,category,Incomplete +CWE-999,DEPRECATED: Weaknesses without Software Fault Patterns,view,Deprecated +CWE-1000,Research Concepts,view,Draft +CWE-1001,SFP Secondary Cluster: Use of an Improper API,category,Incomplete +CWE-1002,SFP Secondary Cluster: Unexpected Entry Points,category,Incomplete +CWE-1003,Weaknesses for Simplified Mapping of Published Vulnerabilities,view,Incomplete +CWE-1004,Sensitive Cookie Without 'HttpOnly' Flag,weakness,Incomplete +CWE-1005,7PK - Input Validation and Representation,category,Draft +CWE-1006,Bad Coding Practices,category,Draft +CWE-1007,Insufficient Visual Distinction of Homoglyphs Presented to User,weakness,Incomplete +CWE-1008,Architectural Concepts,view,Incomplete +CWE-1009,Audit,category,Draft +CWE-1010,Authenticate Actors,category,Draft +CWE-1011,Authorize Actors,category,Draft +CWE-1012,Cross Cutting,category,Draft +CWE-1013,Encrypt Data,category,Draft +CWE-1014,Identify Actors,category,Draft +CWE-1015,Limit Access,category,Draft +CWE-1016,Limit Exposure,category,Draft +CWE-1017,Lock Computer,category,Draft +CWE-1018,Manage User Sessions,category,Draft +CWE-1019,Validate Inputs,category,Draft +CWE-1020,Verify Message Integrity,category,Draft +CWE-1021,Improper Restriction of Rendered UI Layers or Frames,weakness,Incomplete +CWE-1022,Use of Web Link to Untrusted Target with window.opener Access,weakness,Incomplete +CWE-1023,Incomplete Comparison with Missing Factors,weakness,Incomplete +CWE-1024,Comparison of Incompatible Types,weakness,Incomplete +CWE-1025,Comparison Using Wrong Factors,weakness,Incomplete +CWE-1026,Weaknesses in OWASP Top Ten (2017),view,Incomplete +CWE-1027,OWASP Top Ten 2017 Category A1 - Injection,category,Incomplete +CWE-1028,OWASP Top Ten 2017 Category A2 - Broken Authentication,category,Incomplete +CWE-1029,OWASP Top Ten 2017 Category A3 - Sensitive Data Exposure,category,Incomplete +CWE-1030,OWASP Top Ten 2017 Category A4 - XML External Entities (XXE),category,Incomplete +CWE-1031,OWASP Top Ten 2017 Category A5 - Broken Access Control,category,Incomplete +CWE-1032,OWASP Top Ten 2017 Category A6 - Security Misconfiguration,category,Incomplete +CWE-1033,OWASP Top Ten 2017 Category A7 - Cross-Site Scripting (XSS),category,Incomplete +CWE-1034,OWASP Top Ten 2017 Category A8 - Insecure Deserialization,category,Incomplete +CWE-1035,OWASP Top Ten 2017 Category A9 - Using Components with Known Vulnerabilities,category,Incomplete +CWE-1036,OWASP Top Ten 2017 Category A10 - Insufficient Logging & Monitoring,category,Incomplete +CWE-1037,Processor Optimization Removal or Modification of Security-critical Code,weakness,Incomplete +CWE-1038,Insecure Automated Optimizations,weakness,Draft +CWE-1039,Inadequate Detection or Handling of Adversarial Input Perturbations in Automated Recognition Mechanism,weakness,Incomplete +CWE-1040,Quality Weaknesses with Indirect Security Impacts,view,Incomplete +CWE-1041,Use of Redundant Code,weakness,Incomplete +CWE-1042,Static Member Data Element outside of a Singleton Class Element,weakness,Incomplete +CWE-1043,Data Element Aggregating an Excessively Large Number of Non-Primitive Elements,weakness,Incomplete +CWE-1044,Architecture with Number of Horizontal Layers Outside of Expected Range,weakness,Incomplete +CWE-1045,Parent Class with a Virtual Destructor and a Child Class without a Virtual Destructor,weakness,Incomplete +CWE-1046,Creation of Immutable Text Using String Concatenation,weakness,Incomplete +CWE-1047,Modules with Circular Dependencies,weakness,Incomplete +CWE-1048,Invokable Control Element with Large Number of Outward Calls,weakness,Incomplete +CWE-1049,Excessive Data Query Operations in a Large Data Table,weakness,Incomplete +CWE-1050,Excessive Platform Resource Consumption within a Loop,weakness,Incomplete +CWE-1051,Initialization with Hard-Coded Network Resource Configuration Data,weakness,Incomplete +CWE-1052,Excessive Use of Hard-Coded Literals in Initialization,weakness,Incomplete +CWE-1053,Missing Documentation for Design,weakness,Incomplete +CWE-1054,Invocation of a Control Element at an Unnecessarily Deep Horizontal Layer,weakness,Incomplete +CWE-1055,Multiple Inheritance from Concrete Classes,weakness,Incomplete +CWE-1056,Invokable Control Element with Variadic Parameters,weakness,Incomplete +CWE-1057,Data Access Operations Outside of Expected Data Manager Component,weakness,Incomplete +CWE-1058,Invokable Control Element in Multi-Thread Context with non-Final Static Storable or Member Element,weakness,Incomplete +CWE-1059,Insufficient Technical Documentation,weakness,Incomplete +CWE-1060,Excessive Number of Inefficient Server-Side Data Accesses,weakness,Incomplete +CWE-1061,Insufficient Encapsulation,weakness,Incomplete +CWE-1062,Parent Class with References to Child Class,weakness,Incomplete +CWE-1063,Creation of Class Instance within a Static Code Block,weakness,Incomplete +CWE-1064,Invokable Control Element with Signature Containing an Excessive Number of Parameters,weakness,Incomplete +CWE-1065,Runtime Resource Management Control Element in a Component Built to Run on Application Servers,weakness,Incomplete +CWE-1066,Missing Serialization Control Element,weakness,Incomplete +CWE-1067,Excessive Execution of Sequential Searches of Data Resource,weakness,Incomplete +CWE-1068,Inconsistency Between Implementation and Documented Design,weakness,Incomplete +CWE-1069,Empty Exception Block,weakness,Incomplete +CWE-1070,Serializable Data Element Containing non-Serializable Item Elements,weakness,Incomplete +CWE-1071,Empty Code Block,weakness,Incomplete +CWE-1072,Data Resource Access without Use of Connection Pooling,weakness,Incomplete +CWE-1073,Non-SQL Invokable Control Element with Excessive Number of Data Resource Accesses,weakness,Incomplete +CWE-1074,Class with Excessively Deep Inheritance,weakness,Incomplete +CWE-1075,Unconditional Control Flow Transfer outside of Switch Block,weakness,Incomplete +CWE-1076,Insufficient Adherence to Expected Conventions,weakness,Incomplete +CWE-1077,Floating Point Comparison with Incorrect Operator,weakness,Incomplete +CWE-1078,Inappropriate Source Code Style or Formatting,weakness,Incomplete +CWE-1079,Parent Class without Virtual Destructor Method,weakness,Incomplete +CWE-1080,Source Code File with Excessive Number of Lines of Code,weakness,Incomplete +CWE-1081,Entries with Maintenance Notes,view,Draft +CWE-1082,Class Instance Self Destruction Control Element,weakness,Incomplete +CWE-1083,Data Access from Outside Expected Data Manager Component,weakness,Incomplete +CWE-1084,Invokable Control Element with Excessive File or Data Access Operations,weakness,Incomplete +CWE-1085,Invokable Control Element with Excessive Volume of Commented-out Code,weakness,Incomplete +CWE-1086,Class with Excessive Number of Child Classes,weakness,Incomplete +CWE-1087,Class with Virtual Method without a Virtual Destructor,weakness,Incomplete +CWE-1088,Synchronous Access of Remote Resource without Timeout,weakness,Incomplete +CWE-1089,Large Data Table with Excessive Number of Indices,weakness,Incomplete +CWE-1090,Method Containing Access of a Member Element from Another Class,weakness,Incomplete +CWE-1091,Use of Object without Invoking Destructor Method,weakness,Incomplete +CWE-1092,Use of Same Invokable Control Element in Multiple Architectural Layers,weakness,Incomplete +CWE-1093,Excessively Complex Data Representation,weakness,Incomplete +CWE-1094,Excessive Index Range Scan for a Data Resource,weakness,Incomplete +CWE-1095,Loop Condition Value Update within the Loop,weakness,Incomplete +CWE-1096,Singleton Class Instance Creation without Proper Locking or Synchronization,weakness,Incomplete +CWE-1097,Persistent Storable Data Element without Associated Comparison Control Element,weakness,Incomplete +CWE-1098,Data Element containing Pointer Item without Proper Copy Control Element,weakness,Incomplete +CWE-1099,Inconsistent Naming Conventions for Identifiers,weakness,Incomplete +CWE-1100,Insufficient Isolation of System-Dependent Functions,weakness,Incomplete +CWE-1101,Reliance on Runtime Component in Generated Code,weakness,Incomplete +CWE-1102,Reliance on Machine-Dependent Data Representation,weakness,Incomplete +CWE-1103,Use of Platform-Dependent Third Party Components,weakness,Incomplete +CWE-1104,Use of Unmaintained Third Party Components,weakness,Incomplete +CWE-1105,Insufficient Encapsulation of Machine-Dependent Functionality,weakness,Incomplete +CWE-1106,Insufficient Use of Symbolic Constants,weakness,Incomplete +CWE-1107,Insufficient Isolation of Symbolic Constant Definitions,weakness,Incomplete +CWE-1108,Excessive Reliance on Global Variables,weakness,Incomplete +CWE-1109,Use of Same Variable for Multiple Purposes,weakness,Incomplete +CWE-1110,Incomplete Design Documentation,weakness,Incomplete +CWE-1111,Incomplete I/O Documentation,weakness,Incomplete +CWE-1112,Incomplete Documentation of Program Execution,weakness,Incomplete +CWE-1113,Inappropriate Comment Style,weakness,Incomplete +CWE-1114,Inappropriate Whitespace Style,weakness,Incomplete +CWE-1115,Source Code Element without Standard Prologue,weakness,Incomplete +CWE-1116,Inaccurate Source Code Comments,weakness,Incomplete +CWE-1117,Callable with Insufficient Behavioral Summary,weakness,Incomplete +CWE-1118,Insufficient Documentation of Error Handling Techniques,weakness,Incomplete +CWE-1119,Excessive Use of Unconditional Branching,weakness,Incomplete +CWE-1120,Excessive Code Complexity,weakness,Incomplete +CWE-1121,Excessive McCabe Cyclomatic Complexity,weakness,Incomplete +CWE-1122,Excessive Halstead Complexity,weakness,Incomplete +CWE-1123,Excessive Use of Self-Modifying Code,weakness,Incomplete +CWE-1124,Excessively Deep Nesting,weakness,Incomplete +CWE-1125,Excessive Attack Surface,weakness,Incomplete +CWE-1126,Declaration of Variable with Unnecessarily Wide Scope,weakness,Incomplete +CWE-1127,Compilation with Insufficient Warnings or Errors,weakness,Incomplete +CWE-1128,CISQ Quality Measures (2016),view,Incomplete +CWE-1129,CISQ Quality Measures (2016) - Reliability,category,Draft +CWE-1130,CISQ Quality Measures (2016) - Maintainability,category,Draft +CWE-1131,CISQ Quality Measures (2016) - Security,category,Draft +CWE-1132,CISQ Quality Measures (2016) - Performance Efficiency,category,Draft +CWE-1133,Weaknesses Addressed by the SEI CERT Oracle Coding Standard for Java,view,Stable +CWE-1134,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 00. Input Validation and Data Sanitization (IDS),category,Stable +CWE-1135,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 01. Declarations and Initialization (DCL),category,Stable +CWE-1136,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 02. Expressions (EXP),category,Stable +CWE-1137,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 03. Numeric Types and Operations (NUM),category,Stable +CWE-1138,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 04. Characters and Strings (STR),category,Stable +CWE-1139,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 05. Object Orientation (OBJ),category,Stable +CWE-1140,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 06. Methods (MET),category,Stable +CWE-1141,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 07. Exceptional Behavior (ERR),category,Stable +CWE-1142,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 08. Visibility and Atomicity (VNA),category,Stable +CWE-1143,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 09. Locking (LCK),category,Stable +CWE-1144,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 10. Thread APIs (THI),category,Stable +CWE-1145,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 11. Thread Pools (TPS),category,Stable +CWE-1146,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 12. Thread-Safety Miscellaneous (TSM),category,Stable +CWE-1147,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 13. Input Output (FIO),category,Stable +CWE-1148,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 14. Serialization (SER),category,Stable +CWE-1149,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 15. Platform Security (SEC),category,Stable +CWE-1150,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 16. Runtime Environment (ENV),category,Stable +CWE-1151,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 17. Java Native Interface (JNI),category,Stable +CWE-1152,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 49. Miscellaneous (MSC),category,Stable +CWE-1153,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 50. Android (DRD),category,Stable +CWE-1154,Weaknesses Addressed by the SEI CERT C Coding Standard,view,Stable +CWE-1155,SEI CERT C Coding Standard - Guidelines 01. Preprocessor (PRE),category,Stable +CWE-1156,SEI CERT C Coding Standard - Guidelines 02. Declarations and Initialization (DCL),category,Stable +CWE-1157,SEI CERT C Coding Standard - Guidelines 03. Expressions (EXP),category,Stable +CWE-1158,SEI CERT C Coding Standard - Guidelines 04. Integers (INT),category,Stable +CWE-1159,SEI CERT C Coding Standard - Guidelines 05. Floating Point (FLP),category,Stable +CWE-1160,SEI CERT C Coding Standard - Guidelines 06. Arrays (ARR),category,Stable +CWE-1161,SEI CERT C Coding Standard - Guidelines 07. Characters and Strings (STR),category,Stable +CWE-1162,SEI CERT C Coding Standard - Guidelines 08. Memory Management (MEM),category,Stable +CWE-1163,SEI CERT C Coding Standard - Guidelines 09. Input Output (FIO),category,Stable +CWE-1164,Irrelevant Code,weakness,Incomplete +CWE-1165,SEI CERT C Coding Standard - Guidelines 10. Environment (ENV),category,Stable +CWE-1166,SEI CERT C Coding Standard - Guidelines 11. Signals (SIG),category,Stable +CWE-1167,SEI CERT C Coding Standard - Guidelines 12. Error Handling (ERR),category,Stable +CWE-1168,SEI CERT C Coding Standard - Guidelines 13. Application Programming Interfaces (API),category,Stable +CWE-1169,SEI CERT C Coding Standard - Guidelines 14. Concurrency (CON),category,Stable +CWE-1170,SEI CERT C Coding Standard - Guidelines 48. Miscellaneous (MSC),category,Stable +CWE-1171,SEI CERT C Coding Standard - Guidelines 50. POSIX (POS),category,Stable +CWE-1172,SEI CERT C Coding Standard - Guidelines 51. Microsoft Windows (WIN),category,Stable +CWE-1173,Improper Use of Validation Framework,weakness,Draft +CWE-1174,ASP.NET Misconfiguration: Improper Model Validation,weakness,Draft +CWE-1175,SEI CERT Oracle Secure Coding Standard for Java - Guidelines 18. Concurrency (CON),category,Stable +CWE-1176,Inefficient CPU Computation,weakness,Incomplete +CWE-1177,Use of Prohibited Code,weakness,Incomplete +CWE-1178,Weaknesses Addressed by the SEI CERT Perl Coding Standard,view,Stable +CWE-1179,SEI CERT Perl Coding Standard - Guidelines 01. Input Validation and Data Sanitization (IDS),category,Stable +CWE-1180,SEI CERT Perl Coding Standard - Guidelines 02. Declarations and Initialization (DCL),category,Stable +CWE-1181,SEI CERT Perl Coding Standard - Guidelines 03. Expressions (EXP),category,Stable +CWE-1182,SEI CERT Perl Coding Standard - Guidelines 04. Integers (INT),category,Stable +CWE-1183,SEI CERT Perl Coding Standard - Guidelines 05. Strings (STR),category,Stable +CWE-1184,SEI CERT Perl Coding Standard - Guidelines 06. Object-Oriented Programming (OOP),category,Stable +CWE-1185,SEI CERT Perl Coding Standard - Guidelines 07. File Input and Output (FIO),category,Stable +CWE-1186,SEI CERT Perl Coding Standard - Guidelines 50. Miscellaneous (MSC),category,Stable +CWE-1187,DEPRECATED: Use of Uninitialized Resource,weakness,Deprecated +CWE-1188,Initialization of a Resource with an Insecure Default,weakness,Incomplete +CWE-1189,Improper Isolation of Shared Resources on System-on-a-Chip (SoC),weakness,Stable +CWE-1190,DMA Device Enabled Too Early in Boot Phase,weakness,Draft +CWE-1191,On-Chip Debug and Test Interface With Improper Access Control,weakness,Stable +CWE-1192,Improper Identifier for IP Block used in System-On-Chip (SOC),weakness,Draft +CWE-1193,Power-On of Untrusted Execution Core Before Enabling Fabric Access Control,weakness,Draft +CWE-1194,Hardware Design,view,Draft +CWE-1195,Manufacturing and Life Cycle Management Concerns,category,Draft +CWE-1196,Security Flow Issues,category,Draft +CWE-1197,Integration Issues,category,Draft +CWE-1198,Privilege Separation and Access Control Issues,category,Draft +CWE-1199,General Circuit and Logic Design Concerns,category,Draft +CWE-1200,Weaknesses in the 2019 CWE Top 25 Most Dangerous Software Errors,view,Stable +CWE-1201,Core and Compute Issues,category,Draft +CWE-1202,Memory and Storage Issues,category,Draft +CWE-1203,"Peripherals, On-chip Fabric, and Interface/IO Problems",category,Draft +CWE-1204,Generation of Weak Initialization Vector (IV),weakness,Incomplete +CWE-1205,Security Primitives and Cryptography Issues,category,Draft +CWE-1206,"Power, Clock, Thermal, and Reset Concerns",category,Draft +CWE-1207,Debug and Test Problems,category,Draft +CWE-1208,Cross-Cutting Problems,category,Draft +CWE-1209,Failure to Disable Reserved Bits,weakness,Incomplete +CWE-1210,Audit / Logging Errors,category,Draft +CWE-1211,Authentication Errors,category,Draft +CWE-1212,Authorization Errors,category,Draft +CWE-1213,Random Number Issues,category,Draft +CWE-1214,Data Integrity Issues,category,Draft +CWE-1215,Data Validation Issues,category,Draft +CWE-1216,Lockout Mechanism Errors,category,Draft +CWE-1217,User Session Errors,category,Draft +CWE-1218,Memory Buffer Errors,category,Draft +CWE-1219,File Handling Issues,category,Draft +CWE-1220,Insufficient Granularity of Access Control,weakness,Incomplete +CWE-1221,Incorrect Register Defaults or Module Parameters,weakness,Incomplete +CWE-1222,Insufficient Granularity of Address Regions Protected by Register Locks,weakness,Incomplete +CWE-1223,Race Condition for Write-Once Attributes,weakness,Incomplete +CWE-1224,Improper Restriction of Write-Once Bit Fields,weakness,Incomplete +CWE-1225,Documentation Issues,category,Draft +CWE-1226,Complexity Issues,category,Draft +CWE-1227,Encapsulation Issues,category,Draft +CWE-1228,API / Function Errors,category,Draft +CWE-1229,Creation of Emergent Resource,weakness,Incomplete +CWE-1230,Exposure of Sensitive Information Through Metadata,weakness,Incomplete +CWE-1231,Improper Prevention of Lock Bit Modification,weakness,Stable +CWE-1232,Improper Lock Behavior After Power State Transition,weakness,Incomplete +CWE-1233,Security-Sensitive Hardware Controls with Missing Lock Bit Protection,weakness,Stable +CWE-1234,Hardware Internal or Debug Modes Allow Override of Locks,weakness,Incomplete +CWE-1235,Incorrect Use of Autoboxing and Unboxing for Performance Critical Operations,weakness,Incomplete +CWE-1236,Improper Neutralization of Formula Elements in a CSV File,weakness,Incomplete +CWE-1237,SFP Primary Cluster: Faulty Resource Release,category,Incomplete +CWE-1238,SFP Primary Cluster: Failure to Release Memory,category,Incomplete +CWE-1239,Improper Zeroization of Hardware Register,weakness,Draft +CWE-1240,Use of a Cryptographic Primitive with a Risky Implementation,weakness,Draft +CWE-1241,Use of Predictable Algorithm in Random Number Generator,weakness,Draft +CWE-1242,Inclusion of Undocumented Features or Chicken Bits,weakness,Incomplete +CWE-1243,Sensitive Non-Volatile Information Not Protected During Debug,weakness,Incomplete +CWE-1244,Internal Asset Exposed to Unsafe Debug Access Level or State,weakness,Stable +CWE-1245,Improper Finite State Machines (FSMs) in Hardware Logic,weakness,Incomplete +CWE-1246,Improper Write Handling in Limited-write Non-Volatile Memories,weakness,Incomplete +CWE-1247,Improper Protection Against Voltage and Clock Glitches,weakness,Stable +CWE-1248,Semiconductor Defects in Hardware Logic with Security-Sensitive Implications,weakness,Incomplete +CWE-1249,Application-Level Admin Tool with Inconsistent View of Underlying Operating System,weakness,Incomplete +CWE-1250,Improper Preservation of Consistency Between Independent Representations of Shared State,weakness,Incomplete +CWE-1251,Mirrored Regions with Different Values,weakness,Incomplete +CWE-1252,CPU Hardware Not Configured to Support Exclusivity of Write and Execute Operations,weakness,Incomplete +CWE-1253,Incorrect Selection of Fuse Values,weakness,Draft +CWE-1254,Incorrect Comparison Logic Granularity,weakness,Draft +CWE-1255,Comparison Logic is Vulnerable to Power Side-Channel Attacks,weakness,Draft +CWE-1256,Improper Restriction of Software Interfaces to Hardware Features,weakness,Stable +CWE-1257,Improper Access Control Applied to Mirrored or Aliased Memory Regions,weakness,Incomplete +CWE-1258,Exposure of Sensitive System Information Due to Uncleared Debug Information,weakness,Draft +CWE-1259,Improper Restriction of Security Token Assignment,weakness,Incomplete +CWE-1260,Improper Handling of Overlap Between Protected Memory Ranges,weakness,Stable +CWE-1261,Improper Handling of Single Event Upsets,weakness,Draft +CWE-1262,Improper Access Control for Register Interface,weakness,Stable +CWE-1263,Improper Physical Access Control,weakness,Incomplete +CWE-1264,Hardware Logic with Insecure De-Synchronization between Control and Data Channels,weakness,Incomplete +CWE-1265,Unintended Reentrant Invocation of Non-reentrant Code Via Nested Calls,weakness,Draft +CWE-1266,Improper Scrubbing of Sensitive Data from Decommissioned Device,weakness,Incomplete +CWE-1267,Policy Uses Obsolete Encoding,weakness,Draft +CWE-1268,Policy Privileges are not Assigned Consistently Between Control and Data Agents,weakness,Draft +CWE-1269,Product Released in Non-Release Configuration,weakness,Incomplete +CWE-1270,Generation of Incorrect Security Tokens,weakness,Incomplete +CWE-1271,Uninitialized Value on Reset for Registers Holding Security Settings,weakness,Incomplete +CWE-1272,Sensitive Information Uncleared Before Debug/Power State Transition,weakness,Stable +CWE-1273,Device Unlock Credential Sharing,weakness,Incomplete +CWE-1274,Improper Access Control for Volatile Memory Containing Boot Code,weakness,Stable +CWE-1275,Sensitive Cookie with Improper SameSite Attribute,weakness,Incomplete +CWE-1276,Hardware Child Block Incorrectly Connected to Parent System,weakness,Incomplete +CWE-1277,Firmware Not Updateable,weakness,Draft +CWE-1278,Missing Protection Against Hardware Reverse Engineering Using Integrated Circuit (IC) Imaging Techniques,weakness,Incomplete +CWE-1279,Cryptographic Operations are run Before Supporting Units are Ready,weakness,Incomplete +CWE-1280,Access Control Check Implemented After Asset is Accessed,weakness,Incomplete +CWE-1281,Sequence of Processor Instructions Leads to Unexpected Behavior,weakness,Incomplete +CWE-1282,Assumed-Immutable Data is Stored in Writable Memory,weakness,Incomplete +CWE-1283,Mutable Attestation or Measurement Reporting Data,weakness,Incomplete +CWE-1284,Improper Validation of Specified Quantity in Input,weakness,Incomplete +CWE-1285,"Improper Validation of Specified Index, Position, or Offset in Input",weakness,Incomplete +CWE-1286,Improper Validation of Syntactic Correctness of Input,weakness,Incomplete +CWE-1287,Improper Validation of Specified Type of Input,weakness,Incomplete +CWE-1288,Improper Validation of Consistency within Input,weakness,Incomplete +CWE-1289,Improper Validation of Unsafe Equivalence in Input,weakness,Incomplete +CWE-1290,Incorrect Decoding of Security Identifiers,weakness,Incomplete +CWE-1291,Public Key Re-Use for Signing both Debug and Production Code,weakness,Draft +CWE-1292,Incorrect Conversion of Security Identifiers,weakness,Draft +CWE-1293,Missing Source Correlation of Multiple Independent Data,weakness,Draft +CWE-1294,Insecure Security Identifier Mechanism,weakness,Incomplete +CWE-1295,Debug Messages Revealing Unnecessary Information,weakness,Incomplete +CWE-1296,Incorrect Chaining or Granularity of Debug Components,weakness,Incomplete +CWE-1297,Unprotected Confidential Information on Device is Accessible by OSAT Vendors,weakness,Incomplete +CWE-1298,Hardware Logic Contains Race Conditions,weakness,Draft +CWE-1299,Missing Protection Mechanism for Alternate Hardware Interface,weakness,Draft +CWE-1300,Improper Protection of Physical Side Channels,weakness,Stable +CWE-1301,Insufficient or Incomplete Data Removal within Hardware Component,weakness,Incomplete +CWE-1302,Missing Source Identifier in Entity Transactions on a System-On-Chip (SOC),weakness,Incomplete +CWE-1303,Non-Transparent Sharing of Microarchitectural Resources,weakness,Draft +CWE-1304,Improperly Preserved Integrity of Hardware Configuration State During a Power Save/Restore Operation,weakness,Draft +CWE-1305,CISQ Quality Measures (2020),view,Incomplete +CWE-1306,CISQ Quality Measures - Reliability,category,Incomplete +CWE-1307,CISQ Quality Measures - Maintainability,category,Incomplete +CWE-1308,CISQ Quality Measures - Security,category,Incomplete +CWE-1309,CISQ Quality Measures - Efficiency,category,Incomplete +CWE-1310,Missing Ability to Patch ROM Code,weakness,Draft +CWE-1311,Improper Translation of Security Attributes by Fabric Bridge,weakness,Draft +CWE-1312,Missing Protection for Mirrored Regions in On-Chip Fabric Firewall,weakness,Draft +CWE-1313,Hardware Allows Activation of Test or Debug Logic at Runtime,weakness,Draft +CWE-1314,Missing Write Protection for Parametric Data Values,weakness,Draft +CWE-1315,Improper Setting of Bus Controlling Capability in Fabric End-point,weakness,Incomplete +CWE-1316,Fabric-Address Map Allows Programming of Unwarranted Overlaps of Protected and Unprotected Ranges,weakness,Draft +CWE-1317,Improper Access Control in Fabric Bridge,weakness,Draft +CWE-1318,Missing Support for Security Features in On-chip Fabrics or Buses,weakness,Incomplete +CWE-1319,Improper Protection against Electromagnetic Fault Injection (EM-FI),weakness,Incomplete +CWE-1320,Improper Protection for Outbound Error Messages and Alert Signals,weakness,Draft +CWE-1321,Improperly Controlled Modification of Object Prototype Attributes ('Prototype Pollution'),weakness,Incomplete +CWE-1322,"Use of Blocking Code in Single-threaded, Non-blocking Context",weakness,Incomplete +CWE-1323,Improper Management of Sensitive Trace Data,weakness,Draft +CWE-1324,DEPRECATED: Sensitive Information Accessible by Physical Probing of JTAG Interface,weakness,Deprecated +CWE-1325,Improperly Controlled Sequential Memory Allocation,weakness,Incomplete +CWE-1326,Missing Immutable Root of Trust in Hardware,weakness,Draft +CWE-1327,Binding to an Unrestricted IP Address,weakness,Incomplete +CWE-1328,Security Version Number Mutable to Older Versions,weakness,Draft +CWE-1329,Reliance on Component That is Not Updateable,weakness,Incomplete +CWE-1330,Remanent Data Readable after Memory Erase,weakness,Draft +CWE-1331,Improper Isolation of Shared Resources in Network On Chip (NoC),weakness,Stable +CWE-1332,Improper Handling of Faults that Lead to Instruction Skips,weakness,Stable +CWE-1333,Inefficient Regular Expression Complexity,weakness,Draft +CWE-1334,Unauthorized Error Injection Can Degrade Hardware Redundancy,weakness,Draft +CWE-1335,Incorrect Bitwise Shift of Integer,weakness,Draft +CWE-1336,Improper Neutralization of Special Elements Used in a Template Engine,weakness,Incomplete +CWE-1337,Weaknesses in the 2021 CWE Top 25 Most Dangerous Software Weaknesses,view,Stable +CWE-1338,Improper Protections Against Hardware Overheating,weakness,Draft +CWE-1339,Insufficient Precision or Accuracy of a Real Number,weakness,Draft +CWE-1340,CISQ Data Protection Measures,view,Incomplete +CWE-1341,Multiple Releases of Same Resource or Handle,weakness,Incomplete +CWE-1342,Information Exposure through Microarchitectural State after Transient Execution,weakness,Incomplete +CWE-1343,Weaknesses in the 2021 CWE Most Important Hardware Weaknesses List,view,Stable +CWE-1344,Weaknesses in OWASP Top Ten (2021),view,Incomplete +CWE-1345,OWASP Top Ten 2021 Category A01:2021 - Broken Access Control,category,Incomplete +CWE-1346,OWASP Top Ten 2021 Category A02:2021 - Cryptographic Failures,category,Incomplete +CWE-1347,OWASP Top Ten 2021 Category A03:2021 - Injection,category,Incomplete +CWE-1348,OWASP Top Ten 2021 Category A04:2021 - Insecure Design,category,Incomplete +CWE-1349,OWASP Top Ten 2021 Category A05:2021 - Security Misconfiguration,category,Incomplete +CWE-1350,Weaknesses in the 2020 CWE Top 25 Most Dangerous Software Weaknesses,view,Stable +CWE-1351,Improper Handling of Hardware Behavior in Exceptionally Cold Environments,weakness,Incomplete +CWE-1352,OWASP Top Ten 2021 Category A06:2021 - Vulnerable and Outdated Components,category,Incomplete +CWE-1353,OWASP Top Ten 2021 Category A07:2021 - Identification and Authentication Failures,category,Incomplete +CWE-1354,OWASP Top Ten 2021 Category A08:2021 - Software and Data Integrity Failures,category,Incomplete +CWE-1355,OWASP Top Ten 2021 Category A09:2021 - Security Logging and Monitoring Failures,category,Incomplete +CWE-1356,OWASP Top Ten 2021 Category A10:2021 - Server-Side Request Forgery (SSRF),category,Incomplete +CWE-1357,Reliance on Insufficiently Trustworthy Component,weakness,Incomplete +CWE-1358,Weaknesses in SEI ETF Categories of Security Vulnerabilities in ICS,view,Incomplete +CWE-1359,ICS Communications,category,Incomplete +CWE-1360,ICS Dependencies (& Architecture),category,Incomplete +CWE-1361,ICS Supply Chain,category,Incomplete +CWE-1362,ICS Engineering (Constructions/Deployment),category,Incomplete +CWE-1363,ICS Operations (& Maintenance),category,Incomplete +CWE-1364,ICS Communications: Zone Boundary Failures,category,Incomplete +CWE-1365,ICS Communications: Unreliability,category,Incomplete +CWE-1366,ICS Communications: Frail Security in Protocols,category,Incomplete +CWE-1367,ICS Dependencies (& Architecture): External Physical Systems,category,Incomplete +CWE-1368,ICS Dependencies (& Architecture): External Digital Systems,category,Incomplete +CWE-1369,ICS Supply Chain: IT/OT Convergence/Expansion,category,Incomplete +CWE-1370,ICS Supply Chain: Common Mode Frailties,category,Incomplete +CWE-1371,ICS Supply Chain: Poorly Documented or Undocumented Features,category,Incomplete +CWE-1372,ICS Supply Chain: OT Counterfeit and Malicious Corruption,category,Incomplete +CWE-1373,ICS Engineering (Construction/Deployment): Trust Model Problems,category,Incomplete +CWE-1374,ICS Engineering (Construction/Deployment): Maker Breaker Blindness,category,Incomplete +CWE-1375,ICS Engineering (Construction/Deployment): Gaps in Details/Data,category,Incomplete +CWE-1376,ICS Engineering (Construction/Deployment): Security Gaps in Commissioning,category,Incomplete +CWE-1377,ICS Engineering (Construction/Deployment): Inherent Predictability in Design,category,Incomplete +CWE-1378,ICS Operations (& Maintenance): Gaps in obligations and training,category,Incomplete +CWE-1379,ICS Operations (& Maintenance): Human factors in ICS environments,category,Incomplete +CWE-1380,ICS Operations (& Maintenance): Post-analysis changes,category,Incomplete +CWE-1381,ICS Operations (& Maintenance): Exploitable Standard Operational Procedures,category,Incomplete +CWE-1382,ICS Operations (& Maintenance): Emerging Energy Technologies,category,Incomplete +CWE-1383,ICS Operations (& Maintenance): Compliance/Conformance with Regulatory Requirements,category,Incomplete +CWE-1384,Improper Handling of Physical or Environmental Conditions,weakness,Incomplete +CWE-1385,Missing Origin Validation in WebSockets,weakness,Incomplete +CWE-1386,Insecure Operation on Windows Junction / Mount Point,weakness,Incomplete +CWE-1387,Weaknesses in the 2022 CWE Top 25 Most Dangerous Software Weaknesses,view,Draft +CWE-1388,Physical Access Issues and Concerns,category,Draft +CWE-1389,Incorrect Parsing of Numbers with Different Radices,weakness,Incomplete +CWE-1390,Weak Authentication,weakness,Incomplete +CWE-1391,Use of Weak Credentials,weakness,Incomplete +CWE-1392,Use of Default Credentials,weakness,Incomplete +CWE-1393,Use of Default Password,weakness,Incomplete +CWE-1394,Use of Default Cryptographic Key,weakness,Incomplete +CWE-1395,Dependency on Vulnerable Third-Party Component,weakness,Incomplete +CWE-1396,Comprehensive Categorization: Access Control,category,Incomplete +CWE-1397,Comprehensive Categorization: Comparison,category,Incomplete +CWE-1398,Comprehensive Categorization: Component Interaction,category,Incomplete +CWE-1399,Comprehensive Categorization: Memory Safety,category,Incomplete +CWE-1400,Comprehensive Categorization for Software Assurance Trends,view,Draft +CWE-1401,Comprehensive Categorization: Concurrency,category,Incomplete +CWE-1402,Comprehensive Categorization: Encryption,category,Incomplete +CWE-1403,Comprehensive Categorization: Exposed Resource,category,Incomplete +CWE-1404,Comprehensive Categorization: File Handling,category,Incomplete +CWE-1405,Comprehensive Categorization: Improper Check or Handling of Exceptional Conditions,category,Incomplete +CWE-1406,Comprehensive Categorization: Improper Input Validation,category,Incomplete +CWE-1407,Comprehensive Categorization: Improper Neutralization,category,Incomplete +CWE-1408,Comprehensive Categorization: Incorrect Calculation,category,Incomplete +CWE-1409,Comprehensive Categorization: Injection,category,Incomplete +CWE-1410,Comprehensive Categorization: Insufficient Control Flow Management,category,Incomplete +CWE-1411,Comprehensive Categorization: Insufficient Verification of Data Authenticity,category,Incomplete +CWE-1412,Comprehensive Categorization: Poor Coding Practices,category,Incomplete +CWE-1413,Comprehensive Categorization: Protection Mechanism Failure,category,Incomplete +CWE-1414,Comprehensive Categorization: Randomness,category,Incomplete +CWE-1415,Comprehensive Categorization: Resource Control,category,Incomplete +CWE-1416,Comprehensive Categorization: Resource Lifecycle Management,category,Incomplete +CWE-1417,Comprehensive Categorization: Sensitive Information Exposure,category,Incomplete +CWE-1418,Comprehensive Categorization: Violation of Secure Design Principles,category,Incomplete +CWE-1419,Incorrect Initialization of Resource,weakness,Incomplete +CWE-1420,Exposure of Sensitive Information during Transient Execution,weakness,Incomplete +CWE-1421,Exposure of Sensitive Information in Shared Microarchitectural Structures during Transient Execution,weakness,Incomplete +CWE-1422,Exposure of Sensitive Information caused by Incorrect Data Forwarding during Transient Execution,weakness,Incomplete +CWE-1423,Exposure of Sensitive Information caused by Shared Microarchitectural Predictor State that Influences Transient Execution,weakness,Incomplete +CWE-1424,Weaknesses Addressed by ISA/IEC 62443 Requirements,view,Draft +CWE-1425,Weaknesses in the 2023 CWE Top 25 Most Dangerous Software Weaknesses,view,Draft +CWE-1426,Improper Validation of Generative AI Output,weakness,Incomplete +CWE-1427,Improper Neutralization of Input Used for LLM Prompting,weakness,Incomplete +CWE-1428,Reliance on HTTP instead of HTTPS,weakness,Incomplete +CWE-1429,Missing Security-Relevant Feedback for Unexecuted Operations in Hardware Interface,weakness,Incomplete +CWE-1430,Weaknesses in the 2024 CWE Top 25 Most Dangerous Software Weaknesses,view,Draft +CWE-1431,Driving Intermediate Cryptographic State/Results to Hardware Module Outputs,weakness,Incomplete +CWE-1432,Weaknesses in the 2025 CWE Most Important Hardware Weaknesses List,view,Stable +CWE-1433,2025 MIHW Supplement: Expert Insights,category,Obsolete +CWE-1434,Insecure Setting of Generative AI/ML Model Inference Parameters,weakness,Draft +CWE-1435,Weaknesses in the 2025 CWE Top 25 Most Dangerous Software Weaknesses,view,Draft +CWE-1436,OWASP Top Ten 2025 Category A01:2025 - Broken Access Control,category,Incomplete +CWE-1437,OWASP Top Ten 2025 Category A02:2025 - Security Misconfiguration,category,Incomplete +CWE-1438,OWASP Top Ten 2025 Category A03:2025 - Software Supply Chain Failures,category,Incomplete +CWE-1439,OWASP Top Ten 2025 Category A04:2025 - Cryptographic Failures,category,Incomplete +CWE-1440,OWASP Top Ten 2025 Category A05:2025 - Injection,category,Incomplete +CWE-1441,OWASP Top Ten 2025 Category A06:2025 - Insecure Design,category,Incomplete +CWE-1442,OWASP Top Ten 2025 Category A07:2025 - Authentication Failures,category,Incomplete +CWE-1443,OWASP Top Ten 2025 Category A08:2025 - Software or Data Integrity Failures,category,Incomplete +CWE-1444,OWASP Top Ten 2025 Category A09:2025 - Logging & Alerting Failures,category,Incomplete +CWE-1445,OWASP Top Ten 2025 Category A10:2025 - Mishandling of Exceptional Conditions,category,Incomplete +CWE-1446,Weaknesses That are Specific to AI/ML Technology,category,Incomplete +CWE-1447,General Software Weaknesses that Appear in Products that Use or Support AI/ML Technology,category,Incomplete +CWE-1448,Weaknesses Related to AI/ML Products,view,Incomplete +CWE-1450,Weaknesses in OWASP Top Ten RC1 (2025),view,Incomplete +CWE-2000,Comprehensive CWE Dictionary,view,Draft diff --git a/data_validation/reference_data/framework_control_ids/cis_controls_v8.csv b/data_validation/reference_data/framework_control_ids/cis_controls_v8.csv deleted file mode 100644 index 782187f..0000000 --- a/data_validation/reference_data/framework_control_ids/cis_controls_v8.csv +++ /dev/null @@ -1,2 +0,0 @@ -control_id,name,description -# TODO: Populate with CIS Controls v8 safeguards diff --git a/data_validation/reference_data/framework_control_ids/iso27001_annexa.csv b/data_validation/reference_data/framework_control_ids/iso27001_annexa.csv deleted file mode 100644 index 89261dc..0000000 --- a/data_validation/reference_data/framework_control_ids/iso27001_annexa.csv +++ /dev/null @@ -1,2 +0,0 @@ -control_id,name,description -# TODO: Populate with ISO 27001 Annex A controls diff --git a/data_validation/reference_data/framework_control_ids/iso42001.csv b/data_validation/reference_data/framework_control_ids/iso42001.csv deleted file mode 100644 index 87b0e01..0000000 --- a/data_validation/reference_data/framework_control_ids/iso42001.csv +++ /dev/null @@ -1,2 +0,0 @@ -control_id,name,description -# TODO: Populate with ISO/IEC 42001 controls diff --git a/data_validation/reference_data/framework_control_ids/mitre_attack.csv b/data_validation/reference_data/framework_control_ids/mitre_attack.csv deleted file mode 100644 index e8713d5..0000000 --- a/data_validation/reference_data/framework_control_ids/mitre_attack.csv +++ /dev/null @@ -1,2 +0,0 @@ -control_id,name,description -# TODO: Populate with relevant MITRE ATT&CK techniques diff --git a/data_validation/reference_data/framework_control_ids/nist_ai_rmf.csv b/data_validation/reference_data/framework_control_ids/nist_ai_rmf.csv deleted file mode 100644 index d918409..0000000 --- a/data_validation/reference_data/framework_control_ids/nist_ai_rmf.csv +++ /dev/null @@ -1,2 +0,0 @@ -control_id,name,description -# TODO: Populate with NIST AI RMF controls diff --git a/data_validation/reference_data/framework_control_ids/nist_csf2.csv b/data_validation/reference_data/framework_control_ids/nist_csf2.csv deleted file mode 100644 index 0b92249..0000000 --- a/data_validation/reference_data/framework_control_ids/nist_csf2.csv +++ /dev/null @@ -1,12 +0,0 @@ -control_id,function,category,description -GV.OC-01,Govern,Organizational Context,Organizational mission is understood and informs cybersecurity risk management -GV.RM-01,Govern,Risk Management Strategy,Risk management objectives are established and agreed to -ID.AM-01,Identify,Asset Management,Inventories of hardware managed by the organization are maintained -ID.AM-02,Identify,Asset Management,Inventories of software platforms and applications are maintained -PR.AA-01,Protect,Identity Management and Access Control,Identities and credentials for authorized users are managed -PR.DS-01,Protect,Data Security,Data-at-rest is protected -PR.DS-02,Protect,Data Security,Data-in-transit is protected -DE.CM-01,Detect,Continuous Monitoring,Networks and network services are monitored for anomalous events -DE.AE-02,Detect,Adverse Event Analysis,Potentially adverse events are analyzed -RS.MA-01,Respond,Incident Management,Incident response plan is executed -RC.RP-01,Recover,Incident Recovery Plan Execution,Recovery plan is executed during or after a cybersecurity incident diff --git a/data_validation/reference_data/mitre_atlas_techniques.csv b/data_validation/reference_data/mitre_atlas_techniques.csv index 616dda0..9b4aca2 100644 --- a/data_validation/reference_data/mitre_atlas_techniques.csv +++ b/data_validation/reference_data/mitre_atlas_techniques.csv @@ -1,29 +1,212 @@ -technique_id,name,tactic -AML.T0002,Data Poisoning,Initial Access -AML.T0003,Publish Poisoned Datasets,Resource Development -AML.T0004,ML Supply Chain Compromise,Initial Access -AML.T0010,ML Model Inference API Access,Initial Access -AML.T0013,Discover ML Artifacts,Collection -AML.T0014,Discover ML Model Ontology,Discovery -AML.T0015,Evade ML Model,Defense Evasion -AML.T0018,Backdoor ML Model,Persistence -AML.T0019,Publish Poisoned Models,Resource Development -AML.T0020,Poison Training Data,Impact -AML.T0024,Exfiltration via ML Inference API,Exfiltration -AML.T0025,Exfiltration via Cyber Means,Exfiltration -AML.T0029,Denial of ML Service,Impact -AML.T0031,Erode ML Model Integrity,Impact -AML.T0034,Cost Harvesting,Impact -AML.T0035,ML Intellectual Property Theft,Impact -AML.T0036,Data Reconstruction,Exfiltration -AML.T0040,ML Model Inversion,Exfiltration -AML.T0043,Craft Adversarial Data,Execution -AML.T0044,Full ML Model Access,Collection -AML.T0045,Extract ML Model,Exfiltration -AML.T0046,Prompt Injection - Direct,Initial Access -AML.T0048,Prompt Injection - Indirect,Initial Access -AML.T0050,Command Injection via Prompt,Execution -AML.T0051,LLM Prompt Injection,Initial Access -AML.T0054,LLM Jailbreak,Defense Evasion -AML.T0056,LLM Meta Prompt Extraction,Collection -AML.T0057,LLM Data Leakage,Exfiltration +technique_id,name,parent_id,tactics,status +AML.T0000,Search Open Technical Databases,,Reconnaissance,current +AML.T0000.000,Journals and Conference Proceedings,AML.T0000,Reconnaissance,current +AML.T0000.001,Pre-Print Repositories,AML.T0000,Reconnaissance,current +AML.T0000.002,Technical Blogs,AML.T0000,Reconnaissance,current +AML.T0000.003,Scan Databases,AML.T0000,Reconnaissance,current +AML.T0001,Search Open AI Vulnerability Analysis,,Reconnaissance,current +AML.T0002,Acquire Public AI Artifacts,,Resource Development,current +AML.T0002.000,Datasets,AML.T0002,Resource Development,current +AML.T0002.001,Models,AML.T0002,Resource Development,current +AML.T0002.002,AI Agent Configuration,AML.T0002,Resource Development,current +AML.T0003,Search Victim-Owned Websites,,Reconnaissance,current +AML.T0004,Search Application Repositories,,Reconnaissance,current +AML.T0005,Create Proxy AI Model,,AI Attack Adaptation,current +AML.T0005.000,Train Proxy via Gathered AI Artifacts,AML.T0005,AI Attack Adaptation,current +AML.T0005.001,Train Proxy via Replication,AML.T0005,AI Attack Adaptation,current +AML.T0005.002,Use Pre-Trained Model,AML.T0005,AI Attack Adaptation,current +AML.T0006,Active Scanning,,Reconnaissance,current +AML.T0006.000,Enumerate Hosted AI Resources,AML.T0006,Reconnaissance,current +AML.T0006.001,Query Platform Metadata APIs,AML.T0006,Reconnaissance,current +AML.T0006.002,Scan for Exposed AI Infrastructure,AML.T0006,Reconnaissance,current +AML.T0006.003,Probe AI Agent Trigger Channels,AML.T0006,Reconnaissance,current +AML.T0007,Discover AI Artifacts,,Discovery,current +AML.T0008,Acquire Infrastructure,,Resource Development,current +AML.T0008.000,AI Development Workspaces,AML.T0008,Resource Development,current +AML.T0008.001,Consumer Hardware,AML.T0008,Resource Development,current +AML.T0008.002,Domains,AML.T0008,Resource Development,current +AML.T0008.003,Physical Countermeasures,AML.T0008,Resource Development,current +AML.T0008.004,Serverless,AML.T0008,Resource Development,current +AML.T0008.005,AI Service Proxies,AML.T0008,Resource Development,current +AML.T0010,AI Supply Chain Compromise,,Initial Access,current +AML.T0010.000,Hardware,AML.T0010,Initial Access,current +AML.T0010.001,AI Software,AML.T0010,Initial Access,current +AML.T0010.002,Data,AML.T0010,Initial Access,current +AML.T0010.003,Model,AML.T0010,Initial Access,current +AML.T0010.004,Container Registry,AML.T0010,Initial Access,current +AML.T0010.005,AI Agent Tool,AML.T0010,Initial Access,current +AML.T0011,User Execution,,Execution,current +AML.T0011.000,Unsafe AI Artifacts,AML.T0011,Execution,current +AML.T0011.001,Malicious Package,AML.T0011,Execution,current +AML.T0011.002,Poisoned AI Agent Tool,AML.T0011,Execution,current +AML.T0011.003,Malicious Link,AML.T0011,Execution,current +AML.T0012,Valid Accounts,,Initial Access; Privilege Escalation; Lateral Movement,current +AML.T0013,Discover AI Model Ontology,,Discovery,current +AML.T0014,Discover AI Model Family,,Discovery,current +AML.T0015,Evade AI Model,,Initial Access; Defense Evasion; Impact,current +AML.T0016,Obtain Capabilities,,Resource Development,current +AML.T0016.000,Adversarial AI Attack Implementations,AML.T0016,Resource Development,current +AML.T0016.001,Software Tools,AML.T0016,Resource Development,current +AML.T0016.002,Generative AI,AML.T0016,Resource Development,current +AML.T0016.003,Exploits,AML.T0016,Resource Development,current +AML.T0016.004,AI Agent Tools,AML.T0016,Resource Development,current +AML.T0017,Develop Capabilities,,Resource Development,current +AML.T0017.000,Adversarial AI Attacks,AML.T0017,Resource Development,current +AML.T0017.001,Autonomous Exploit Development,AML.T0017,Resource Development,current +AML.T0017.002,AI Agent Tools,AML.T0017,Resource Development,current +AML.T0018,Manipulate AI Model,,AI Attack Adaptation; Persistence,current +AML.T0018.000,Poison AI Model,AML.T0018,AI Attack Adaptation; Persistence,current +AML.T0018.001,Modify AI Model Architecture,AML.T0018,AI Attack Adaptation; Persistence,current +AML.T0018.002,Embed Malware,AML.T0018,AI Attack Adaptation; Persistence,current +AML.T0018.003,Modify Prompt Construction Logic,AML.T0018,AI Attack Adaptation; Persistence,current +AML.T0019,Publish Poisoned Datasets,,,retired +AML.T0020,Training Data Poisoning,,Persistence,current +AML.T0021,Establish Accounts,,Resource Development,current +AML.T0024,Exfiltration via AI Inference API,,Exfiltration,current +AML.T0024.000,Infer Training Data Membership,AML.T0024,Exfiltration,current +AML.T0024.001,Invert AI Model,AML.T0024,Exfiltration,current +AML.T0024.002,Extract AI Model,AML.T0024,Exfiltration,current +AML.T0025,Exfiltration via Cyber Means,,Exfiltration,current +AML.T0029,Denial of AI Service,,Impact,current +AML.T0031,Erode AI Model Integrity,,Impact,current +AML.T0034,Cost Harvesting,,Impact,current +AML.T0034.000,Excessive Queries,AML.T0034,Impact,current +AML.T0034.001,Resource-Intensive Queries,AML.T0034,Impact,current +AML.T0034.002,Agentic Resource Consumption,AML.T0034,Impact,current +AML.T0035,AI Artifact Collection,,Collection,current +AML.T0036,Data from Information Repositories,,Collection,current +AML.T0037,Data from Local System,,Collection,current +AML.T0040,AI Model Inference API Access,,AI Model Access,current +AML.T0041,Physical Environment Access,,AI Model Access,current +AML.T0042,Verify Attack,,AI Attack Adaptation,current +AML.T0043,Craft Adversarial Data,,AI Attack Adaptation,current +AML.T0043.000,White-Box Optimization,AML.T0043,AI Attack Adaptation,current +AML.T0043.001,Black-Box Optimization,AML.T0043,AI Attack Adaptation,current +AML.T0043.002,Black-Box Transfer,AML.T0043,AI Attack Adaptation,current +AML.T0043.003,Manual Modification,AML.T0043,AI Attack Adaptation,current +AML.T0043.004,Insert Backdoor Trigger,AML.T0043,AI Attack Adaptation,current +AML.T0044,Full AI Model Access,,AI Model Access,current +AML.T0046,Spamming AI System with Chaff Data,,Impact,current +AML.T0047,AI-Enabled Product or Service,,AI Model Access,current +AML.T0048,External Harms,,Impact,current +AML.T0048.000,Financial Harm,AML.T0048,Impact,current +AML.T0048.001,Reputational Harm,AML.T0048,Impact,current +AML.T0048.002,Societal Harm,AML.T0048,Impact,current +AML.T0048.003,User Harm,AML.T0048,Impact,current +AML.T0048.004,AI Intellectual Property Theft,AML.T0048,Impact,current +AML.T0049,Exploit Public-Facing Application,,Initial Access,current +AML.T0050,Command and Scripting Interpreter,,Execution,current +AML.T0051,LLM Prompt Injection,,Execution,current +AML.T0051.000,Direct,AML.T0051,Execution,current +AML.T0051.001,Indirect,AML.T0051,Execution,current +AML.T0051.002,Triggered,AML.T0051,Execution,current +AML.T0052,Phishing,,Initial Access; Lateral Movement,current +AML.T0052.000,Spearphishing via Social Engineering LLM,AML.T0052,Initial Access; Lateral Movement,current +AML.T0052.001,Deepfake-Assisted Phishing,AML.T0052,Initial Access; Lateral Movement,current +AML.T0053,AI Agent Tool Invocation,,Execution; Privilege Escalation; Lateral Movement,current +AML.T0054,LLM Jailbreak,,Defense Evasion; Privilege Escalation,current +AML.T0055,Unsecured Credentials,,Credential Access,current +AML.T0056,Extract LLM System Prompt,,Exfiltration,current +AML.T0057,LLM Data Leakage,,Exfiltration,current +AML.T0058,Publish Poisoned Models,,,retired +AML.T0059,Erode Dataset Integrity,,Impact,current +AML.T0060,Publish Hallucinated Entities,,Resource Development,current +AML.T0061,LLM Prompt Self-Replication,,Persistence,current +AML.T0062,Discover LLM Hallucinations,,Discovery,current +AML.T0063,Discover AI Model Outputs,,Discovery,current +AML.T0064,Gather RAG-Indexed Targets,,Reconnaissance,current +AML.T0065,LLM Prompt Crafting,,AI Attack Adaptation,current +AML.T0066,Retrieval Content Crafting,,AI Attack Adaptation,current +AML.T0067,LLM Trusted Output Components Manipulation,,Defense Evasion,current +AML.T0067.000,Citations,AML.T0067,Defense Evasion,current +AML.T0068,LLM Prompt Obfuscation,,Defense Evasion,current +AML.T0069,Discover LLM System Information,,Discovery,current +AML.T0069.000,Special Character Sets,AML.T0069,Discovery,current +AML.T0069.001,System Instruction Keywords,AML.T0069,Discovery,current +AML.T0069.002,System Prompt,AML.T0069,Discovery,current +AML.T0070,RAG Poisoning,,Persistence,current +AML.T0071,False RAG Entry Injection,,Defense Evasion,current +AML.T0072,Cyber Communication Channel,,Command and Control,current +AML.T0073,Impersonation,,Defense Evasion,current +AML.T0074,Masquerading,,Defense Evasion,current +AML.T0075,Enterprise Resource Discovery,,Discovery,current +AML.T0076,Corrupt AI Model,,Defense Evasion,current +AML.T0077,LLM Response Rendering,,Exfiltration,current +AML.T0078,Drive-by Compromise,,Initial Access,current +AML.T0079,Stage Capabilities,,Resource Development,current +AML.T0080,AI Agent Context Poisoning,,Persistence,current +AML.T0080.000,Memory,AML.T0080,Persistence,current +AML.T0080.001,Thread,AML.T0080,Persistence,current +AML.T0081,Modify AI Agent Configuration,,Persistence; Defense Evasion,current +AML.T0082,RAG Credential Harvesting,,Credential Access,current +AML.T0083,Credentials from AI Agent Configuration,,Credential Access,current +AML.T0084,Discover AI Agent Configuration,,Discovery,current +AML.T0084.000,Embedded Knowledge,AML.T0084,Discovery,current +AML.T0084.001,Tool Definitions,AML.T0084,Discovery,current +AML.T0084.002,Activation Triggers,AML.T0084,Discovery,current +AML.T0084.003,Call Chains,AML.T0084,Discovery,current +AML.T0085,Data from AI Services,,Collection,current +AML.T0085.000,RAG Databases,AML.T0085,Collection,current +AML.T0085.001,AI Agent Tools,AML.T0085,Collection,current +AML.T0086,Exfiltration via AI Agent Tool Invocation,,Exfiltration,current +AML.T0087,Gather Victim Identity Information,,Reconnaissance,current +AML.T0088,Generate Deepfakes,,AI Attack Adaptation,current +AML.T0089,Enterprise Environment Discovery,,Discovery,current +AML.T0090,OS Credential Dumping,,Credential Access,current +AML.T0091,Use Alternate Authentication Material,,Lateral Movement,current +AML.T0091.000,Application Access Token,AML.T0091,Lateral Movement,current +AML.T0091.001,Web Session Cookie,AML.T0091,Lateral Movement,current +AML.T0092,Manipulate User LLM Chat History,,Defense Evasion,current +AML.T0093,Prompt Infiltration via Public-Facing Application,,Initial Access; Persistence,current +AML.T0094,Delay Execution of LLM Instructions,,Defense Evasion,current +AML.T0095,Search Open Websites/Domains,,Reconnaissance,current +AML.T0095.000,Code Repositories,AML.T0095,Reconnaissance,current +AML.T0096,AI Service API,,Command and Control,current +AML.T0097,Virtualization/Sandbox Evasion,,Defense Evasion,current +AML.T0098,AI Agent Tool Credential Harvesting,,Credential Access,current +AML.T0099,AI Agent Tool Data Poisoning,,Persistence,current +AML.T0100,AI Agent Clickbait,,Execution,current +AML.T0101,Data Destruction via AI Agent Tool Invocation,,Impact,current +AML.T0102,Generate Malicious Commands,,AI Attack Adaptation,current +AML.T0103,Deploy AI Agent,,Execution,current +AML.T0104,Publish Poisoned AI Agent Tool,,,retired +AML.T0105,Escape to Host,,Privilege Escalation,current +AML.T0106,Exploitation for Credential Access,,Credential Access,current +AML.T0107,Exploitation for Defense Evasion,,Defense Evasion,current +AML.T0108,AI Agent,,Command and Control,current +AML.T0109,AI Supply Chain Rug Pull,,Defense Evasion,current +AML.T0110,AI Agent Tool Poisoning,,Persistence,current +AML.T0110.000,Definition and Instructions,AML.T0110,Persistence,current +AML.T0110.001,Implementation,AML.T0110,Persistence,current +AML.T0110.002,Runtime Response,AML.T0110,Persistence,current +AML.T0111,AI Supply Chain Reputation Inflation,,Defense Evasion,current +AML.T0112,Machine Compromise,,Impact,current +AML.T0112.000,Local AI Agent,AML.T0112,Impact,current +AML.T0112.001,AI Artifacts,AML.T0112,Impact,current +AML.T0113,Steal Web Session Cookie,,Credential Access,current +AML.T0114,AI Service Web Interface,,Command and Control,current +AML.T0115,Publish Poisoned AI Artifacts,,Resource Development,current +AML.T0115.000,Datasets,AML.T0115,Resource Development,current +AML.T0115.001,Models,AML.T0115,Resource Development,current +AML.T0115.002,AI Agent Tools,AML.T0115,Resource Development,current +AML.T0116,Autonomous Reconnaissance,,Reconnaissance,current +AML.T0117,Autonomous Attack-Path Adaptation,,AI Attack Adaptation,current +AML.T0118,Autonomous AI Agent Communication,,AI Attack Adaptation,current +AML.T0118.000,Communication via Shared Artifacts,AML.T0118,AI Attack Adaptation,current +AML.T0118.001,Direct Agent Communication,AML.T0118,AI Attack Adaptation,current +AML.T0119,Exploit Automated Artifact Processing Pipeline,,Initial Access,current +AML.T0120,AI Artifact Repository,,Command and Control,current +AML.T0121,AI Agent Environment Reconstruction,,Persistence,current +AML.T0122,Exploitation of Remote Services,,Lateral Movement,current +AML.T0123,Obfuscated Files or Information,,Defense Evasion,current +AML.T0124,Autonomous Attack Orchestration,,AI Attack Adaptation,current +AML.T0125,Create Account,,Persistence,current +AML.T0126,Automated Collection,,Collection,current +AML.T0127,Data Staged,,Collection,current +AML.T0128,Compromise Infrastructure,,Resource Development,current +AML.T0129,Triggers in Multimodal Inputs,,Defense Evasion,current +AML.T0130,AI Agent Response Biasing,,Impact,current +AML.T0131,Crafted AI Assistant Links,,Initial Access,current +AML.T0132,Misconfigured or Publicly Exposed AI Services,,Initial Access,current +AML.T0133,Discover AI Agent Runtime Capabilities,,Discovery,current +AML.T0134,AI Targeted Cloaking,,Defense Evasion,current diff --git a/data_validation/reference_data/update_reference_data.py b/data_validation/reference_data/update_reference_data.py new file mode 100644 index 0000000..2c58e60 --- /dev/null +++ b/data_validation/reference_data/update_reference_data.py @@ -0,0 +1,123 @@ +""" +Regenerates the MITRE ATLAS and CWE lookup tables from MITRE's published data. + + python update_reference_data.py # both + python update_reference_data.py --atlas # mitre_atlas_techniques.csv only + python update_reference_data.py --cwe # cwe_ids.csv only + +Run it when ATLAS or CWE publish a new release, and commit the regenerated +CSVs together with the version line this script updates in SOURCES.md. +Requires pyyaml for the ATLAS data. Needs network access; nothing else in +data_validation does. +""" +from __future__ import annotations + +import argparse +import csv +import io +import re +import sys +import urllib.request +# stdlib ElementTree does not fetch external entities; the input is MITRE's +# own release file over HTTPS, read only by maintainers running this script. +import xml.etree.ElementTree as ET # noqa: S405 +import zipfile +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ATLAS_DIST = "https://raw.githubusercontent.com/mitre-atlas/atlas-data/main/dist/" +# manifest.yaml lists every release, newest first; dist/ATLAS.yaml is the +# deprecated pre-2026 format and is no longer updated. +ATLAS_MANIFEST = ATLAS_DIST + "manifest.yaml" +# Last release in the old format. IDs in it that the current release no longer +# has are kept as "retired" rows, so data citing them gets a warning that says +# so instead of an "unknown ID" error. +ATLAS_LEGACY = ATLAS_DIST + "ATLAS.yaml" +CWE_URL = "https://cwe.mitre.org/data/xml/cwec_latest.xml.zip" +SOURCES = HERE / "SOURCES.md" + + +def fetch(url: str) -> bytes: + with urllib.request.urlopen(url, timeout=60) as resp: # noqa: S310 - fixed https URLs + return resp.read() + + +def set_version(label: str, version: str) -> None: + text = SOURCES.read_text(encoding="utf-8") + text = re.sub(rf"(\| {re.escape(label)} \|[^|]*\| )[^|]*(\|)", rf"\g<1>{version} \2", text) + SOURCES.write_text(text, encoding="utf-8") + + +def update_atlas() -> None: + import yaml + latest = yaml.safe_load(fetch(ATLAS_MANIFEST))[0] + path = next(v["path"] for v in latest["versions"] if v["format-version"].startswith("6.")) + data = yaml.safe_load(fetch(ATLAS_DIST + path)) + tactics = {tid: t["name"] for tid, t in data["tactics"].items()} + rels = data["relationships"] + rows = [] + for tid in sorted(data["techniques"]): + links = rels.get(tid, {}) + rows.append({ + "technique_id": tid, + "name": data["techniques"][tid]["name"], + "parent_id": "; ".join(r["target"] for r in links.get("specializes", [])), + "tactics": "; ".join(tactics.get(r["target"], r["target"]) for r in links.get("achieves", [])), + "status": "current", + }) + legacy = yaml.safe_load(fetch(ATLAS_LEGACY)) + for matrix in legacy["matrices"]: + for t in matrix["techniques"]: + if t["id"] not in data["techniques"]: + rows.append({"technique_id": t["id"], "name": t["name"], "parent_id": t.get("specializes", ""), + "tactics": "", "status": "retired"}) + rows.sort(key=lambda r: r["technique_id"]) + with open(HERE / "mitre_atlas_techniques.csv", "w", newline="", encoding="utf-8") as fh: + w = csv.DictWriter(fh, fieldnames=["technique_id", "name", "parent_id", "tactics", "status"], lineterminator="\n") + w.writeheader() + w.writerows(rows) + version = data["collection"]["version"] + set_version("MITRE ATLAS", str(version)) + retired = sum(r["status"] == "retired" for r in rows) + print(f"mitre_atlas_techniques.csv: {len(rows) - retired} techniques, {retired} retired (ATLAS {version})") + + +def update_cwe() -> None: + with zipfile.ZipFile(io.BytesIO(fetch(CWE_URL))) as zf: + root = ET.fromstring(zf.read(zf.namelist()[0])) + ns = {"c": root.tag.split("}")[0].strip("{")} + rows = [] + for kind, xpath in (("weakness", "c:Weaknesses/c:Weakness"), + ("category", "c:Categories/c:Category"), + ("view", "c:Views/c:View")): + for el in root.findall(xpath, ns): + rows.append({ + "cwe_id": f"CWE-{el.get('ID')}", + "name": el.get("Name"), + "type": kind, + "status": el.get("Status"), + }) + rows.sort(key=lambda r: int(r["cwe_id"].split("-")[1])) + with open(HERE / "cwe_ids.csv", "w", newline="", encoding="utf-8") as fh: + w = csv.DictWriter(fh, fieldnames=["cwe_id", "name", "type", "status"], lineterminator="\n") + w.writeheader() + w.writerows(rows) + set_version("CWE", root.get("Version")) + print(f"cwe_ids.csv: {len(rows)} entries (CWE {root.get('Version')})") + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description="Regenerate ATLAS and CWE reference tables") + parser.add_argument("--atlas", action="store_true") + parser.add_argument("--cwe", action="store_true") + args = parser.parse_args(argv) + both = not (args.atlas or args.cwe) + if both or args.atlas: + update_atlas() + if both or args.cwe: + update_cwe() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/data_validation/requirements.txt b/data_validation/requirements.txt index d4ac8e7..ba66d86 100644 --- a/data_validation/requirements.txt +++ b/data_validation/requirements.txt @@ -1,5 +1,5 @@ -jsonschema>=4.17.0 -pandas>=2.0.0 +# Validators, QC tools and their tests +jsonschema>=4.18.0 pytest>=7.0.0 +# Only for reference_data/update_reference_data.py (refreshing the ATLAS table) pyyaml>=6.0 -regex>=2023.0 diff --git a/data_validation/run_all_checks.py b/data_validation/run_all_checks.py index 2e54acc..e764e63 100644 --- a/data_validation/run_all_checks.py +++ b/data_validation/run_all_checks.py @@ -1,17 +1,27 @@ """ -Run every dataset's own validator and report one result per dataset. - -Each dataset that ships a validate.py (schema.json + entries/) is checked by -running that script in its own directory. Datasets without a validate.py are -listed as "no validator" so the gap is visible instead of silently passing. +Run every dataset's own validator, then the shared checks, and report one +result per dataset. + +1. Dataset validators. Each validate.py inside a dataset (at its root, or in + a sub-collection such as one contributor's folder of cases) is run in its + own directory. These are authoritative for their data. Datasets without + one are listed as "no validator" so the gap is visible instead of + silently passing. +2. Shared checks (data_validation/validators/) on every data file: + schema (for files no dataset validator covers), DSGAI IDs, CVE/CWE/ATLAS + IDs, anonymization patterns and duplicates. Errors fail the run; + heuristic warnings (anonymization, near-duplicates, retired IDs) are + counted, and printed with --verbose. Usage: python run_all_checks.py # every dataset under ../datasets python run_all_checks.py --dataset ../datasets/exploit_dataset - python run_all_checks.py --verbose # print each validator's output + python run_all_checks.py --verbose # print each validator's output and every warning + python run_all_checks.py --strict # warnings fail the run too -Exit code: 0 when every validator that ran passed, 1 when any failed, -2 when the arguments name no dataset that can be checked. +Exit code: 0 when everything that ran passed, 1 when any validator failed +or any shared check reported an error, 2 when the arguments name nothing +that can be checked. """ from __future__ import annotations @@ -20,7 +30,17 @@ import sys from pathlib import Path +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from validators import ( # noqa: E402 + anonymization_scanner, crossref_validator, dedup_checker, dsgai_mapping_check, schema_validator, +) +from validators._common import ( # noqa: E402 + ERROR, SKIP_DIRS, Finding, ensure_utf8_stdout, iter_data_files, load_json, load_taxonomy_ids, +) + DATASETS_ROOT = Path(__file__).resolve().parent.parent / "datasets" +DEFAULT_TIMEOUT = 300 def find_datasets(root: Path) -> list[Path]: @@ -31,18 +51,32 @@ def find_datasets(root: Path) -> list[Path]: ) -def run_validator(dataset: Path) -> tuple[str, str]: +def find_validators(dataset: Path) -> list[Path]: + """Every validate.py in the dataset, outside test and hidden folders.""" + return sorted( + p for p in dataset.rglob("validate.py") + if not any(part in SKIP_DIRS or part.startswith(".") for part in p.relative_to(dataset).parts[:-1]) + ) + + +def run_validator(dataset: Path, timeout: float = DEFAULT_TIMEOUT) -> tuple[str, str]: """Run dataset/validate.py. Returns (status, combined output).""" script = dataset / "validate.py" if not script.is_file(): return "NO VALIDATOR", "" - proc = subprocess.run( - [sys.executable, script.name], - cwd=dataset, - capture_output=True, - text=True, - check=False, - ) + try: + proc = subprocess.run( + [sys.executable, script.name], + cwd=dataset, + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=timeout, + check=False, + ) + except subprocess.TimeoutExpired: + return "ERROR", f"timed out after {timeout:g}s" output = (proc.stdout + proc.stderr).strip() if proc.returncode == 0: return "PASS", output @@ -51,8 +85,30 @@ def run_validator(dataset: Path) -> tuple[str, str]: return "FAIL", output +def shared_checks(dataset: Path, datasets_root: Path) -> tuple[int, list[Finding]]: + """(files checked, findings) for the shared checks on one dataset.""" + files = iter_data_files(dataset) + covered_dirs = [v.parent for v in find_validators(dataset)] + taxonomy = load_taxonomy_ids(datasets_root) + findings: list[Finding] = [] + for path in files: + data, err = load_json(path, "load") + if err: + findings.append(err) + continue + if not any(path.is_relative_to(d) for d in covered_dirs): + findings += schema_validator.validate_file(path, dataset_root=dataset) + findings += dsgai_mapping_check.check_data(path, data, taxonomy) + findings += crossref_validator.check_data(path, data) + findings += anonymization_scanner.check_data(path, data) + records, load_errors = dedup_checker.load_records(files) + findings += load_errors + dedup_checker.check_records(records) + return len(files), findings + + def main(argv: list[str] | None = None) -> int: - parser = argparse.ArgumentParser(description="Run all dataset validators") + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Run all dataset validators and shared checks") parser.add_argument( "--dataset", help="Path to one dataset directory (default: every dataset)", @@ -62,18 +118,25 @@ def main(argv: list[str] | None = None) -> int: help="Check every dataset (the default; kept for the documented usage)", ) parser.add_argument("--verbose", "-v", action="store_true", - help="Print each validator's output") + help="Print each validator's output and every warning") + parser.add_argument("--strict", action="store_true", + help="Treat shared-check warnings as failures") + parser.add_argument("--no-shared", action="store_true", + help="Run only the dataset validators") + parser.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT, + help=f"Seconds each dataset validator may run (default {DEFAULT_TIMEOUT})") args = parser.parse_args(argv) + datasets_root = DATASETS_ROOT.resolve() if args.dataset: target = Path(args.dataset).resolve() if not target.is_dir(): print(f"Directory not found: {args.dataset}") return 2 # The datasets root itself means "all of them"; anything else is one dataset. - datasets = find_datasets(target) if target == DATASETS_ROOT.resolve() else [target] + datasets = find_datasets(target) if target == datasets_root else [target] else: - datasets = find_datasets(DATASETS_ROOT) + datasets = find_datasets(datasets_root) if not datasets: print("No dataset directories found") @@ -81,20 +144,48 @@ def main(argv: list[str] | None = None) -> int: results = [] for dataset in datasets: - status, output = run_validator(dataset) - results.append(status) - print(f"{status:<13} {dataset.name}") - if output and (args.verbose or status in ("FAIL", "ERROR")): - for line in output.splitlines(): - print(f" {line}") + validators = find_validators(dataset) + if not validators: + results.append("NO VALIDATOR") + print(f"{'NO VALIDATOR':<13} {dataset.name}") + continue + for script in validators: + label = dataset.name + if script.parent != dataset: + label += "/" + script.parent.relative_to(dataset).as_posix() + status, output = run_validator(script.parent, args.timeout) + results.append(status) + print(f"{status:<13} {label}") + if output and (args.verbose or status in ("FAIL", "ERROR")): + for line in output.splitlines(): + print(f" {line}") + + checked_files = errors = warnings = 0 + if not args.no_shared: + print("\nShared checks (schema, DSGAI IDs, CVE/CWE/ATLAS IDs, anonymization, duplicates):") + for dataset in datasets: + n, findings = shared_checks(dataset, datasets_root) + if not n: + continue + e = sum(f.level == ERROR for f in findings) + w = len(findings) - e + checked_files, errors, warnings = checked_files + n, errors + e, warnings + w + status = "FAIL" if e or (args.strict and w) else "WARN" if w else "PASS" + print(f"{status:<13} {dataset.name} ({n} file(s), {e} error(s), {w} warning(s))") + for f in findings: + if f.level == ERROR or args.verbose or args.strict: + print(f" {f.format(datasets_root.parent)}") ran = [s for s in results if s != "NO VALIDATOR"] failed = [s for s in ran if s != "PASS"] print(f"\n{len(ran)} validator(s) ran, {len(failed)} failed, " f"{len(results) - len(ran)} dataset(s) have no validator yet") - if not ran: + if not args.no_shared: + print(f"Shared checks: {checked_files} file(s), {errors} error(s), {warnings} warning(s)" + + ("" if args.verbose or not warnings else " (--verbose lists the warnings)")) + if not ran and not checked_files: return 2 - return 1 if failed else 0 + return 1 if failed or errors or (args.strict and warnings) else 0 if __name__ == "__main__": diff --git a/data_validation/tests/fixtures/duplicate_pair.json b/data_validation/tests/fixtures/duplicate_pair.json index d106bea..fc8d0d2 100644 --- a/data_validation/tests/fixtures/duplicate_pair.json +++ b/data_validation/tests/fixtures/duplicate_pair.json @@ -1,17 +1,20 @@ { - "_comment": "These two entries describe the same incident and should trigger dedup detection", - "entry_a": { - "incident_id": "INC-0010", - "date": "2025-05-01", - "summary": "Multi-tenant vector store returned documents from another tenant due to missing tenant ID filtering at query time", - "dsgai_mapping": ["DSGAI11", "DSGAI13"], - "attack_vector": "Cross-tenant retrieval via unscoped vector queries" - }, - "entry_b": { - "incident_id": "INC-0011", - "date": "2025-05-01", - "summary": "Vector database serving multiple customers leaked documents across tenant boundaries because tenant scoping was not enforced during retrieval", - "dsgai_mapping": ["DSGAI11", "DSGAI13"], - "attack_vector": "Unscoped vector store queries crossing tenant boundaries" - } + "_comment": "Entries a and b describe the same incident with light edits, so dedup_checker flags them. Entry c is a paraphrase of a: the check is lexical, so it is NOT expected to catch it.", + "entries": [ + { + "incident_id": "INC-0010", + "summary": "Multi-tenant vector store returned documents from another tenant due to missing tenant ID filtering at query time", + "attack_vector": "Cross-tenant retrieval via unscoped vector queries" + }, + { + "incident_id": "INC-0011", + "summary": "A multi-tenant vector store returned documents from another tenant due to missing tenant ID filtering at query time", + "attack_vector": "Cross-tenant retrieval via unscoped vector store queries" + }, + { + "incident_id": "INC-0012", + "summary": "Vector database serving multiple customers leaked documents across tenant boundaries because tenant scoping was not enforced during retrieval", + "attack_vector": "Unscoped vector store queries crossing tenant boundaries" + } + ] } diff --git a/data_validation/tests/fixtures/valid_vulnerability.json b/data_validation/tests/fixtures/valid_vulnerability.json index eb5bd90..7af7217 100644 --- a/data_validation/tests/fixtures/valid_vulnerability.json +++ b/data_validation/tests/fixtures/valid_vulnerability.json @@ -1,11 +1,12 @@ { - "vulnerability_id": "VULN-0001", - "description": "RAG pipeline returns verbatim PII from training documents when prompted with specific context-setting instructions followed by enumeration queries.", + "vulnerability_id": "DSGAI-VULN-fixture-rag-pii-enumeration", + "title": "RAG pipeline returns verbatim PII under enumeration prompts", + "description": "A retrieval-augmented assistant returns verbatim personal data from indexed support documents when a user sets up a conversational context and then asks enumeration-style questions. No retrieval-time redaction is applied to the retrieved chunks.", "affected_component": "rag_pipeline", - "dsgai_mapping": ["DSGAI01"], - "llm_top10_mapping": ["LLM06:2025"], - "severity": "High", - "cvss_score": 7.5, - "source": "https://example.com/advisory/2025-001", - "date_reported": "2025-06-15" + "dsgai_mapping": ["DSGAI01", "DSGAI15"], + "owasp_llm_top10_mapping": ["LLM02:2025"], + "severity": {"qualitative": "High", "cvss_v3_score": 7.5}, + "source_urls": ["https://example.com/advisory/2025-001"], + "date_added": "2025-06-15", + "disclosure_status": "research_publication" } diff --git a/data_validation/tests/test_anonymization_scanner.py b/data_validation/tests/test_anonymization_scanner.py index 0080b38..e56a951 100644 --- a/data_validation/tests/test_anonymization_scanner.py +++ b/data_validation/tests/test_anonymization_scanner.py @@ -1,12 +1,65 @@ """Tests for anonymization_scanner.py.""" +from __future__ import annotations + +import sys +from pathlib import Path + import pytest +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators import anonymization_scanner as an # noqa: E402 +from validators._common import WARN # noqa: E402 + +FIXTURES = Path(__file__).resolve().parent / "fixtures" + + +def kinds(value: str) -> set[str]: + return {k for k, _ in an.scan_value(value)} + + def test_clean_entry(): - """An entry with no PII should pass the scan.""" - # TODO: Implement when anonymization_scanner.scan_file() is built - pass + assert an.scan_file(FIXTURES / "valid_incident.json") == [] + def test_pii_detected(): - """An entry with an email address should be flagged.""" - # TODO: Implement when anonymization_scanner.scan_file() is built - pass + findings = an.scan_file(FIXTURES / "incident_with_pii.json") + assert findings and all(f.level == WARN for f in findings) + found = " ".join(f.message for f in findings) + for kind in ("email address", "private ip address", "openai-style key"): + assert kind in found + + +@pytest.mark.parametrize("value,kind", [ + ("contact jane.doe@realcorp.io", "email address"), + ("key AKIAABCDEFGHIJKLMNOP", "aws access key id"), + ("token ghp_abcdefghijklmnopqrstuvwxyz0123", "github token"), + ("-----BEGIN RSA PRIVATE KEY-----", "private key block"), + ("password=hunter2!", "secret-looking assignment"), + ("db at db01.prod.corp", "internal hostname"), + ("see /home/alice/.ssh", "user home path"), + ("callback to http://8.8.8.8/x", "public ip address"), + ("listening on 8.8.4.4:8080", "public ip address"), + ("SSN 123-45-6789", "us social security number"), +]) +def test_patterns(value, kind): + assert kind in kinds(value) + + +@pytest.mark.parametrize("value", [ + "mail admin@example.com or ops@service.test", + "documentation address 192.0.2.10 and 203.0.113.5:443", + "cloud metadata endpoint 169.254.169.254", + "fixed in versions 23.12.4.0 through 24.7.4.1", + "API_KEY=", + "API_KEY=[SYNTHETIC_KEY]", + "", + "password: ...", +]) +def test_safe_values_are_not_flagged(value): + assert kinds(value) == set() + + +def test_long_matches_are_redacted_in_messages(): + [f] = an.check_data(Path("e.json"), {"k": "sk-proj-abcdefghijklmnopqrstuvwx"}) + assert "abcdefghijklmnop" not in f.message diff --git a/data_validation/tests/test_crossref_validator.py b/data_validation/tests/test_crossref_validator.py new file mode 100644 index 0000000..e49e2d3 --- /dev/null +++ b/data_validation/tests/test_crossref_validator.py @@ -0,0 +1,59 @@ +"""Tests for crossref_validator.py.""" +from __future__ import annotations + +import sys +from datetime import date +from pathlib import Path + +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators import crossref_validator as cr # noqa: E402 +from validators._common import ERROR, WARN # noqa: E402 + +P = Path("entry.json") + + +def levels(data) -> list[str]: + return [f.level for f in cr.check_data(P, data)] + + +@pytest.mark.parametrize("value", [ + "CVE-2024-3568", "GHSA-jfh8-c2jp-5v3q", "CWE-79", "CWE-1426", "AML.T0051", "AML.T0051.001", +]) +def test_well_formed_known_ids_pass(value): + assert levels({"id": value}) == [] + + +@pytest.mark.parametrize("value", ["CVE-24-1234", "CVE-2024-12", "GHSA-xxxx-yyyy", "CWE-abc", "AML.T51"]) +def test_malformed_whole_value_is_an_error(value): + assert levels({"id": value}) == [ERROR] + + +def test_future_cve_year(): + assert levels({"cve_id": f"CVE-{date.today().year + 1}-0001"}) == [ERROR] + + +def test_unknown_cwe_and_atlas(): + assert levels({"cwe_ids": ["CWE-999999"]}) == [ERROR] + assert levels({"mitre_atlas_mapping": ["AML.T9999"]}) == [ERROR] + + +def test_deprecated_cwe_and_retired_atlas_warn(): + assert levels({"cwe_ids": ["CWE-1"]}) == [WARN] + assert levels({"mitre_atlas_mapping": ["AML.T0019"]}) == [WARN] + + +def test_ids_in_free_text(): + assert levels({"notes": "Maps to AML.T0051 and CWE-79, see CVE-2024-3568."}) == [] + assert levels({"notes": "Like AML.T9999."}) == [ERROR] + assert levels({"notes": "Placeholder CVE-XXXX-XXXX in a template."}) == [WARN] + assert levels({"url": "https://atlas.mitre.org/techniques/AML.T0051"}) == [] + + +def test_reference_tables_are_complete(): + assert len(cr.atlas_status()) > 150 + assert cr.atlas_status()["AML.T0051"] == "current" + assert len(cr.cwe_status()) > 1000 + assert cr.cwe_status()["CWE-79"] != "Deprecated" diff --git a/data_validation/tests/test_dedup_checker.py b/data_validation/tests/test_dedup_checker.py new file mode 100644 index 0000000..48353b7 --- /dev/null +++ b/data_validation/tests/test_dedup_checker.py @@ -0,0 +1,56 @@ +"""Tests for dedup_checker.py.""" +from __future__ import annotations + +import json +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators import dedup_checker as dd # noqa: E402 +from validators._common import ERROR, WARN # noqa: E402 + +FIXTURES = Path(__file__).resolve().parent / "fixtures" + + +def test_near_duplicate_pair_flagged_paraphrase_not(): + records, errors = dd.load_records([FIXTURES / "duplicate_pair.json"]) + assert errors == [] and len(records) == 3 + findings = dd.check_records(records) + assert [(f.level, f.message.split()[0]) for f in findings] == [(WARN, "INC-0011")] + assert "INC-0010" in findings[0].message + + +def test_duplicate_id_is_an_error(tmp_path): + for name, text in (("a.json", "one thing entirely"), ("b.json", "something else")): + (tmp_path / name).write_text(json.dumps({"exploit_id": "AML.T0051", "description": text})) + findings = dd.check_dataset(tmp_path) + assert [f.level for f in findings] == [ERROR] + + +def test_example_json_is_skipped(tmp_path): + entry = {"vulnerability_id": "CVE-2024-0001", "description": "x" * 50} + (tmp_path / "example.json").write_text(json.dumps(entry)) + (tmp_path / "entries").mkdir() + (tmp_path / "entries" / "CVE-2024-0001.json").write_text(json.dumps(entry)) + assert dd.check_dataset(tmp_path) == [] + + +def test_shared_boilerplate_is_ignored(tmp_path): + licence = "Adapted from a CC BY 4.0 source dataset; see the collection README for attribution." + texts = ["alpha beta gamma delta epsilon", "zeta eta theta iota kappa", + "lambda mu nu xi omicron", "pi rho sigma tau upsilon"] + for i, t in enumerate(texts): + (tmp_path / f"{i}.json").write_text(json.dumps({"id": f"T-{i}", "licence": licence, "prompt": t * 3})) + assert dd.check_dataset(tmp_path) == [] + + +def test_new_file_mode_only_reports_pairs_with_the_new_file(tmp_path): + same = "the same long description of an issue that two old entries share by mistake" + for i in range(2): + (tmp_path / f"old{i}.json").write_text(json.dumps({"id": f"O-{i}", "description": same})) + new = tmp_path / "new.json" + new.write_text(json.dumps({"id": "N-1", "description": "an unrelated and entirely different description here"})) + records, _ = dd.load_records(sorted(tmp_path.glob("*.json"))) + assert dd.check_records(records, new={new}) == [] + assert len(dd.check_records(records)) == 1 diff --git a/data_validation/tests/test_dsgai_mapping_check.py b/data_validation/tests/test_dsgai_mapping_check.py index cfc54ea..3bf3e08 100644 --- a/data_validation/tests/test_dsgai_mapping_check.py +++ b/data_validation/tests/test_dsgai_mapping_check.py @@ -1,12 +1,62 @@ -"""Tests for dsgai_mapping_check.py.""" -import pytest +"""Tests for dsgai_mapping_check.py and the DSGAI reference lists.""" +from __future__ import annotations + +import json +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators import dsgai_mapping_check as dm # noqa: E402 +from validators._common import ERROR, REFERENCE_DIR, TAXONOMY_PATH, WARN, load_taxonomy_ids # noqa: E402 + +VALID = load_taxonomy_ids() +P = Path("entry.json") + + +def check(data): + return dm.check_data(P, data, VALID) + def test_valid_mapping(): - """A file referencing DSGAI01 through DSGAI21 should pass.""" - # TODO: Implement when dsgai_mapping_check is built - pass + assert check({"dsgai_mapping": [f"DSGAI{n:02d}" for n in range(1, 7)]}) == [] + def test_invalid_dsgai_id(): - """A file referencing DSGAI99 should fail.""" - # TODO: Implement when dsgai_mapping_check is built - pass + [f] = check({"dsgai_mapping": ["DSGAI01", "DSGAI99"]}) + assert f.level == ERROR and "DSGAI99" in f.message and f.location == "/dsgai_mapping/1" + + +def test_unknown_id_in_free_text(): + [f] = check({"notes": "Related to DSGAI22."}) + assert f.level == ERROR and "DSGAI22" in f.message + + +def test_mapping_value_must_be_an_id(): + [f] = check({"risks": [{"dsgai_id": "Sensitive Data Leakage"}]}) + assert f.level == ERROR and f.location == "/risks/0/dsgai_id" + + +def test_internal_slugs_are_not_dsgai_ids(): + assert check({"related": ["DSGAI-VULN-2024-x", "DSGAI-EXP-y"]}) == [] + + +def test_duplicate_and_overlong_mappings_warn(): + findings = check({"dsgai_mapping": ["DSGAI01", "DSGAI01"]}) + assert [f.level for f in findings] == [WARN] + findings = check({"dsgai_mapping": [f"DSGAI{n:02d}" for n in range(1, 9)]}) + assert [f.level for f in findings] == [WARN] + + +def test_record_dsgai_ids_reads_nested_risks(): + record = {"dsgai_mapping": ["DSGAI01"], "risks_identified": [{"dsgai_id": "DSGAI14"}], + "existing_controls": [{"addresses_dsgai": ["DSGAI02"]}]} + assert dm.record_dsgai_ids(record) == {"DSGAI01", "DSGAI14"} + + +def test_reference_lists_agree(): + """dsgai_entries.json (read by the Turkish validator) matches the shared taxonomy.""" + shared = json.loads(TAXONOMY_PATH.read_text(encoding="utf-8"))["entries"] + legacy = json.loads((REFERENCE_DIR / "dsgai_entries.json").read_text(encoding="utf-8")) + assert [(e["id"], e["name"]) for e in shared] == [(e["id"], e["name"]) for e in legacy] + assert [e["id"] for e in shared] == [f"DSGAI{n:02d}" for n in range(1, 22)] diff --git a/data_validation/tests/test_qc_tools.py b/data_validation/tests/test_qc_tools.py new file mode 100644 index 0000000..048407a --- /dev/null +++ b/data_validation/tests/test_qc_tools.py @@ -0,0 +1,68 @@ +"""Tests for the QC tools against throwaway dataset trees.""" +from __future__ import annotations + +import json +import shutil +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from qc_tools import anomaly_detector, bias_report, consistency_check # noqa: E402 +from validators._common import ERROR, TAXONOMY_PATH # noqa: E402 + + +def make_tree(tmp_path: Path, datasets: dict[str, list[dict]]) -> Path: + root = tmp_path / "datasets" + (root / "_shared").mkdir(parents=True) + shutil.copy(TAXONOMY_PATH, root / "_shared" / "dsgai_taxonomy.json") + for name, records in datasets.items(): + entries = root / name / "entries" + entries.mkdir(parents=True) + for i, rec in enumerate(records): + (entries / f"{i}.json").write_text(json.dumps(rec)) + return root + + +def test_bias_report_counts_and_gaps(tmp_path): + root = make_tree(tmp_path, {"vulnerability_dataset": [ + {"vulnerability_id": "V-1", "dsgai_mapping": ["DSGAI01"], "severity": {"qualitative": "High"}}, + {"vulnerability_id": "V-2", "dsgai_mapping": ["DSGAI01", "DSGAI02"], "severity": {"qualitative": "Low"}}, + ]}) + report = bias_report.build_report(root) + assert "| DSGAI01 | Sensitive Data Leakage | 2 | 2 |" in report + assert "DSGAI entries no record maps to: DSGAI03" in report + assert "**severity**: High 1 (50%), Low 1 (50%)" in report + + +def test_anomaly_detector_flags_concentration_and_outliers(tmp_path): + records = [{"id": f"R-{i}", "dsgai_mapping": ["DSGAI01"], "severity": "High"} for i in range(11)] + records.append({"id": "R-big", "dsgai_mapping": [f"DSGAI{n:02d}" for n in range(1, 9)], "severity": "Low"}) + root = make_tree(tmp_path, {"d": records}) + messages = [f.message for f in anomaly_detector.detect(root)] + assert any("map to DSGAI01" in m for m in messages) + assert any(m.startswith("R-big maps to 8") for m in messages) + assert any(m.startswith("severity is 'High'") for m in messages) + + +def test_anomaly_detector_skips_small_datasets(tmp_path): + root = make_tree(tmp_path, {"d": [{"id": "R-1", "dsgai_mapping": ["DSGAI01"]}]}) + assert anomaly_detector.detect(root) == [] + + +def test_consistency_check_dangling_internal_reference(tmp_path): + root = make_tree(tmp_path, { + "vulnerability_dataset": [{"vulnerability_id": "DSGAI-VULN-real"}, {"vulnerability_id": "CVE-2024-0001"}], + "exploit_dataset": [{"exploit_id": "DSGAI-EXP-a", + "related_vulnerabilities": ["DSGAI-VULN-real", "DSGAI-VULN-typo", "CVE-2024-0001", "CVE-2024-9999"]}], + }) + findings, missing_cves = consistency_check.check(root) + assert [(f.level, f.message) for f in findings] == [(ERROR, "DSGAI-VULN-typo is not an entry in vulnerability_dataset")] + assert missing_cves == ["CVE-2024-9999"] + + +def test_qc_tools_run_on_the_real_datasets(): + """The real datasets have no dangling internal references and every DSGAI entry is covered.""" + findings, _ = consistency_check.check() + assert findings == [] + assert "DSGAI entries no record maps to: none." in bias_report.build_report() diff --git a/data_validation/tests/test_run_all_checks.py b/data_validation/tests/test_run_all_checks.py index 544a6a4..29ae53f 100644 --- a/data_validation/tests/test_run_all_checks.py +++ b/data_validation/tests/test_run_all_checks.py @@ -56,3 +56,50 @@ def test_nothing_to_run_is_not_a_pass(tmp_path, monkeypatch): def test_missing_directory(tmp_path): assert run_all_checks.main(["--dataset", str(tmp_path / "nope")]) == 2 + + +def test_nested_validator_is_found_and_tests_dir_is_not(tmp_path, monkeypatch, capsys): + root = make_tree(tmp_path, {"a": None}) + (root / "a" / "sub").mkdir() + (root / "a" / "sub" / "validate.py").write_text(FAILING) + (root / "a" / "tests").mkdir() + (root / "a" / "tests" / "validate.py").write_text(FAILING) + monkeypatch.setattr(run_all_checks, "DATASETS_ROOT", root) + assert run_all_checks.main(["--no-shared"]) == 1 + out = capsys.readouterr().out + assert "FAIL a/sub" in out + assert "1 validator(s) ran, 1 failed, 0 dataset(s) have no validator yet" in out + + +def test_validator_timeout_is_an_error(tmp_path): + d = tmp_path / "slow" + d.mkdir() + (d / "validate.py").write_text("import time\ntime.sleep(10)\n") + status, output = run_all_checks.run_validator(d, timeout=0.5) + assert status == "ERROR" and "timed out" in output + + +def test_shared_check_error_fails_the_run(tmp_path, monkeypatch, capsys): + root = make_tree(tmp_path, {"a": None}) + (root / "a" / "entry.json").write_text('{"id": "X-1", "dsgai_mapping": ["DSGAI99"]}') + monkeypatch.setattr(run_all_checks, "DATASETS_ROOT", root) + assert run_all_checks.main([]) == 1 + out = capsys.readouterr().out + assert "FAIL a (1 file(s), 1 error(s), 1 warning(s))" in out # warning: no schema + assert "DSGAI99 is not in the DSGAI taxonomy" in out + + +def test_warnings_pass_unless_strict(tmp_path, monkeypatch): + root = make_tree(tmp_path, {"a": PASSING}) + (root / "a" / "entry.json").write_text('{"id": "X-1", "contact": "jane@realcorp.io"}') + monkeypatch.setattr(run_all_checks, "DATASETS_ROOT", root) + assert run_all_checks.main([]) == 0 + assert run_all_checks.main(["--strict"]) == 1 + + +def test_files_covered_by_a_dataset_validator_skip_the_shared_schema_check(tmp_path, monkeypatch, capsys): + root = make_tree(tmp_path, {"a": PASSING}) + (root / "a" / "entry.json").write_text('{"id": "X-1"}') + monkeypatch.setattr(run_all_checks, "DATASETS_ROOT", root) + assert run_all_checks.main([]) == 0 + assert "no schema found" not in capsys.readouterr().out diff --git a/data_validation/tests/test_schema_validator.py b/data_validation/tests/test_schema_validator.py index 6250798..c2cdb2e 100644 --- a/data_validation/tests/test_schema_validator.py +++ b/data_validation/tests/test_schema_validator.py @@ -1,16 +1,91 @@ -"""Tests for schema_validator.py using sample fixtures.""" -import pytest, json, os +"""Tests for schema_validator.py.""" +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators import schema_validator # noqa: E402 +from validators._common import DATASETS_ROOT, ERROR, SCHEMAS_DIR, WARN # noqa: E402 + +FIXTURES = Path(__file__).resolve().parent / "fixtures" +VULN_SCHEMA = DATASETS_ROOT / "vulnerability_dataset" / "schema.json" -FIXTURES_DIR = os.path.join(os.path.dirname(__file__), "fixtures") def test_valid_vulnerability(): - """A correctly formatted vulnerability entry should pass validation.""" - # TODO: Implement when schema_validator.validate_file() is built - filepath = os.path.join(FIXTURES_DIR, "valid_vulnerability.json") - assert os.path.exists(filepath), "Fixture file missing" + assert schema_validator.validate_file(FIXTURES / "valid_vulnerability.json", VULN_SCHEMA) == [] + def test_invalid_vulnerability(): - """An entry with missing required fields should fail validation.""" - # TODO: Implement when schema_validator.validate_file() is built - filepath = os.path.join(FIXTURES_DIR, "invalid_vulnerability.json") - assert os.path.exists(filepath), "Fixture file missing" + findings = schema_validator.validate_file(FIXTURES / "invalid_vulnerability.json", VULN_SCHEMA) + messages = " ".join(f.message for f in findings) + assert all(f.level == ERROR for f in findings) + assert "'title' is a required property" in messages + assert "not_a_valid_component" in messages + + +def test_valid_incident_against_fallback_schema(): + assert schema_validator.validate_file(FIXTURES / "valid_incident.json", SCHEMAS_DIR / "incident.schema.json") == [] + + +def test_finds_schema_from_dollar_schema(tmp_path): + (tmp_path / "schema.json").write_text(json.dumps({"type": "object", "required": ["x"]})) + entries = tmp_path / "entries" + entries.mkdir() + entry = entries / "e.json" + entry.write_text(json.dumps({"$schema": "../schema.json"})) + findings = schema_validator.validate_file(entry, dataset_root=tmp_path) + assert [f.message for f in findings] == ["'x' is a required property"] + + +def test_finds_nearest_schema_json(tmp_path): + (tmp_path / "schema.json").write_text(json.dumps({"type": "object", "required": ["x"]})) + sub = tmp_path / "entries" + sub.mkdir() + (sub / "e.json").write_text(json.dumps({"x": 1})) + assert schema_validator.find_schema(sub / "e.json", {"x": 1}, tmp_path) == tmp_path / "schema.json" + + +def test_fallback_schema_for_rag_dataset(): + rag = DATASETS_ROOT / "rag_dataset" + assert schema_validator.find_schema(rag / "entries" / "RAG-0001.json", {}, rag) == SCHEMAS_DIR / "rag.schema.json" + + +def test_missing_schema_is_an_error(tmp_path): + entry = tmp_path / "e.json" + entry.write_text(json.dumps({"$schema": "./nope.json"})) + [finding] = schema_validator.validate_file(entry, dataset_root=tmp_path) + assert finding.level == ERROR and "schema not found" in finding.message + + +def test_no_schema_is_a_warning(tmp_path): + entry = tmp_path / "e.json" + entry.write_text("{}") + [finding] = schema_validator.validate_file(entry, dataset_root=tmp_path) + assert finding.level == WARN + + +def test_relative_ref_between_schema_files(tmp_path): + (tmp_path / "inner.json").write_text(json.dumps({"type": "object", "required": ["y"]})) + (tmp_path / "outer.json").write_text(json.dumps({"$schema": "https://json-schema.org/draft/2020-12/schema", + "$ref": "inner.json"})) + entry = tmp_path / "e.json" + entry.write_text("{}") + findings = schema_validator.validate_file(entry, tmp_path / "outer.json") + assert [f.message for f in findings] == ["'y' is a required property"] + + +@pytest.mark.parametrize("example", sorted(DATASETS_ROOT.glob("*/example.json")), ids=lambda p: p.parent.name) +def test_dataset_examples_pass_their_schema(example): + assert schema_validator.validate_file(example, dataset_root=example.parent) == [] + + +@pytest.mark.parametrize("stub", ["exploit", "vulnerability", "riskassessment", "agentdataflow_trace"]) +def test_superseded_stub_schemas_resolve_to_dataset_schema(stub): + """data_validation/schemas/.schema.json $refs the dataset-local schema.""" + example = next(DATASETS_ROOT.glob(f"{stub.split('_')[0]}*/example.json")) + assert schema_validator.validate_file(example, SCHEMAS_DIR / f"{stub}.schema.json") == [] diff --git a/data_validation/validators/_common.py b/data_validation/validators/_common.py new file mode 100644 index 0000000..8a3e16e --- /dev/null +++ b/data_validation/validators/_common.py @@ -0,0 +1,208 @@ +"""Shared helpers for the validators and QC tools. + +Every check reports Finding objects. An ERROR is something that is wrong +(an unknown DSGAI ID, a schema violation, a malformed CVE ID) and fails a +run. A WARN is a heuristic hit (something that looks like PII, a likely +duplicate) that a reviewer should look at, and does not fail a run. +""" +from __future__ import annotations + +import csv +import json +import sys +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Iterator + +DV_ROOT = Path(__file__).resolve().parent.parent +REPO_ROOT = DV_ROOT.parent +DATASETS_ROOT = REPO_ROOT / "datasets" +REFERENCE_DIR = DV_ROOT / "reference_data" +SCHEMAS_DIR = DV_ROOT / "schemas" +TAXONOMY_PATH = DATASETS_ROOT / "_shared" / "dsgai_taxonomy.json" + +ERROR = "ERROR" +WARN = "WARN" + +# Folders under a dataset that hold tooling or deliberately invalid fixtures, +# not data. +SKIP_DIRS = {"tests", "fixtures", "__pycache__", "node_modules"} + +# Container keys: a file whose top-level object holds a list of records under +# one of these keys is treated as one record per list item. +CONTAINER_KEYS = ("test_cases", "entries", "cases") + +# The field that identifies a record, in the order they are tried. +ID_FIELDS = ( + "vulnerability_id", "exploit_id", "assessment_id", "trace_id", + "testcase_id", "document_id", "incident_id", "id", +) + + +@dataclass(frozen=True) +class Finding: + level: str + check: str + path: Path + location: str + message: str + + def format(self, base: Path | None = None) -> str: + shown = self.path + if base is not None: + try: + shown = self.path.resolve().relative_to(base.resolve()) + except ValueError: + pass + where = f"{shown.as_posix()}: {self.location}" if self.location else shown.as_posix() + return f"{self.level:<5} [{self.check}] {where}: {self.message}" + + +def is_data_file(path: Path, root: Path) -> bool: + """A JSON file under root that holds data, not a schema or a test fixture.""" + if path.suffix != ".json" or path.name == "schema.json" or path.name.endswith(".schema.json"): + return False + rel = path.relative_to(root).parts[:-1] + return not any(p in SKIP_DIRS or p.startswith((".", "_")) for p in rel) + + +def iter_data_files(root: Path) -> list[Path]: + """Data files under root (a dataset directory, or a single file).""" + if root.is_file(): + return [root] + return sorted(p for p in root.rglob("*.json") if is_data_file(p, root)) + + +def load_json(path: Path, check: str) -> tuple[Any, Finding | None]: + try: + return json.loads(path.read_text(encoding="utf-8")), None + except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc: + return None, Finding(ERROR, check, path, "", f"cannot be read as JSON: {exc}") + + +def iter_records(data: Any) -> Iterator[tuple[str, dict]]: + """(JSON pointer, record) for each record in a file.""" + if isinstance(data, list): + for i, item in enumerate(data): + if isinstance(item, dict): + yield f"/{i}", item + return + if not isinstance(data, dict): + return + for key in CONTAINER_KEYS: + items = data.get(key) + if isinstance(items, list) and items and all(isinstance(i, dict) for i in items): + for i, item in enumerate(items): + yield f"/{key}/{i}", item + return + yield "", data + + +def record_id(record: dict) -> str | None: + for field in ID_FIELDS: + value = record.get(field) + if isinstance(value, str) and value: + return value + return None + + +def walk_strings(obj: Any, pointer: str = "", key: str | None = None) -> Iterator[tuple[str, str | None, str]]: + """(JSON pointer, key the string sits under, string) for every string in obj.""" + if isinstance(obj, str): + yield pointer or "", key, obj + elif isinstance(obj, dict): + for k, v in obj.items(): + yield from walk_strings(v, f"{pointer}/{k}", str(k)) + elif isinstance(obj, list): + for i, v in enumerate(obj): + yield from walk_strings(v, f"{pointer}/{i}", key) + + +def load_taxonomy_ids(datasets_root: Path | None = None) -> set[str]: + """DSGAI IDs from datasets/_shared/dsgai_taxonomy.json.""" + path = TAXONOMY_PATH + if datasets_root is not None: + candidate = datasets_root / "_shared" / "dsgai_taxonomy.json" + if candidate.is_file(): + path = candidate + data = json.loads(path.read_text(encoding="utf-8")) + return {e["id"] for e in data["entries"]} + + +def load_reference_ids(filename: str, column: str) -> set[str]: + with open(REFERENCE_DIR / filename, newline="", encoding="utf-8") as fh: + return {row[column] for row in csv.DictReader(fh) if row.get(column)} + + +def dataset_dirs(datasets_root: Path = DATASETS_ROOT) -> list[Path]: + """Dataset directories, skipping shared (_shared) and hidden folders.""" + return sorted(p for p in datasets_root.iterdir() if p.is_dir() and not p.name.startswith(("_", "."))) + + +def load_dataset_records(datasets_root: Path = DATASETS_ROOT) -> dict[str, list[tuple[Path, dict]]]: + """{dataset name: [(file, record), ...]} for every dataset; example.json files skipped.""" + out: dict[str, list[tuple[Path, dict]]] = {} + for d in dataset_dirs(datasets_root): + records = [] + for path in iter_data_files(d): + if path.name == "example.json": + continue + data, err = load_json(path, "load") + if err is None: + records.extend((path, rec) for _ptr, rec in iter_records(data)) + out[d.name] = records + return out + + +def dataset_root_of(path: Path, datasets_root: Path = DATASETS_ROOT) -> Path: + """The top-level dataset directory a file belongs to (its parent if outside datasets/).""" + path = path.resolve() + try: + rel = path.relative_to(datasets_root.resolve()) + except ValueError: + return path.parent if path.is_file() else path + return datasets_root.resolve() / rel.parts[0] if len(rel.parts) > 1 else path + + +def add_target_args(parser) -> None: + parser.add_argument("--file", action="append", default=[], + help="A data file to check (repeatable)") + parser.add_argument("--dataset", action="append", default=[], + help="A dataset directory to check (repeatable; default: every dataset)") + + +def resolve_targets(args, datasets_root: Path = DATASETS_ROOT) -> list[Path] | None: + """Files named by --file/--dataset, or every data file. None if a path is missing.""" + files: list[Path] = [] + for f in args.file: + p = Path(f) + if not p.is_file(): + print(f"File not found: {f}") + return None + files.append(p.resolve()) + for d in args.dataset: + p = Path(d) + if not p.is_dir(): + print(f"Directory not found: {d}") + return None + files.extend(iter_data_files(p.resolve())) + if not args.file and not args.dataset: + files = iter_data_files(datasets_root) + return files + + +def print_findings(findings: list[Finding], base: Path | None = None) -> int: + """Print findings and a summary line. Returns the exit code (1 if any ERROR).""" + for f in findings: + print(f.format(base)) + errors = sum(f.level == ERROR for f in findings) + warnings = len(findings) - errors + print(f"{errors} error(s), {warnings} warning(s)") + return 1 if errors else 0 + + +def ensure_utf8_stdout() -> None: + """Dataset text is multilingual; don't crash on a cp1252 console.""" + for stream in (sys.stdout, sys.stderr): + if hasattr(stream, "reconfigure"): + stream.reconfigure(errors="replace") diff --git a/data_validation/validators/anonymization_scanner.py b/data_validation/validators/anonymization_scanner.py index 6f98556..2d66c76 100644 --- a/data_validation/validators/anonymization_scanner.py +++ b/data_validation/validators/anonymization_scanner.py @@ -1,30 +1,137 @@ """ -Scans dataset entries for residual PII, credentials, hostnames, and identifying information. +Scans data files for things that look like residual PII, credentials or +internal network detail. + +Every hit is a WARN: the patterns cannot tell a real secret from a synthetic +one, and the prompt-injection datasets contain fake credentials on purpose. +A reviewer decides. Use obviously fake values to stay quiet: RFC 2606 +domains (example.com, *.example, *.test, *.invalid), RFC 5737 documentation +addresses (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24), or a placeholder +in angle or square brackets such as or [API_KEY]. + +The patterns match the ones in +datasets/agentdataflow_toolexchange_traces/validate.py, which fails its own +dataset on them. Usage: - python anonymization_scanner.py --file ../../datasets/incident_dataset/INC-0001.json + python anonymization_scanner.py --file ../../datasets/rag_dataset/entries/RAG-0001.json + python anonymization_scanner.py # every dataset """ -import argparse, re - -PII_PATTERNS = { - "email": r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}", - "ip_address": r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b", - "api_key_generic": r"(?i)(sk-|api[_-]?key|bearer\s+[a-z0-9])", - "aws_key": r"AKIA[0-9A-Z]{16}", - "phone": r"\b\d{3}[-.]?\d{3}[-.]?\d{4}\b", - "ssn": r"\b\d{3}-\d{2}-\d{4}\b", -} - -def scan_file(filepath): - """Scan a file for PII patterns. Returns (clean: bool, findings: list).""" - # TODO: Implement full scanning logic - pass - -def main(): - parser = argparse.ArgumentParser(description="Scan for residual PII and credentials") - parser.add_argument("--file", required=True, help="Path to file to scan") - args = parser.parse_args() - print("Anonymization scanner stub - contributions welcome!") +from __future__ import annotations + +import argparse +import ipaddress +import re +import sys +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import ( # noqa: E402 + DATASETS_ROOT, WARN, Finding, add_target_args, ensure_utf8_stdout, + load_json, print_findings, resolve_targets, walk_strings, +) + +CHECK = "anonymization" + +PII_PATTERNS: list[tuple[str, re.Pattern[str]]] = [ + ("email address", re.compile(r"\b[A-Za-z0-9._%+-]+@([A-Za-z0-9-]+\.)+[A-Za-z]{2,}\b")), + ("openai-style key", re.compile(r"\bsk-[A-Za-z0-9_-]{16,}")), + ("aws access key id", re.compile(r"\bAKIA[0-9A-Z]{16}\b")), + ("github token", re.compile(r"\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{20,}")), + ("github fine-grained pat", re.compile(r"\bgithub_pat_[A-Za-z0-9_]{20,}")), + ("slack token", re.compile(r"\bxox[abposr]-[A-Za-z0-9-]{10,}")), + ("google api key", re.compile(r"\bAIza[0-9A-Za-z_-]{35}\b")), + ("json web token", re.compile(r"\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}")), + ("private key block", re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----")), + ("bearer credential", re.compile(r"\bBearer\s+[A-Za-z0-9._~+/-]{20,}")), + ("secret-looking assignment", re.compile( + r"\b(?:password|passwd|secret|api[_-]?key|access[_-]?key|client[_-]?secret)\s*[:=]\s*(?![<\[{]|\.{3}|\*{3})\S{4,}", re.IGNORECASE)), + ("us social security number", re.compile(r"\b\d{3}-\d{2}-\d{4}\b")), + ("internal hostname", re.compile( + r"\b[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?)+\.(?:internal|corp|lan|intra)\b", + re.IGNORECASE)), + ("user home path", re.compile(r"(?:/Users/|/home/|[A-Za-z]:\\Users\\)[A-Za-z0-9._-]+")), +] + +IPV4_RE = re.compile(r"(?]+>|\[[^\[\]]+\])$") +RESERVED_EMAIL_DOMAINS = re.compile(r"(^|\.)(example\.(com|org|net)|example|test|invalid|localhost)$", re.IGNORECASE) +DOC_NETWORKS = [ipaddress.ip_network(n) for n in ("192.0.2.0/24", "198.51.100.0/24", "203.0.113.0/24")] + + +def _ip_finding(text: str, before: str, after: str) -> str | None: + try: + ip = ipaddress.ip_address(text) + except ValueError: + return None # e.g. a version number like 1.2.300.4 + # Link-local covers 169.254.169.254, the well-known cloud metadata address + # that SSRF write-ups cite; it identifies no one. + if (any(ip in n for n in DOC_NETWORKS) or ip.is_loopback or ip.is_unspecified + or ip.is_multicast or ip.is_link_local): + return None + if ip.is_private: + return "private ip address" + if before.endswith(NET_CONTEXT_BEFORE) or PORT_AFTER_RE.match(after): + return "public ip address" + return None + + +def scan_value(value: str) -> list[tuple[str, str]]: + """(kind, matched text) for each suspicious thing in one string.""" + if PLACEHOLDER_RE.match(value.strip()): + return [] + hits = [] + for kind, pattern in PII_PATTERNS: + for m in pattern.finditer(value): + text = m.group(0) + if kind == "email address" and RESERVED_EMAIL_DOMAINS.search(text.split("@", 1)[1]): + continue + hits.append((kind, text)) + for m in IPV4_RE.finditer(value): + kind = _ip_finding(m.group(1), value[:m.start()], value[m.end():]) + if kind: + hits.append((kind, m.group(1))) + return hits + + +def _redact(text: str) -> str: + return text if len(text) <= 12 else f"{text[:8]}...{text[-2:]}" + + +def check_data(path: Path, data) -> list[Finding]: + findings = [] + for pointer, _key, value in walk_strings(data): + for kind, text in scan_value(value): + findings.append(Finding(WARN, CHECK, path, pointer, f"looks like a {kind}: {_redact(text)!r}")) + return findings + + +def scan_file(path: Path) -> list[Finding]: + data, err = load_json(path, CHECK) + if err: + return [err] + return check_data(path, data) + + +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Scan for residual PII, credentials and internal network detail") + add_target_args(parser) + args = parser.parse_args(argv) + files = resolve_targets(args) + if files is None: + return 2 + findings = [f for p in files for f in scan_file(p)] + print(f"Checked {len(files)} file(s)") + return print_findings(findings, DATASETS_ROOT.parent) + if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/validators/completeness_check.py b/data_validation/validators/completeness_check.py deleted file mode 100644 index c40f53c..0000000 --- a/data_validation/validators/completeness_check.py +++ /dev/null @@ -1,16 +0,0 @@ -""" -Reports missing required fields and invalid optional field values. - -Usage: - python completeness_check.py --file ../../datasets/incident_dataset/INC-0001.json -""" -import argparse - -def main(): - parser = argparse.ArgumentParser(description="Check dataset entry completeness") - parser.add_argument("--file", required=True, help="Path to file to check") - args = parser.parse_args() - print("Completeness checker stub - contributions welcome!") - -if __name__ == "__main__": - main() diff --git a/data_validation/validators/crossref_validator.py b/data_validation/validators/crossref_validator.py index 79e9476..e6c98f0 100644 --- a/data_validation/validators/crossref_validator.py +++ b/data_validation/validators/crossref_validator.py @@ -1,16 +1,135 @@ """ -Checks CVE, CWE, MITRE ATLAS, and framework control ID references against known-valid lists. +Checks CVE, GHSA, CWE and MITRE ATLAS identifiers. + +ERROR: a field whose whole value is meant to be one of these IDs but is + malformed (e.g. "CVE-24-1234"); a CVE year in the future; a CWE ID + that is not in reference_data/cwe_ids.csv; an ATLAS technique ID that + is not in reference_data/mitre_atlas_techniques.csv. ATLAS and CWE IDs + are checked wherever they appear, including free text. +WARN: a CWE that MITRE has deprecated, or an ATLAS ID that a later ATLAS + release retired (merged into another technique); something in free text that starts + like an ID but is malformed (often a placeholder such as CVE-XXXX-XXXX). + +CVE and GHSA IDs are format-checked only: there is no offline list to check +that they exist. Framework control IDs (ISO, NIST, CIS) are not checked; see +reference_data/SOURCES.md. Usage: - python crossref_validator.py --file ../../datasets/vulnerability_dataset/VULN-0001.json + python crossref_validator.py --file ../../datasets/exploit_dataset/entries/AML.T0051.json + python crossref_validator.py # every dataset """ +from __future__ import annotations + import argparse +import csv +import re +import sys +from datetime import date +from functools import lru_cache +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import ( # noqa: E402 + DATASETS_ROOT, ERROR, REFERENCE_DIR, WARN, Finding, add_target_args, + ensure_utf8_stdout, load_json, print_findings, resolve_targets, walk_strings, +) + +CHECK = "crossref" + +# (name, strict full form, loose "starts like one" form) +ID_KINDS = ( + ("CVE", re.compile(r"CVE-(\d{4})-\d{4,}"), re.compile(r"\bCVE-[A-Za-z0-9-]+")), + ("GHSA", re.compile(r"GHSA(-[23456789cfghjmpqrvwx]{4}){3}"), re.compile(r"\bGHSA-[A-Za-z0-9-]+")), + ("CWE", re.compile(r"CWE-\d+"), re.compile(r"\bCWE-[A-Za-z0-9]+")), + ("ATLAS", re.compile(r"AML\.T\d{4}(\.\d{3})?"), re.compile(r"\bAML\.T[A-Za-z0-9.]*[A-Za-z0-9]")), +) + + +@lru_cache(maxsize=None) +def atlas_status() -> dict[str, str]: + with open(REFERENCE_DIR / "mitre_atlas_techniques.csv", newline="", encoding="utf-8") as fh: + return {row["technique_id"]: row["status"] for row in csv.DictReader(fh)} + + +@lru_cache(maxsize=None) +def cwe_status() -> dict[str, str]: + with open(REFERENCE_DIR / "cwe_ids.csv", newline="", encoding="utf-8") as fh: + return {row["cwe_id"]: row["status"] for row in csv.DictReader(fh)} + + +def _check_id(kind: str, value: str, m: re.Match) -> tuple[str, str] | None: + """(level, message) for a well-formed ID that is still wrong, else None.""" + if kind == "CVE" and int(m.group(1)) > date.today().year: + return ERROR, f"{value}: CVE year is in the future" + if kind == "CWE": + status = cwe_status().get(value) + if status is None: + return ERROR, f"{value} is not a CWE ID (not in reference_data/cwe_ids.csv)" + if status == "Deprecated": + return WARN, f"{value} is deprecated by MITRE; use the CWE it points to" + if kind == "ATLAS": + status = atlas_status().get(value) + if status is None: + return ERROR, f"{value} is not a MITRE ATLAS technique (not in reference_data/mitre_atlas_techniques.csv)" + if status == "retired": + return WARN, f"{value} was retired by MITRE ATLAS; check the ATLAS changelog for the technique that replaced it" + return None + + +def check_data(path: Path, data) -> list[Finding]: + findings: list[Finding] = [] + for pointer, _key, value in walk_strings(data): + stripped = value.strip() + whole_value = False + for kind, strict, loose in ID_KINDS: + if not loose.fullmatch(stripped): + continue + # The whole value is meant to be an ID of this kind. + whole_value = True + m = strict.fullmatch(stripped) + if not m: + findings.append(Finding(ERROR, CHECK, path, pointer, f"{stripped!r} is not a well-formed {kind} ID")) + elif problem := _check_id(kind, stripped, m): + findings.append(Finding(problem[0], CHECK, path, pointer, problem[1])) + if whole_value: + continue + for kind, strict, loose in ID_KINDS: + for token in loose.finditer(value): + text = token.group(0) + m = strict.match(text) + if not m or m.end() != len(text): + # "CVE-2024-1234-style" or a sub-technique suffix is not malformed. + if m and kind in ("CVE", "GHSA", "CWE") and text[m.end()] == "-": + text = m.group(0) + else: + findings.append(Finding(WARN, CHECK, path, pointer, f"{text!r} looks like a {kind} ID but is not well-formed")) + continue + if problem := _check_id(kind, m.group(0), m): + findings.append(Finding(problem[0], CHECK, path, pointer, problem[1])) + return findings + + +def check_file(path: Path) -> list[Finding]: + data, err = load_json(path, CHECK) + if err: + return [err] + return check_data(path, data) + + +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Check CVE, GHSA, CWE and MITRE ATLAS identifiers") + add_target_args(parser) + args = parser.parse_args(argv) + files = resolve_targets(args) + if files is None: + return 2 + findings = [f for p in files for f in check_file(p)] + print(f"Checked {len(files)} file(s)") + return print_findings(findings, DATASETS_ROOT.parent) -def main(): - parser = argparse.ArgumentParser(description="Validate external references (CVE, CWE, ATLAS, framework controls)") - parser.add_argument("--file", required=True, help="Path to file to check") - args = parser.parse_args() - print("Cross-reference validator stub - contributions welcome!") if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/validators/dedup_checker.py b/data_validation/validators/dedup_checker.py index 4ee1258..09a9fc2 100644 --- a/data_validation/validators/dedup_checker.py +++ b/data_validation/validators/dedup_checker.py @@ -1,17 +1,163 @@ """ -Identifies potential duplicate entries within and across datasets. +Finds duplicate records within a dataset. + +ERROR: two records in one dataset with the same ID. +WARN: two records whose text is identical, or whose word-trigram (Jaccard) + similarity is at least the threshold (default 0.8). Similar is not + necessarily wrong: a reviewer decides. + +A record's text is its string values of 40 characters or more, minus URLs +and minus any string that appears in more than half of the dataset's records +(shared provenance or licence text would otherwise make every pair look +alike). example.json files are skipped: they copy a real entry on purpose. +Similarity is lexical only; paraphrases are not caught. Usage: - python dedup_checker.py --file ../../datasets/incident_dataset/INC-0001.json --dataset ../../datasets/incident_dataset/ + python dedup_checker.py # every dataset + python dedup_checker.py --dataset ../../datasets/exploit_dataset/ + python dedup_checker.py --file new_entry.json --dataset ../../datasets/exploit_dataset/ """ +from __future__ import annotations + import argparse +import re +import sys +import unicodedata +from collections import Counter +from itertools import combinations +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import ( # noqa: E402 + DATASETS_ROOT, ERROR, WARN, Finding, ensure_utf8_stdout, iter_data_files, + iter_records, load_json, print_findings, record_id, walk_strings, +) + +CHECK = "dedup" +DEFAULT_THRESHOLD = 0.8 +MIN_TEXT_LEN = 40 +URL_RE = re.compile(r"^\w+://\S+$") +WORD_RE = re.compile(r"\w+") + + +class Record: + def __init__(self, path: Path, pointer: str, data: dict): + self.path = path + self.pointer = pointer + self.id = record_id(data) + self.strings = [ + v for _p, _k, v in walk_strings(data) + if len(v) >= MIN_TEXT_LEN and not URL_RE.match(v.strip()) + ] + self.text = "" + self.shingles: frozenset[tuple[str, ...]] = frozenset() + + @property + def label(self) -> str: + return self.id or f"{self.path.name}{self.pointer}" + + +def _normalize(text: str) -> str: + return " ".join(unicodedata.normalize("NFKC", text).casefold().split()) + + +def _shingles(text: str) -> frozenset[tuple[str, ...]]: + words = WORD_RE.findall(text) + if len(words) < 3: + return frozenset([tuple(words)]) if words else frozenset() + return frozenset(zip(words, words[1:], words[2:])) + + +def load_records(files: list[Path]) -> tuple[list[Record], list[Finding]]: + records, findings = [], [] + for path in files: + if path.name == "example.json": + continue + data, err = load_json(path, CHECK) + if err: + findings.append(err) + continue + records.extend(Record(path, ptr, rec) for ptr, rec in iter_records(data)) + # Drop boilerplate: strings shared by more than half of the records, in + # the whole dataset or in one folder of it (a sub-collection such as one + # contributor's cases repeats its own provenance text in every file). + groups: dict[Path | None, list[Record]] = {None: records} + for r in records: + groups.setdefault(r.path.parent, []).append(r) + common: set[str] = set() + for group in groups.values(): + if len(group) >= 4: + counts = Counter(s for r in group for s in set(r.strings)) + common |= {s for s, n in counts.items() if n > len(group) / 2} + for r in records: + r.text = _normalize(" ".join(s for s in r.strings if s not in common)) + r.shingles = _shingles(r.text) + return records, findings + + +def check_records(records: list[Record], threshold: float = DEFAULT_THRESHOLD, + new: set[Path] | None = None) -> list[Finding]: + """Findings for duplicate IDs and similar text. With new, only pairs involving a new file.""" + findings: list[Finding] = [] + by_id: dict[str, Record] = {} + for r in records: + if r.id is None: + continue + first = by_id.setdefault(r.id, r) + if first is not r and (new is None or r.path in new or first.path in new): + findings.append(Finding(ERROR, CHECK, r.path, r.pointer, + f"ID {r.id} is also used by {first.path.name}{first.pointer}")) + for a, b in combinations(records, 2): + if new is not None and a.path not in new and b.path not in new: + continue + if not a.shingles or not b.shingles: + continue + if a.text == b.text: + score = 1.0 + else: + score = len(a.shingles & b.shingles) / len(a.shingles | b.shingles) + if score >= threshold: + findings.append(Finding(WARN, CHECK, b.path, b.pointer, + f"{b.label} is {score:.0%} similar to {a.label} ({a.path.name}); possible duplicate")) + return findings + + +def check_dataset(dataset: Path, threshold: float = DEFAULT_THRESHOLD) -> list[Finding]: + records, findings = load_records(iter_data_files(dataset)) + return findings + check_records(records, threshold) + + +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Find duplicate records within a dataset") + parser.add_argument("--dataset", action="append", default=[], + help="Dataset directory (repeatable; default: every dataset)") + parser.add_argument("--file", action="append", default=[], + help="New file(s) to compare against --dataset (repeatable)") + parser.add_argument("--threshold", type=float, default=DEFAULT_THRESHOLD, + help=f"Similarity that counts as a possible duplicate (default {DEFAULT_THRESHOLD})") + args = parser.parse_args(argv) + for p in args.dataset + args.file: + if not Path(p).exists(): + print(f"Not found: {p}") + return 2 + if args.file and len(args.dataset) != 1: + print("--file needs exactly one --dataset to compare against") + return 2 + datasets = [Path(d).resolve() for d in args.dataset] or sorted( + p for p in DATASETS_ROOT.iterdir() if p.is_dir() and not p.name.startswith(("_", "."))) + findings: list[Finding] = [] + if args.file: + new = {Path(f).resolve() for f in args.file} + records, findings = load_records(sorted(set(iter_data_files(datasets[0])) | new)) + findings += check_records(records, args.threshold, new) + else: + for d in datasets: + findings += check_dataset(d, args.threshold) + return print_findings(findings, DATASETS_ROOT.parent) -def main(): - parser = argparse.ArgumentParser(description="Check for duplicate entries") - parser.add_argument("--file", required=True, help="Path to new file to check for duplicates") - parser.add_argument("--dataset", required=True, help="Path to existing dataset directory") - args = parser.parse_args() - print("Dedup checker stub - contributions welcome!") if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/validators/dsgai_mapping_check.py b/data_validation/validators/dsgai_mapping_check.py index e1dfb5a..421719e 100644 --- a/data_validation/validators/dsgai_mapping_check.py +++ b/data_validation/validators/dsgai_mapping_check.py @@ -1,28 +1,95 @@ """ -Verifies DSGAI ID references exist (DSGAI01-DSGAI21) and flags implausible mappings. +Verifies DSGAI ID references against datasets/_shared/dsgai_taxonomy.json. + +ERROR: a DSGAI ID that is not in the taxonomy, wherever it appears (a + mapping field or free text), or a dsgai_mapping value that is not a + DSGAI ID at all. +WARN: the same ID listed twice in one mapping, or a mapping with more IDs + than MAX_MAPPINGS (usually a sign the entry was mapped to everything + that looked related rather than to the risks it actually shows). + +Whether a mapping is the right one is a reviewer judgment; this only catches +what can be checked mechanically. Usage: - python dsgai_mapping_check.py --file ../../datasets/incident_dataset/INC-0001.json + python dsgai_mapping_check.py --file ../../datasets/rag_dataset/entries/RAG-0001.json + python dsgai_mapping_check.py # every dataset """ -import argparse, json, os +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import ( # noqa: E402 + DATASETS_ROOT, ERROR, WARN, Finding, add_target_args, ensure_utf8_stdout, + iter_records, load_json, load_taxonomy_ids, print_findings, resolve_targets, walk_strings, +) + +CHECK = "dsgai" +# DSGAI followed directly by digits. Slugs such as DSGAI-VULN-... do not match. +DSGAI_TOKEN_RE = re.compile(r"\bDSGAI(\d+)\b") +# Keys whose values must each be exactly one DSGAI ID. +MAPPING_KEYS = {"dsgai_mapping", "dsgai_id", "addresses_dsgai"} +# Keys that say which risks a record shows (addresses_dsgai is a control's +# coverage, not a risk the record shows). Used by the QC reports. +RISK_KEYS = {"dsgai_mapping", "dsgai_id"} +MAX_MAPPINGS = 6 + + +def record_dsgai_ids(record) -> set[str]: + """DSGAI IDs a record maps to, wherever in it they are listed.""" + return {v.strip() for _p, k, v in walk_strings(record) if k in RISK_KEYS} + + +def check_data(path: Path, data, valid_ids: set[str]) -> list[Finding]: + findings: list[Finding] = [] + for pointer, key, value in walk_strings(data): + if key in MAPPING_KEYS and not DSGAI_TOKEN_RE.fullmatch(value.strip()): + findings.append(Finding(ERROR, CHECK, path, pointer, f"{value!r} is not a DSGAI ID (expected DSGAI01-DSGAI21)")) + continue + for m in DSGAI_TOKEN_RE.finditer(value): + if m.group(0) not in valid_ids: + findings.append(Finding(ERROR, CHECK, path, pointer, f"{m.group(0)} is not in the DSGAI taxonomy")) + for pointer, record in iter_records(data): + mapping = record.get("dsgai_mapping") + if not isinstance(mapping, list): + continue + seen = set() + for value in mapping: + if value in seen: + findings.append(Finding(WARN, CHECK, path, f"{pointer}/dsgai_mapping", f"{value} is listed more than once")) + seen.add(value) + if len(seen) > MAX_MAPPINGS: + findings.append(Finding(WARN, CHECK, path, f"{pointer}/dsgai_mapping", + f"maps to {len(seen)} DSGAI entries; check each is a risk the entry actually shows")) + return findings + -DSGAI_REFERENCE = os.path.join(os.path.dirname(__file__), "..", "reference_data", "dsgai_entries.json") +def check_file(path: Path, valid_ids: set[str] | None = None) -> list[Finding]: + data, err = load_json(path, CHECK) + if err: + return [err] + return check_data(path, data, valid_ids if valid_ids is not None else load_taxonomy_ids()) -def load_valid_ids(): - with open(DSGAI_REFERENCE, "r") as f: - entries = json.load(f) - return {entry["id"] for entry in entries} -def check_mappings(filepath, valid_ids): - """Check that all DSGAI references in a file are valid. Returns (passed: bool, errors: list).""" - # TODO: Implement mapping plausibility checks - pass +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Verify DSGAI ID references") + add_target_args(parser) + args = parser.parse_args(argv) + files = resolve_targets(args) + if files is None: + return 2 + valid = load_taxonomy_ids() + findings = [f for p in files for f in check_file(p, valid)] + print(f"Checked {len(files)} file(s)") + return print_findings(findings, DATASETS_ROOT.parent) -def main(): - parser = argparse.ArgumentParser(description="Verify DSGAI ID mappings in dataset files") - parser.add_argument("--file", required=True, help="Path to file to check") - args = parser.parse_args() - print("DSGAI mapping checker stub - contributions welcome!") if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/data_validation/validators/schema_validator.py b/data_validation/validators/schema_validator.py index a20948b..52a68b5 100644 --- a/data_validation/validators/schema_validator.py +++ b/data_validation/validators/schema_validator.py @@ -1,24 +1,120 @@ """ -Validates contributed files against dataset-specific JSON schemas. +Validates data files against their JSON Schema. Also covers completeness: +required fields and allowed values are what the schemas define. + +The schema for a file is found in this order: + 1. the file's own "$schema" key, when it is a relative path + (entries/*.json use "../schema.json"; the Turkish test cases point at + data_validation/schemas/promptinj_testcase.schema.json) + 2. the nearest schema.json in the file's folder or a parent folder, + up to the dataset directory + 3. data_validation/schemas/ for datasets that have no schema.json of + their own (incident, rag, crossframework_mapping); the rag and + incident entries also name it in "$schema" +A file with no schema found is reported as a warning. Usage: - python schema_validator.py --file ../../datasets/incident_dataset/INC-0001.json - python schema_validator.py --directory ../../datasets/incident_dataset/ --schema ../schemas/incident.schema.json + python schema_validator.py --file ../../datasets/rag_dataset/entries/RAG-0001.json + python schema_validator.py --dataset ../../datasets/rag_dataset/ + python schema_validator.py --file entry.json --schema ../schemas/rag.schema.json """ -import argparse, json, sys, os +from __future__ import annotations + +import argparse +import json +import sys +from functools import lru_cache +from pathlib import Path +from urllib.parse import unquote, urlparse +from urllib.request import url2pathname + +if __package__ in (None, ""): + sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from validators._common import ( # noqa: E402 + DATASETS_ROOT, ERROR, SCHEMAS_DIR, WARN, Finding, add_target_args, + dataset_root_of, ensure_utf8_stdout, load_json, print_findings, resolve_targets, +) + +CHECK = "schema" + +# Datasets whose only schema lives in data_validation/schemas/. +FALLBACK_SCHEMAS = { + "incident_dataset": "incident.schema.json", + "rag_dataset": "rag.schema.json", + "crossframework_mapping_dataset": "crossframework_mapping.schema.json", +} + + +def find_schema(path: Path, data, dataset_root: Path | None = None) -> Path | None: + """The schema file for a data file, or None (see the module docstring).""" + ref = data.get("$schema") if isinstance(data, dict) else None + if isinstance(ref, str) and not ref.startswith(("http://", "https://")): + return (path.parent / ref).resolve() + root = (dataset_root or dataset_root_of(path)).resolve() + folder = path.resolve().parent + while True: + if (folder / "schema.json").is_file(): + return folder / "schema.json" + if folder == root or folder.parent == folder: + break + folder = folder.parent + fallback = FALLBACK_SCHEMAS.get(root.name) + return SCHEMAS_DIR / fallback if fallback else None + + +def _retrieve(uri: str): + from referencing import Resource + from referencing.jsonschema import DRAFT202012 + parsed = urlparse(uri) + if parsed.scheme != "file": + raise LookupError(f"only local schema references are resolved: {uri}") + contents = json.loads(Path(url2pathname(unquote(parsed.path))).read_text(encoding="utf-8")) + return Resource.from_contents(contents, default_specification=DRAFT202012) + + +@lru_cache(maxsize=None) +def _validator(schema_path: Path): + from jsonschema.validators import validator_for + from referencing import Registry + schema = json.loads(schema_path.read_text(encoding="utf-8")) + # An $id lets relative "$ref"s in the schema resolve against its own location. + schema = {**schema, "$id": schema_path.as_uri()} + cls = validator_for(schema) + return cls(schema, registry=Registry(retrieve=_retrieve), format_checker=cls.FORMAT_CHECKER) + + +def validate_file(path: Path, schema_path: Path | None = None, dataset_root: Path | None = None) -> list[Finding]: + """Validate one file. Returns findings (empty when it passes).""" + data, err = load_json(path, CHECK) + if err: + return [err] + schema_path = schema_path or find_schema(path, data, dataset_root) + if schema_path is None: + return [Finding(WARN, CHECK, path, "", "no schema found for this file, so it was not schema-checked")] + if not schema_path.is_file(): + return [Finding(ERROR, CHECK, path, "/$schema", f"schema not found: {schema_path}")] + findings = [] + for e in _validator(schema_path.resolve()).iter_errors(data): + loc = "/" + "/".join(str(p) for p in e.absolute_path) if e.absolute_path else "" + findings.append(Finding(ERROR, CHECK, path, loc, e.message)) + return findings + -def validate_file(filepath, schema): - """Validate a single JSON file against a schema. Returns (passed: bool, errors: list).""" - # TODO: Implement using jsonschema library - pass +def main(argv: list[str] | None = None) -> int: + ensure_utf8_stdout() + parser = argparse.ArgumentParser(description="Validate data files against their JSON Schema") + add_target_args(parser) + parser.add_argument("--schema", help="Use this schema instead of finding one per file") + args = parser.parse_args(argv) + files = resolve_targets(args) + if files is None: + return 2 + schema = Path(args.schema).resolve() if args.schema else None + findings = [f for p in files for f in validate_file(p, schema)] + print(f"Checked {len(files)} file(s)") + return print_findings(findings, DATASETS_ROOT.parent) -def main(): - parser = argparse.ArgumentParser(description="Validate dataset files against JSON schemas") - parser.add_argument("--file", help="Path to a single file to validate") - parser.add_argument("--directory", help="Path to a dataset directory to validate all files") - parser.add_argument("--schema", help="Path to schema file (auto-detected if not provided)") - args = parser.parse_args() - print("Schema validator stub - contributions welcome! See data_validation/README.md") if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/datasets/_shared/dsgai_taxonomy.json b/datasets/_shared/dsgai_taxonomy.json index 27b4089..a73aebf 100644 --- a/datasets/_shared/dsgai_taxonomy.json +++ b/datasets/_shared/dsgai_taxonomy.json @@ -37,7 +37,7 @@ }, { "id": "DSGAI07", - "name": "Data Governance, Lifecycle & Classification for AI", + "name": "Data Governance, Lifecycle & Classification for AI Systems", "summary": "Gaps in lifecycle management, classification, provenance, and traceability of AI-related data, treated as enabling conditions for every other DSGAI risk." }, { @@ -62,7 +62,7 @@ }, { "id": "DSGAI12", - "name": "Unsafe Natural-Language Data Gateways", + "name": "Unsafe Natural-Language Data Gateways (LLM-to-SQL/Graph)", "summary": "Natural-language interfaces that translate user prompts into database queries (SQL, GraphQL, NoSQL, vector search) generating unsafe or unscoped data access." }, {