[nuclearcat]

Files that fail JSON decoding or schema validation were only logged and moved to the failed directory, so Grafana could not show them. Add the kcidb_ingestion_failures counter, labelled with reason=json_decode, schema_validation or other.

Increment the counter only after a successful move to failed/. Failed moves leave the file in the spool without counting it again on each retry. Deferred files with log excerpt upload errors remain excluded.

Make origin extraction tolerate malformed submission structures so inputs such as [1] and {"checkouts":["a"]} reach failure handling instead of crashing the worker while logging the error.

Validation: all 61 tests in the ingester test file pass, along with Ruff lint and formatting checks. Regression tests exercise malformed data, successful moves, repeated failed moves, and deferred files.

To reproduce from backend/:

poetry run pytest kernelCI_app/tests/unitTests/commands/monitorSubmissions/kcidbng_ingester_test.py -o addopts='' -p no:rerunfailures -q
poetry run ruff check kernelCI_app/management/commands/helpers/kcidbng_ingester.py kernelCI_app/tests/unitTests/commands/monitorSubmissions/kcidbng_ingester_test.py
poetry run ruff format --check kernelCI_app/management/commands/helpers/kcidbng_ingester.py kernelCI_app/tests/unitTests/commands/monitorSubmissions/kcidbng_ingester_test.py

The test command runs serially with automatic reruns disabled and does not require a running database.

Closes #1505

[felipebergamin]

When the JSON parses but has the wrong shape, prepare_file_data raises inside the except block:

• [1]: standardize_tree_names raises AttributeError, then _extract_origins_info(data) raises again inside the handler.
• {"checkouts": ["a"]}: same result.

The exception leaves prepare_file_data. process_batch has no try around it, so the worker exits. The file stays in the spool and kcidb_ingestion_failures is not incremented.

It seems this bug predates this PR. But would be good to fix it, we can do it separately though

[alanpeixinho]

Agreed. This may come from #1790, which added _extract_origins_info to this except.

We could also benefit from moving the increment to after the file is moved to failed/. A file that stays in the spool is counted again on the next pass, so counting after the move counts each file once.

[nuclearcat]

Fixed, Now will handle malformed submissions without crashing the worker, and increment the failure counter only after successfully moving the file to failed/.