refactor(pchronicle): unify unknown fields and JSON streaming

[reiase]

Summary

• replace StorylinePresence and format-specific residual metadata with one bounded RFC 6901 unknown_fields mechanism
• preserve unknown keys and values across ATIF, ACTF, OpenAI message, AgenticMD, and Storyline Lance conversions while treating known missing/null object fields as semantically equivalent
• reject target-format unknown fields inside the _storyline carry envelope instead of repairing unknown_key_counts after decoding
• add shared bounded JSON streaming plus ATIF/ACTF projected steps decoding and filter pushdown
• add required schema_version: 1 to the atomic Storyline Lance CURRENT pointer and fail closed before opening tables when it is missing or unsupported
• make standalone dlcapt emit authoritative Storyline metadata in AgenticMD frontmatter and blocks
• remove obsolete whole-string parsers, redundant count writebacks, panic-only invariant handling, unreachable format branches, test-only production wrappers, and stale terminology
• update pChronicle capability, storage, projection, ACTF, and test documentation

Review follow-up

• removed the owned_counts writeback and closed the envelope validation gap
• expanded semantic evidence across all six directed ATIF, ACTF, and OpenAI conversion hops, including agent, source, reasoning, metrics, timestamps, model, latency, and tool duration
• versioned the Storyline Lance snapshot pointer while the layout has no production migration cost
• added regression coverage for target-source envelope rejection and missing or unsupported Lance schema versions

JSON streaming benchmark

Local release benchmark on Darwin 25.5.0 arm64, Apple M4, 24 GiB RAM, rustc 1.97.1. Scale 128, 20 iterations, PCHRONICLE_BENCH_PATH=both.

Input	Dataset	Projected median	Full median	Speedup	Projected/full allocated bytes	Reduction
ATIF NDJSON	1,024 trajectories / 15,104 steps / 12.57 MB	15.358 ms	129.332 ms	8.42x	16.06 MB / 460.98 MB	28.70x
ATIF array	1,024 trajectories / 15,104 steps / 12.57 MB	22.223 ms	136.625 ms	6.15x	16.06 MB / 460.98 MB	28.70x
ACTF array	256 trajectories / 512 steps / 0.45 MB	1.598 ms	30.118 ms	18.85x	5.14 MB / 46.23 MB	8.99x

Commands:

PCHRONICLE_BENCH_PATH=both cargo bench -p persisting-pchronicle --bench json_streaming
PCHRONICLE_BENCH_PATH=both PCHRONICLE_BENCH_JSON_SHAPE=array cargo bench -p persisting-pchronicle --bench json_streaming
PCHRONICLE_BENCH_PATH=both PCHRONICLE_BENCH_FORMAT=actf PCHRONICLE_BENCH_JSON_SHAPE=array cargo bench -p persisting-pchronicle --bench json_streaming

Observed process peak RSS was 35.016/113.406 MiB for ATIF NDJSON, 34.609/112.750 MiB for ATIF array, and 30.547/38.312 MiB for ACTF array. RSS is the process peak from a shared projected-then-full invocation; latency and allocation counters are the direct paired measurements.

Validation

• cargo test -p persisting-pchronicle --features lance-store — 311 unit and 69 integration tests passed; 5 opt-in or external-service tests ignored
• cargo test -p persisting-dlcapt --locked — 19 unit and 24 integration tests passed
• cargo clippy -p persisting-pchronicle --lib --locked -- -D warnings -D clippy::unwrap_used -D clippy::expect_used -D clippy::unreachable
• cargo clippy -p persisting-pchronicle --all-targets --features lance-store -- -D warnings
• cargo clippy -p persisting-dlcapt --lib --bin dlcapt --locked -- -D warnings
• cargo fmt --all --check
• just docs-links
• git diff --check