feat: consolidate Hermes context optimization and tokenizer telemetry

[SecretSettler]

Summary

This PR consolidates the complete stacked work from #52 through #58 into one reviewable branch targeting main, then adds the tokenizer-only telemetry/reporting corrections currently validated in the Hermes runtime.

Consolidated stack

• modular Hermes context-opportunity analyzer
• processed-payload telemetry and prompt duplicate shadow
• prompt-dedup A/B simulation harness
• default-off skill prompt dedup canary
• trace-derived validation gate
• provenance artifact reuse canary
• lightweight provenance profiler and synthetic provenance-linking evaluation

Token accounting correction

• removes runtime/background chars / 4 token-savings accounting
• reports token savings only from records with actual_token_status=available
• records actual_tokens_before, actual_tokens_after, and actual_tokens_saved from the configured tokenizer
• defaults the optional tokenizer backend to tiktoken:cl100k_base when available; otherwise reports unavailable
• keeps character savings as character metadata only
• makes gateway-log fallback explicitly char-only
• adds monitor coverage for the new char-only log format

Scope

This PR supersedes the code stacks in #52, #53, #54, #55, #56, #57, and #58. Those PRs should remain open until this consolidated PR passes GitHub CI and review.

Intentionally excluded:

• feat(plugins): introduce middleware token proxy plugin suite (reorder, dedup, lookup, skill index) #50 (feature/msc-cache-optimization), which is independent and currently conflicted
• local virtual environments
• PDF/report-generation and drawing scripts
• one-off token-measurement scripts
• speculative_patches/

Verification

Local verification on the consolidated worktree:

• python3 -m pytest tests/ -v --tb=short -m "not slow and not gpu and not integration"
  • 593 passed, 2 skipped, 40 deselected
• focused stack/runtime suite
  • 140 passed
• char-only monitor regression
  • 6 passed
• python3 -m compileall -q __init__.py contextpilot scripts tests
  • passed
• git diff --check
  • passed
• merge-tree against current origin/main
  • clean, zero conflicts
• runtime scan for estimated_chars_div_4 / chars_saved // 4 accounting
  • no forbidden accounting path found

Review notes

• tiktoken:cl100k_base is explicitly recorded as the tokenizer backend. These are tokenizer-measured counts, not provider API usage counts for every possible model family.
• GitHub's clean Python 3.12 CI remains authoritative because the local verifier used Python 3.11 and emitted non-fatal warnings for unavailable pytest-asyncio / pytest-timeout config plugins.
• The repository-wide Black check currently reports pre-existing formatting drift and is configured continue-on-error; this PR does not perform an unrelated whole-repository reformat.