docs: add JSON output format specification - #2303
ReturnKartikey wants to merge 1 commit into
Conversation
Add docs/json_format.md with a field-by-field reference for the JSON output produced by --benchmark_format=json. The existing user_guide.md has a brief overview and one abbreviated example, but doesn't document every field -- this makes it hard for tool authors and CI pipeline maintainers to consume the output without reading json_reporter.cc directly. The new doc covers: - Context object fields (CPU, caches, load_avg, custom context, etc.) - Benchmark result fields (timing, iterations, threads, counters) - Aggregate results (mean, median, stddev, cv) - Big-O complexity and RMS fields - Error/skip, memory metrics, labels - Special float handling (NaN/Infinity) - List mode (--benchmark_list_tests) output - A full realistic example with iteration + aggregate results Also adds a cross-reference from user_guide.md and an entry in docs/index.md. Closes google#720
|
i think i've changed my mind about #720. keeping documentation in sync with the code is going to be a real pain. it's not that hard for folks to just read the code, or run the tests, to see the json output. |
Honestly, i was wrestling myself from posting the same yesterday... |
|
Thanks @dmah42 and @LebedevRI for the feedback.... that makes total sense. Manual documentation markdown files inevitably suffer from drift as new metrics and fields are added to If you're open to it, I'd be happy to pivot this PR from a markdown document to a formal machine-readable JSON Schema (e.g.
All the field types and descriptions already compiled here can be embedded directly into the schema's Would you be receptive to that direction? If so, I'll update the PR with the schema and test. |
This PR documents the JSON output format produced by
--benchmark_format=json, as requested in #720.The existing
user_guide.mdhas an abbreviated example but lacked a complete field-by-field reference. This addsdocs/json_format.mddetailing:contextfields (host, CPU, caches, load averages, schema version)benchmarksfields (iterations, time metrics, aggregates, Big-O, counters, labels)--benchmark_list_tests) formatAlso adds a cross-reference link in
docs/user_guide.mdand updatesdocs/index.md.Closes #720