docs: milestone 8 phase 5 — architecture docs, benchmark suite redesign, UseBogus() perf fix by ncipollina · Pull Request #53 · LayeredCraft/compono

ncipollina · 2026-08-06T03:02:37Z

Summary

Milestone 8 Phase 5, in full:

Docs consolidation: absorbs docs/architecture.md, docs/performance.md, docs/design-principles.md, docs/manifesto.md into the architecture/ hierarchy, tombstones all five pre-existing legacy pages (including docs/public-api.md) so existing ADR links stay resolvable, and writes real roadmap/ content.
ADR-0034: a from-scratch benchmark suite strategy replacing the milestone-by-milestone accretion in benchmarks/Compono.Benchmarks — audience-driven categories, a reused representative-model set, fair-comparison rules, and reporting rules. The suite was fully implemented against it (six categories → later five, see below), run for real, and architecture/current/performance.md rewritten from the results.
Implementation Strategies removed: implemented, then removed via an ADR-0034 Amendment — it compared different systems doing different amounts of work (bare construction vs. Compono's full resolution pipeline), so a result from it couldn't be attributed to a cause or guide an optimization decision.
A real perf bug found and fixed: the redesigned suite surfaced UseBogus() costing ~865x a plain member rule. Root-caused to BogusMemberNameProvider constructing a new Bogus.Faker per resolution, and fixed via ADR-0027's Amendment (one Faker cached per thread, reseeded per request) — cut to ~6.3x isolated, and the full-profile cost from 903.4 μs / 2,229 KB to 5.5 μs / 7.0 KB.

Changes

Documentation

docs/architecture/index.md, design-principles.md, current/{source-generation,generated-plans-and-discovery,provider-pipeline,deterministic-seeding,performance}.md, decision-log.md — real content
docs/roadmap/{index,proposed-adrs,future-packages}.md — real content
docs/{architecture,manifesto,design-principles,performance,public-api}.md — tombstoned (not deleted; 24+ Accepted ADRs link to them by path)
docs/adr/0034-benchmark-suite-strategy-and-redesign.md — new, with a same-day Amendment removing Implementation Strategies
docs/adr/0027-compono-bogus-package-design.md — new Amendment documenting the Faker caching fix and why it differs from the shared-Faker<T> alternative that ADR already rejected
mkdocs.yml — five "(legacy)" nav entries removed

Benchmark suite (benchmarks/Compono.Benchmarks/)

Models/, Baselines/, ConsumerScenarios/, ExternalComparison/, FeatureOverhead/, Scalability/, SourceGeneration/ — replaces all 8 pre-redesign files
Compono.Benchmarks.csproj — adds Compono.NSubstitute/Compono.Bogus references, a real (non-analyzer-only) Compono.Generators reference, conditional Basic.Reference.Assemblies

Bug fix (src/Compono.Bogus/)

BogusMemberNameProvider caches one Faker per thread (ThreadLocal<Faker>, trackAllValues: false) instead of constructing one per request
BogusOptions.AddConvention's XML doc updated for the new reused-Faker contract
New tests: 200-way concurrency test, convention-throws-mid-generate test

Validation

dotnet build Compono.slnx -c Release — 0 warnings, 0 errors
dotnet test — Compono.Bogus.Tests (62/62), Compono.Generators.Tests (84/84), run 5x for the new concurrency test's stability
uv run mkdocs build --clean --strict — clean, no warnings
Full benchmark suite run (--filter '*', DefaultJob, ~15 min) — every category produced real results; architecture/current/performance.md's tables all come from one such run

🤖 Generated with Claude Code

…ark strategy Milestone 8 Phase 5: absorb docs/architecture.md, docs/performance.md, docs/design-principles.md, and docs/manifesto.md's real content into the architecture/ hierarchy (design-principles, current/*, decision-log), tombstone all five pre-existing legacy pages (public-api.md included) so existing ADR links stay resolvable, write real roadmap/ content, and retire the "(legacy)" mkdocs.yml nav entries. Also adds ADR-0034: a full redesign of the benchmark suite's strategy, replacing the milestone-by-milestone accretion in benchmarks/Compono.Benchmarks with an audience-driven category taxonomy (Implementation Strategies, Consumer Scenarios, External Comparison, Feature Overhead, Scalability, Source Generation), a reused representative-model set, fair-comparison rules (handwritten construction as the theoretical ceiling, cached vs. uncached reflection as separate honest baselines), and explicit reporting rules - implementation of the suite itself is tracked as PLAN-0008 Phase 5 Part B, not yet done. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…trategies, fix UseBogus() overhead Implements ADR-0034's benchmark redesign: Models/, Baselines/, and five category folders (ConsumerScenarios, ExternalComparison, FeatureOverhead, Scalability, SourceGeneration), replacing all 8 pre-redesign benchmark files. The Implementation Strategies category was implemented and then removed via an ADR-0034 Amendment - it compared different systems doing different amounts of work (bare construction vs. a full resolution pipeline), so a result from it couldn't be attributed to a cause or guide an optimization decision. Running the redesigned suite surfaced a real finding: UseBogus() cost ~865x a plain member rule, because BogusMemberNameProvider constructed a new Bogus.Faker on every resolution. Fixed via ADR-0027's Amendment: one Faker cached per thread (ThreadLocal<Faker>, trackAllValues: false), reseeded per request - safe under concurrent access because a thread-local instance is never touched by more than one thread, distinct from the shared-Faker<T> alternative ADR-0027 already considered and rejected. New coverage: a 200-way concurrency test and a convention-throws-mid-generate test. The fix cut UseBogus()'s isolated overhead from ~865x to ~6.3x and its full-profile cost from 903.4 us / 2,229 KB to 5.5 us / 7.0 KB. architecture/current/performance.md is rewritten from one full suite run (DefaultJob, all 5 categories) with a clarity pass: explicit AutoFixture-relative-to-Compono ratio direction, precise DefaultJob methodology wording, an explicit "equivalent work" definition, a note connecting the Consumer Scenario and Feature Overhead UseBogus() numbers' different scope, and consistent x formatting throughout. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

chatgpt-codex-connector · 2026-08-06T03:02:42Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

…ed doc comment The reused-Faker contract added to AddConvention's XML doc comment changed the generated API reference; the drift-detection CI gate caught it. Regenerated via .github/scripts/generate-api-reference.sh. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Copilot

Pull request overview

Milestone 8 Phase 5 work: consolidates and retires legacy documentation into the new docs/architecture/ + docs/roadmap/ hierarchy, redesigns the benchmark suite per ADR-0034, and fixes a Compono.Bogus performance regression by caching a Bogus.Faker per thread (with new regression tests and updated docs).

Changes:

Consolidate architecture/roadmap documentation, tombstone legacy pages to preserve existing ADR links, and update mkdocs navigation accordingly.
Replace the legacy benchmark suite with a categorized, model-driven suite (Consumer Scenarios, External Comparison, Feature Overhead, Scalability, Source Generation) and document results in architecture/current/performance.md.
Optimize BogusMemberNameProvider by reusing a per-thread Faker and add concurrency/exception-safety regression coverage.

Reviewed changes

Copilot reviewed 63 out of 63 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/Compono.Bogus.Tests/DeterminismTests.cs	Adds concurrency regression test validating deterministic behavior with a shared provider instance.
test/Compono.Bogus.Tests/BogusMemberNameProviderTests.cs	Adds regression test ensuring exceptions in conventions don’t corrupt subsequent requests on the same thread.
src/Compono.Generators/Compono.Generators.csproj	Grants `InternalsVisibleTo` for benchmarks to directly construct the incremental generator.
src/Compono.Bogus/BogusOptions.cs	Updates XML docs to reflect per-thread `Faker` reuse contract for conventions.
src/Compono.Bogus/BogusMemberNameProvider.cs	Implements per-thread `Faker` caching and reseeding per request to address `UseBogus()` performance.
mkdocs.yml	Removes legacy nav entries after documentation consolidation.
docs/roadmap/proposed-adrs.md	Replaces placeholder with current-state content for proposed ADRs.
docs/roadmap/index.md	Replaces placeholder with roadmap framing and links to relevant indexes.
docs/roadmap/future-packages.md	Replaces placeholder with future-package candidates framing and ADR-backed rationale.
docs/reference/api/Compono.Bogus/Compono.BogusOptions.md	Updates generated API reference summary text for `AddConvention` to remove “request-local” wording.
docs/reference/api/Compono.Bogus/Compono.BogusOptions.AddConvention(string,System.Func_Bogus.Faker,string_).md	Updates generated API reference to document per-thread `Faker` reuse expectations.
docs/public-api.md	Tombstones legacy public-api page and points readers to new canonical docs locations.
docs/plans/0008-milestone-8-public-preview.md	Marks Phase 5 done and records benchmark redesign + perf fix work and verification notes.
docs/plans/0002-milestone-2-core-composition-engine.md	Updates legacy performance doc links to the new canonical performance page.
docs/performance.md	Tombstones legacy performance page and points to `architecture/current/performance.md`.
docs/packages/compono.md	Updates link target from legacy public-api page to new design principles page.
docs/manifesto.md	Tombstones legacy manifesto page and points to `architecture/design-principles.md`.
docs/documentation-architecture.md	Updates documentation-architecture status and notes reflecting consolidated docs and retired legacy pages.
docs/design-principles.md	Tombstones legacy design-principles page and points to `architecture/design-principles.md`.
docs/best-practices/performance-recommendations.md	Updates links to the new canonical performance documentation.
docs/architecture/index.md	Adds real Architecture landing content and links into the new hierarchy.
docs/architecture/design-principles.md	Populates design principles with consolidated manifesto/design-principles content.
docs/architecture/decision-log.md	Adds public-facing ADR index table including ADR-0034.
docs/architecture/current/source-generation.md	Populates source generation architecture documentation (generator responsibilities + reflection policy discussion).
docs/architecture/current/provider-pipeline.md	Populates provider pipeline documentation, including stage ordering and diagnostics notes.
docs/architecture/current/performance.md	Rewrites performance documentation to match ADR-0034 suite and publish results/methodology.
docs/architecture/current/generated-plans-and-discovery.md	Documents plan discovery/dispatch and records open questions around cache collisions/ALC rooting.
docs/architecture/current/deterministic-seeding.md	Documents deterministic seeding internals and `DeriveSeed()` contract.
docs/architecture.md	Tombstones legacy architecture page and points to the new architecture section entry points.
docs/adr/README.md	Adds ADR-0034 to the ADR index.
docs/adr/0034-benchmark-suite-strategy-and-redesign.md	Introduces ADR-0034 defining the redesigned benchmark suite structure and rules (with same-day amendment removing Implementation Strategies).
docs/adr/0027-compono-bogus-package-design.md	Adds amendment documenting the per-thread `Faker` caching fix and rationale.
benchmarks/Compono.Benchmarks/SourceGeneration/GeneratorDriverBenchmarks.cs	Adds in-process generator driver benchmarks (clean vs incremental) per ADR-0034.
benchmarks/Compono.Benchmarks/Scalability/GraphDepthScalingBenchmarks.cs	Adds shallow-vs-deep graph benchmark to catch depth-related scalability issues.
benchmarks/Compono.Benchmarks/Scalability/CollectionSizeScalingBenchmarks.cs	Adds collection-size scaling benchmark to catch regressions in collection dispatch.
benchmarks/Compono.Benchmarks/Scalability/BatchScalingBenchmarks.cs	Adds `CreateMany` scaling benchmark across a larger batch-size matrix.
benchmarks/Compono.Benchmarks/FeatureOverhead/SharingOverheadBenchmarks.cs	Adds benchmark isolating sharing mechanism overhead (`CreateRow`/`ResolveShared`).
benchmarks/Compono.Benchmarks/FeatureOverhead/NSubstituteOverheadBenchmarks.cs	Adds benchmark isolating `UseNSubstitute()` overhead vs an exact registration baseline.
benchmarks/Compono.Benchmarks/FeatureOverhead/ConfigurationOverheadBenchmarks.cs	Adds benchmark isolating incremental cost of member/type rules and a custom semantic provider.
benchmarks/Compono.Benchmarks/FeatureOverhead/BogusOverheadBenchmarks.cs	Adds benchmark isolating `UseBogus()` overhead vs a member-rule baseline.
benchmarks/Compono.Benchmarks/ExternalComparison/SimplePocoComparisonBenchmarks.cs	Adds AutoFixture comparison for the flat representative model.
benchmarks/Compono.Benchmarks/ExternalComparison/MediumAggregateComparisonBenchmarks.cs	Adds AutoFixture comparison for the moderately nested representative model.
benchmarks/Compono.Benchmarks/ConsumerScenarios/SharedValueBenchmarks.cs	Adds shared-value consumer scenario benchmark using core row APIs.
benchmarks/Compono.Benchmarks/ConsumerScenarios/RepresentativeModelBenchmarks.cs	Adds consumer-scenario benchmarks for the representative model set (including large collection).
benchmarks/Compono.Benchmarks/ConsumerScenarios/ProviderEnabledBenchmarks.cs	Adds consumer-scenario benchmarks for Bogus- and NSubstitute-enabled profiles.
benchmarks/Compono.Benchmarks/Compono.Benchmarks.csproj	Updates benchmark project references and adds conditional `Basic.Reference.Assemblies` packages for generator driver benchmarks.
benchmarks/Compono.Benchmarks/Models/SimplePoco.cs	Adds reused flat representative model replacing legacy `Leaf`.
benchmarks/Compono.Benchmarks/Models/SharedValueGraph.cs	Adds reused shared-value representative model set.
benchmarks/Compono.Benchmarks/Models/ProviderBackedModel.cs	Adds reused provider-eligible model for Bogus/NSubstitute scenarios.
benchmarks/Compono.Benchmarks/Models/MediumAggregate.cs	Adds reused “representative graph” model replacing legacy `Customer`/`Address`.
benchmarks/Compono.Benchmarks/Models/LargeCollection.cs	Adds reused model for collection-size scaling benchmarks.
benchmarks/Compono.Benchmarks/Models/DeepGraph.cs	Adds reused deep-graph model replacing legacy one-off deep graph benchmark types.
benchmarks/Compono.Benchmarks/Baselines/AutoFixtureComposer.cs	Adds AutoFixture baseline wrapper in new `Baselines/` location per ADR-0034.
benchmarks/Compono.Benchmarks/ResolutionEcosystemBenchmarks.cs	Deletes legacy benchmark class replaced by the redesigned suite.
benchmarks/Compono.Benchmarks/ResolutionBenchmarkTypes.cs	Deletes legacy benchmark types replaced by `Models/`.
benchmarks/Compono.Benchmarks/ResolutionBenchmarks.cs	Deletes legacy batch scaling benchmark replaced by `Scalability/BatchScalingBenchmarks`.
benchmarks/Compono.Benchmarks/ResolutionArchitectureBenchmarks.cs	Deletes legacy architecture benchmark replaced by redesigned categories.
benchmarks/Compono.Benchmarks/ReflectionComposer.cs	Deletes legacy reflection baseline removed by ADR-0034 redesign/amendment.
benchmarks/Compono.Benchmarks/EcosystemBenchmarks.cs	Deletes legacy benchmark class replaced by `ExternalComparison/*ComparisonBenchmarks`.
benchmarks/Compono.Benchmarks/DeepGraphBenchmarks.cs	Deletes legacy deep-graph benchmark replaced by `Scalability/GraphDepthScalingBenchmarks`.
benchmarks/Compono.Benchmarks/BenchmarkTypes.cs	Deletes legacy flat representative type replaced by `Models/SimplePoco`.
benchmarks/Compono.Benchmarks/AutoFixtureComposer.cs	Deletes legacy baseline location replaced by `Baselines/AutoFixtureComposer`.
benchmarks/Compono.Benchmarks/ArchitectureBenchmarks.cs	Deletes legacy architecture benchmark class superseded by redesigned suite.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

j-d-ha

🛑 Adversarial review found blocking isolation, benchmark-validity, and documentation-link defects. Inline comments cover fixes; targeted Bogus tests pass, but current PR must resolve blockers before merge.

j-d-ha · 2026-08-06T11:14:26Z

🐛 Legacy-page retirement is incomplete: living docs still link visitors to tombstones (docs/index.md:69,77-79, docs/getting-started/learning-paths.md:55-58, docs/concepts/providers.md:47,56-58). ADR links may retain tombstone targets, but active site navigation/content must use canonical Architecture, Concepts, or Reference pages. Also update home-page copy still citing deleted-suite 6.1× results.

…, doc links Real defects found by adversarial review, all fixed: - BogusMemberNameProvider's thread-local Faker reuse was incomplete: a custom AddConvention delegate could mutate Faker state (DateTimeReference, a sub-generator, any of ~20 other public settable properties) that would leak into a later, unrelated built-in-convention request on the same thread - reseeding Random alone didn't restore isolation. Fixed by reusing the per-thread Faker only for built-in/alias conventions (BogusConventions.IsBuiltIn, reference-equality against the ten built-in delegates); a custom delegate now gets its own single-use Faker, matching its pre-optimization behavior. New regression test mutates DateTimeReference in a custom convention and proves it doesn't perturb a later built-in request. - The concurrency test used Parallel.ForEachAsync over a fully-synchronous body, which could pass serially by scheduler luck rather than proving genuine thread overlap. Replaced with real Thread + Barrier so all workers release simultaneously. - GeneratorDriverBenchmarks' "incremental" tree was built via a fresh ParseText call, not derived from the base tree via WithChangedText, so it measured a wholesale reparse under an incremental label. Fixed to derive it via an append-only WithChangedText edit, so unaffected nodes keep their base-tree identity. - GraphDepthScalingBenchmarks compared MediumAggregate (seven strings + a collection) against DeepGraph (one string), conflating depth with total value-generation work. Fixed to compare DeepLevel8 (depth 1, one string) against DeepGraph (depth 8, same leaf shape) - the isolated result is 4.47x the mean and 2.65x the allocation for 8x the depth. - architecture/current/performance.md violated ADR-0034's own Reporting Rules by omitting Error/StdDev/Gen0/Gen1 from most tables. Rewritten with the full mandatory column set on every table, from one fresh full suite run. - Fixed a broken double-hyphen MkDocs anchor (3 occurrences), a link to a heading removed from the rewritten performance page, and nine living docs still linking to tombstoned legacy pages instead of their canonical replacements (including docs/index.md's stale "~6.1x faster" claim tied to a deleted benchmark class). Reworded future-packages.md's claim that Compono.Generators is a fifth installable package - it's an embedded analyzer, not independently installable. One reviewer claim (returning GeneratorDriver from a benchmark method breaks standard net10 dry runs via missing Compono.Generators restore assets) was investigated directly from a fully clean bin/obj state, across both --job Dry and DefaultJob - did not reproduce, left as-is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

ncipollina · 2026-08-06T12:56:26Z

Re: #53 (comment) — Fixed in 0d69cc1. Swept every real link (not just the 3 files you named) to the five tombstoned pages and repointed each to its canonical replacement:

docs/index.md — removed the stale "~6.1x faster" performance claim (tied to a benchmark class that no longer exists after the ADR-0034 redesign) and repointed the Manifesto/Architecture/Design Principles/Public API/Performance link list to architecture/design-principles.md, architecture/index.md, architecture/current/performance.md.
docs/getting-started/learning-paths.md, docs/concepts/providers.md — the two you named — repointed to architecture/current/provider-pipeline.md.
Also found and fixed six more the same sweep should have caught the first time: docs/concepts/composition-model.md (two links, plus an inline docs/public-api.md citation), docs/concepts/index.md, docs/concepts/determinism-and-seeding.md, docs/concepts/registrations-and-rules.md (two links, one to a specific UseServiceProvider reference page), docs/concepts/collections.md, docs/how-to/register-a-type.md — all repointed to the matching canonical page (architecture/current/*.md, architecture/design-principles.md, or reference/index.md for the generic "precise API contract" pointers).

Verified with a comprehensive grep across all of docs/ (excluding adr//plans/, which correctly keep historical tombstone links) for any remaining link to the five retired filenames — zero hits now.

ncipollina · 2026-08-06T12:56:42Z

Re: #53 (review) — All 11 inline findings from this review addressed in 0d69cc1, replied to individually on each thread. Summary:

Fixed:

BogusMemberNameProvider's custom-convention Faker-mutation isolation gap (the real bug — thread-local reuse was safe for concurrency but not for arbitrary AddConvention delegates mutating shared state)
Concurrency test strengthened with a real Thread+Barrier instead of a Parallel.ForEachAsync body that could pass serially
GeneratorDriverBenchmarks' incremental tree now derived via WithChangedText, not a fresh reparse
GraphDepthScalingBenchmarks now isolates depth as the only variable (DeepLevel8 vs. DeepGraph, both one-string leaf shape) — real result is 4.47x/2.65x, not the old 1.03x/1.23x
performance.md rewritten with the full ADR-0034-mandated column set (Mean/Error/StdDev/Ratio/Allocated/Gen0/Gen1) on every table
Broken anchors and stale numbers in docs/plans/, and a broader sweep of 9 living docs still linking to tombstoned pages (found 6 more beyond the 2 you named)
future-packages.md's Compono.Generators-as-installable-package wording

Investigated, did not reproduce: the GeneratorDriver-return-type dry-run failure claim — details on that thread, happy to dig further with a concrete repro if you have one.

All 63 Compono.Bogus.Tests pass (5 consecutive runs for the new concurrency/mutation tests), full solution builds clean, mkdocs build --strict clean, full benchmark suite reran end to end.

ncipollina and others added 2 commits August 5, 2026 16:35

github-actions Bot added the type: docs Documentation change label Aug 6, 2026

ncipollina changed the title ~~docs(docs): Milestone 8 Phase 5 — architecture docs, benchmark suite redesign, UseBogus() perf fix~~ docs: milestone 8 phase 5 — architecture docs, benchmark suite redesign, UseBogus() perf fix Aug 6, 2026

ncipollina requested review from j-d-ha and a lite review from Copilot August 6, 2026 03:13

Copilot started reviewing on behalf of ncipollina August 6, 2026 03:14 View session

Copilot AI reviewed Aug 6, 2026

View reviewed changes

Comment thread src/Compono.Bogus/BogusMemberNameProvider.cs