Group benchmark matrices by database

Keep nine independent database matrices, sequential isolated-cell steps, complete aggregation and authenticated historical result names. Static actionlint, YAML/matrix inventory, JavaScript syntax and governance checks passed. Tests, benchmarks and workflow dispatch were not run at the owner request.

Author: KSemenenko

.github/workflows/AGENTS.md

@@ -63,3 +63,6 @@
 
 ## Independent website queue, 2026-10-04
 - CI website generation MUST NOT wait for unrelated ordinary CI/RF3 execution from another run. Push/manual CI runs use their own run identity; PR keeps its existing ref-based cancellation. Website qualification and deployment use separate bounded job concurrency groups with cancel-in-progress=false, preserving source/latest-evidence freshness and all gates. This implements the owner-authorized independent website action without canceling database or test work.
+
+## Separate database matrices, 2026-10-04
+- Owner correction requires one named job matrix per database in Benchmarks. Each target has its own three checks and thirty canonical node/scenario workloads with readable database/node/scenario names. All nine groups depend only on actual plan/image inputs and run independently without a max-parallel cap; steps within each isolated cell remain sequential. Aggregate joins every group and preserves authenticated failed/null results. This supersedes the shared preflight/CRUD/specialized grouping above without changing native topology, cell isolation, required suites or immutable evidence.

.github/workflows/benchmarks.yml

@@ -40,9 +40,7 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     outputs:
-      crud: ${{ steps.plan.outputs.crud_matrix }}
-      specialized: ${{ steps.plan.outputs.specialized_matrix }}
-      preflight: ${{ steps.plan.outputs.preflight_matrix }}
+      databases: ${{ steps.plan.outputs.database_matrices }}
     steps:
       - name: Download source code
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
@@ -184,24 +182,24 @@ jobs:
             artifacts/code-quality/**
           if-no-files-found: error
           retention-days: 90
-  comparison-preflight:
-    name: Check database / ${{ matrix.id }}
+  comparison-keyload:
+    name: KeyLoad / ${{ matrix.label }}
     needs: [comparison-plan, comparison-images]
     runs-on: ubuntu-latest
     timeout-minutes: 150
     permissions: {contents: read, actions: read}
     strategy:
       fail-fast: false
-      matrix: ${{ fromJSON(needs.comparison-plan.outputs.preflight) }}
-    env:
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).keyload }}
+    env: &database-environment
       GH_TOKEN: ${{ github.token }}
       KEYLOAD_COMPARISON_CELL_ID: ${{ matrix.id }}
-      KEYLOAD_COMPARISON_JOB_NAME: Check database / ${{ matrix.id }}
+      KEYLOAD_COMPARISON_JOB_NAME: ${{ matrix.jobName }}
       Benchmarks__Target: ${{ matrix.target }}
       Benchmarks__NodeCount: ${{ matrix.nodeCount }}
       Benchmarks__Scenario: ${{ matrix.scenario }}
       Benchmarks__EvidenceProfile: ${{ matrix.profile }}
-    steps:
+    steps: &database-steps
       - name: Download source code
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
       - name: Prepare database containers
@@ -235,7 +233,7 @@ jobs:
         if: ${{ !cancelled() }}
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
         with:
-          name: comparison-preflight-${{ matrix.id }}
+          name: ${{ matrix.artifactPrefix }}${{ matrix.id }}
           path: artifacts/comparisons/isolated/workers/${{ matrix.id }}/*
           if-no-files-found: error
           retention-days: 90
@@ -244,132 +242,98 @@ jobs:
         uses: ./.github/workflows/Features/BenchmarkComparisons/IsolatedCellTeardown
         with:
           setup-outcome: ${{ steps.images.outcome }}
-          artifact-name: comparison-preflight-qualification-${{ matrix.id }}
-  comparison-crud:
-    name: Benchmark / ${{ matrix.id }}
+          artifact-name: ${{ matrix.qualificationPrefix }}${{ matrix.id }}
+  comparison-postgresql:
+    name: PostgreSQL + pgvector / ${{ matrix.label }}
     needs: [comparison-plan, comparison-images]
     runs-on: ubuntu-latest
     timeout-minutes: 150
     permissions: {contents: read, actions: read}
     strategy:
       fail-fast: false
-      matrix: ${{ fromJSON(needs.comparison-plan.outputs.crud) }}
-    env:
-      GH_TOKEN: ${{ github.token }}
-      KEYLOAD_COMPARISON_CELL_ID: ${{ matrix.id }}
-      KEYLOAD_COMPARISON_JOB_NAME: Benchmark / ${{ matrix.id }}
-      Benchmarks__Target: ${{ matrix.target }}
-      Benchmarks__NodeCount: ${{ matrix.nodeCount }}
-      Benchmarks__Scenario: ${{ matrix.scenario }}
-      Benchmarks__EvidenceProfile: ${{ matrix.profile }}
-    steps:
-      - name: Download source code
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
-      - name: Prepare database containers
-        id: images
-        uses: ./.github/workflows/Features/BenchmarkComparisons/IsolatedCellSetup
-        with:
-          bundle-artifact-id: ${{ needs.comparison-images.outputs.bundle-artifact-id }}
-      - name: Restore .NET packages
-        run: dotnet restore KeyLoad.slnx
-      - name: Build benchmark tests
-        id: benchmark-build
-        run: dotnet build tests/KeyLoad.ComparisonTests --no-restore --configuration Release
-      - name: Run database workload
-        id: workload
-        if: ${{ !cancelled() }}
-        env:
-          PREPARATION_OUTCOME: ${{ steps.images.outcome }}
-          BUILD_OUTCOME: ${{ steps.benchmark-build.outcome }}
-        run: |
-          if [ "$PREPARATION_OUTCOME" != success ] || [ "$BUILD_OUTCOME" != success ]; then
-            echo '::error::Benchmark preparation failed; no measurement data is available.'
-            exit 1
-          fi
-          dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedNativeComparisonTests/*' --KeyLoadTests:TimeoutMinutes=140
-      - name: Record benchmark availability
-        if: ${{ !cancelled() }}
-        env:
-          KEYLOAD_WORKLOAD_OUTCOME: ${{ steps.workload.outcome }}
-        run: node scripts/Features/BenchmarkComparisons/finalize-worker.mjs
-      - name: Save benchmark results
-        if: ${{ !cancelled() }}
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
-        with:
-          name: comparison-worker-${{ matrix.id }}
-          path: artifacts/comparisons/isolated/workers/${{ matrix.id }}/*
-          if-no-files-found: error
-          retention-days: 90
-      - name: Clean up containers and save logs
-        if: always()
-        uses: ./.github/workflows/Features/BenchmarkComparisons/IsolatedCellTeardown
-        with:
-          setup-outcome: ${{ steps.images.outcome }}
-          artifact-name: comparison-case-qualification-${{ matrix.id }}
-  comparison-specialized:
-    name: Benchmark / ${{ matrix.id }}
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).postgresql }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-qdrant:
+    name: Qdrant / ${{ matrix.label }}
     needs: [comparison-plan, comparison-images]
     runs-on: ubuntu-latest
     timeout-minutes: 150
     permissions: {contents: read, actions: read}
     strategy:
       fail-fast: false
-      matrix: ${{ fromJSON(needs.comparison-plan.outputs.specialized) }}
-    env:
-      GH_TOKEN: ${{ github.token }}
-      KEYLOAD_COMPARISON_CELL_ID: ${{ matrix.id }}
-      KEYLOAD_COMPARISON_JOB_NAME: Benchmark / ${{ matrix.id }}
-      Benchmarks__Target: ${{ matrix.target }}
-      Benchmarks__NodeCount: ${{ matrix.nodeCount }}
-      Benchmarks__Scenario: ${{ matrix.scenario }}
-      Benchmarks__EvidenceProfile: ${{ matrix.profile }}
-    steps:
-      - name: Download source code
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
-      - name: Prepare database containers
-        id: images
-        uses: ./.github/workflows/Features/BenchmarkComparisons/IsolatedCellSetup
-        with:
-          bundle-artifact-id: ${{ needs.comparison-images.outputs.bundle-artifact-id }}
-      - name: Restore .NET packages
-        run: dotnet restore KeyLoad.slnx
-      - name: Build benchmark tests
-        id: benchmark-build
-        run: dotnet build tests/KeyLoad.ComparisonTests --no-restore --configuration Release
-      - name: Run database workload
-        id: workload
-        if: ${{ !cancelled() }}
-        env:
-          PREPARATION_OUTCOME: ${{ steps.images.outcome }}
-          BUILD_OUTCOME: ${{ steps.benchmark-build.outcome }}
-        run: |
-          if [ "$PREPARATION_OUTCOME" != success ] || [ "$BUILD_OUTCOME" != success ]; then
-            echo '::error::Benchmark preparation failed; no measurement data is available.'
-            exit 1
-          fi
-          dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedNativeComparisonTests/*' --KeyLoadTests:TimeoutMinutes=140
-      - name: Record benchmark availability
-        if: ${{ !cancelled() }}
-        env:
-          KEYLOAD_WORKLOAD_OUTCOME: ${{ steps.workload.outcome }}
-        run: node scripts/Features/BenchmarkComparisons/finalize-worker.mjs
-      - name: Save benchmark results
-        if: ${{ !cancelled() }}
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
-        with:
-          name: comparison-worker-${{ matrix.id }}
-          path: artifacts/comparisons/isolated/workers/${{ matrix.id }}/*
-          if-no-files-found: error
-          retention-days: 90
-      - name: Clean up containers and save logs
-        if: always()
-        uses: ./.github/workflows/Features/BenchmarkComparisons/IsolatedCellTeardown
-        with:
-          setup-outcome: ${{ steps.images.outcome }}
-          artifact-name: comparison-case-qualification-${{ matrix.id }}
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).qdrant }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-rabbitmq:
+    name: RabbitMQ / ${{ matrix.label }}
+    needs: [comparison-plan, comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    permissions: {contents: read, actions: read}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).rabbitmq }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-redis:
+    name: Redis / ${{ matrix.label }}
+    needs: [comparison-plan, comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    permissions: {contents: read, actions: read}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).redis }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-neo4j:
+    name: Neo4j / ${{ matrix.label }}
+    needs: [comparison-plan, comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    permissions: {contents: read, actions: read}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).neo4j }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-mongodb:
+    name: MongoDB / ${{ matrix.label }}
+    needs: [comparison-plan, comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    permissions: {contents: read, actions: read}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).mongodb }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-opensearch:
+    name: OpenSearch / ${{ matrix.label }}
+    needs: [comparison-plan, comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    permissions: {contents: read, actions: read}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).opensearch }}
+    env: *database-environment
+    steps: *database-steps
+  comparison-kurrentdb:
+    name: KurrentDB / ${{ matrix.label }}
+    needs: [comparison-plan, comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    permissions: {contents: read, actions: read}
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).kurrentdb }}
+    env: *database-environment
+    steps: *database-steps
   comparison-aggregate:
     name: Combine benchmark results
-    needs: [comparison-build, comparison-plan, comparison-images, comparison-preflight, comparison-crud, comparison-specialized]
+    needs: [comparison-build, comparison-plan, comparison-images, comparison-keyload, comparison-postgresql, comparison-qdrant, comparison-rabbitmq, comparison-redis, comparison-neo4j, comparison-mongodb, comparison-opensearch, comparison-kurrentdb]
     if: ${{ always() && !cancelled() && needs.comparison-plan.result == 'success' && needs.comparison-images.result == 'success' }}
     runs-on: ubuntu-latest
     timeout-minutes: 150

AGENTS.md

@@ -43,6 +43,7 @@ ManagedCode packages are our projects. Fix dependency defects in their owning si
 - Performance qualification MUST cover actual datasets of 100,000, 1,000,000 and 5,000,000 records, with at least 100,000 measured operations per applicable workload cell, sequential and deterministic random reads, ordered/range search, indexing and complex queries. Record dataset size separately from BDN iteration/sample counts, validate actual loaded records and caller-visible results, and retain equal topology, durability, resource and workload contracts across supported competitors. Tiny hot-key fixtures remain explicitly labelled microbenchmark controls and MUST NOT substitute for this scale evidence or justify selecting a performance winner. Repeating more reads over the same tiny corpus does not establish performance at the required record counts. When the owner requests serious performance qualification, complete the representative long workloads rather than presenting short controls as the result; local experiments remain development-only and website figures require authenticated original GitHub artifacts (owner repeated correction 2026-10-03).
 - Qualification MUST use Linux runners only; the owner explicitly replaced the three-OS qualification matrix on 2026-10-03. Preserve every required suite, scalar portability check, recovery and RF3 gate while removing redundant macOS/Windows execution.
 - Each comparison database MUST run the same complete workload suite in its own isolated GitHub Actions runner/job agent, with only that database's native topology and load generator. KeyLoad RF3 and competitor clusters remain genuine clusters; different databases MUST NOT share a runner, process, containers, volumes or measurement session (owner direction 2026-10-03).
+- Owner correction 2026-10-04 requires a separate named GitHub Actions job group and matrix for each comparison database. Keep each database's checks and workload cells together under its readable database name; do not flatten all databases into shared CRUD/specialized matrix groups. Preserve isolated runners per cell, the complete canonical inventory and one authenticated aggregation after every database group finishes.
 - Each isolated comparison agent MUST retain its own source/run/attempt/target/topology/profile-bound JSON. After all target agents finish, one aggregation job MUST validate completeness, comparable settings and provenance, collect their results and generate the website metrics from those JSON files. Failed, missing, skipped or mixed-cohort measurements MUST NOT refresh published performance evidence (owner direction 2026-10-03).
 - The comparison matrix MUST include actual native one-node, two-node and three-node configurations and intensive read/create/update/delete workloads. Each engine/node-count/scenario measurement MUST run on a separate isolated runner agent; record real membership, acknowledgement, correctness, latency, throughput and resource use. Do not relabel client counts or independent standalone databases as cluster node counts. Unsupported native/community topology remains explicitly unavailable, and benchmark topology changes require a fault/consistency ADR before implementation; the initial production RF3 requirement remains mandatory (owner direction 2026-10-03).
 

docs/ADR/ADR-080-benchmark-failure-isolation.md

@@ -294,3 +294,32 @@ serialization, and adds separate qualification/deployment job groups. Website
 work starts independently while all ordinary suites remain mandatory. Root owns
 this CI policy/YAML/regression join; unchanged source/latest-evidence freshness
 and false cancellation protect publication, without canceling existing tests.
+
+## Database groups, 2026-10-04
+
+REQ/AC-BC-GROUP-001/002 replace the shared preflight/CRUD/specialized display groups
+with nine database matrices derived from the unchanged canonical plan. Each matrix
+has 3 checks and 30 workloads; all 297 isolated Linux jobs retain sequential
+prepare/build/run/finalize/upload/cleanup steps. All database groups fan out from
+plan/images concurrently, and aggregate joins every database group even on workload
+failure. YAML anchors share step definitions while leaving native top-level GitHub
+step conclusions visible. Exactly three workflows remain.
+
+Ordered delivery: root updates requirements/policy; root edits planner, workflow
+and exact name validators; a worker updates disjoint source-contract TUnit
+assertions; root joins and statically checks YAML/syntax/full inventory/diff, then
+commits the scoped stage on current main. The owner explicitly excludes executing
+tests, benchmarks or workflow dispatch for this Actions-only task. This evidence
+exception does not assert runtime qualification. Original authenticated historical
+job names remain accepted as exact cell contracts, preserving already produced JSON.
+No data/dependency/topology migration; rollback restores workflow/planner/name
+contracts together without modifying immutable evidence. Agent coordination is
+limited to static contract review and disjoint test-source updates.
+
+```mermaid
+flowchart LR
+  Inputs[Plan and native images] --> KeyLoad[KeyLoad matrix]
+  Inputs --> Databases[Eight separate competitor matrices]
+  KeyLoad --> Aggregate[Complete authenticated aggregation]
+  Databases --> Aggregate
+```

docs/Features/BenchmarkComparisons.md

@@ -974,3 +974,27 @@ flowchart LR
   Failure --> Aggregate
   Aggregate --> Site[Qualified website]
 ```
+
+## Database job groups, 2026-10-04
+
+REQ-BC-GROUP-001 maps to AC-BC-GROUP-001: Benchmarks exposes nine separately
+named database job matrices. Each contains only its own target, three preflight
+checks and all 30 canonical node/scenario workloads. Every cell retains its own
+Linux runner. Database groups run concurrently with only plan/image dependencies;
+steps inside each cell run in order. No shared cross-database matrix or parallelism
+cap. Aggregate waits for every group and retains failed/null results.
+
+REQ-BC-GROUP-002 maps to AC-BC-GROUP-002: readable database/node/scenario job names
+must agree with authenticated job discovery, finalization, aggregation and website
+receipt validation. Canonical plan schema, cell IDs, workload counts, artifacts,
+measurements and topology remain unchanged. Frozen historical names remain valid
+only as exact cell-name contracts in original authenticated receipts; no result
+bytes are rewritten. ADR-080 owns the implementation contract.
+
+TASK-BC-GROUP-001: root owns workflow, planner/name validators and documentation;
+worker owns disjoint source-contract TUnit assertions. Update mapped
+WorkflowLayout/WorkflowBenchmarkFailure/NativeSerializationWorkflow/IsolatedPlanCli
+assertions before joining. Owner explicitly requests no tests or benchmarks for
+this Actions layout task: verify YAML, JavaScript syntax, matrix inventory and diff
+statically; do not dispatch workflows. Runtime/native qualification is unverified.
+Backend/API/storage/transport and database tests are outside this change.