Implement shared native database benchmark methodology

Use one canonical pure, mixed and million-record ingestion schedule at native one and three nodes. Execute real independent 1/10/500-client ingestion with bounded admission, complete stored-state verification, three retained repetitions and explicit timing/resource evidence. Remove active two-node comparison cells and keep historical originals unchanged.

Add an exclusive Aspire RF3 ingestion case with simultaneous SDK and official MCP reads. Authenticate its original job, TRX and image artifacts before qualifying the new document cohort. Keep measurements serial within isolated database jobs and preserve all existing qualification gates.

Verification is pending in GitHub Actions after the owner's GitHub-only correction. Earlier development failures and mapped repairs are recorded honestly; this checkpoint does not claim complete native scale, coverage, endurance or production qualification.

Author: KSemenenko

.github/workflows/AGENTS.md

@@ -92,7 +92,7 @@
 ## Current first-release Website contract, owner corrections 2026-10-06
 
 - The current four workflows are Build and Tests (`build-and-tests.yml`), Benchmarks (`benchmarks.yml`), Website (`website.yml`) and prepared manual Release (`release.yml`). Only Website builds, qualifies and publishes the site on trusted main push/manual; Benchmarks ends with a bounded Website dispatch and Website has no `workflow_run` executor subscription. These explicit owner corrections supersede every earlier three-workflow, filename, executor, placement and completion-subscription clause; every unrelated suite, permission and qualification rule remains mandatory.
-- The sole measured producer contract is the current 1,386-worker/2,530-input cohort in ADR-076. Its executed dependency manifest contains exactly 80 current source paths. Remove old-plan readers, schemas, event-file parsers and their exclusive fixtures or inventory entries under the root owner-only migration/legacy super rule. This supersedes earlier 270/277 and old-reader requirements only; retain bounded authenticated REST producer/artifact provenance, exact current source and input hashes, failed/null accounting, strict selected-evidence rejection, freshness and unchanged 80/70/90 coverage thresholds.
+- The sole measured producer contract is the current 924-worker/1,716-input website cohort plus separately authenticated 418-cell document family in ADR-076. Its executed dependency manifest contains exactly 84 current source paths. Remove old-plan readers, schemas, event-file parsers and their exclusive fixtures or inventory entries under the root owner-only migration/legacy super rule. This supersedes earlier 270/277 and old-reader requirements only; retain bounded authenticated REST producer/artifact provenance, exact current source and input hashes, failed/null accounting, strict selected-evidence rejection, freshness and unchanged 80/70/90 coverage thresholds.
 - When no current authenticated producer is ready, qualify the complete content-only site without figures. Real native TUnit/Node/Chrome operations, no skips in each applicable suite, source/coverage inventories and needs-gated least-privilege Pages remain required. Local tests enter the same Aspire-owned AppHost and are development evidence; genuine exact-source Linux/provider proof closes delivery. Controlled rejection inputs cannot become published measurements.
 
 ## Native TUnit entry, owner correction 2026-10-07
@@ -101,3 +101,7 @@
 ## Website automatic trigger scope, owner correction 2026-10-08
 
 - Website main pushes MUST be filtered to site and actual qualification/build/publication inputs; database-only pushes MUST NOT enqueue it. Benchmarks' final bounded dispatch MUST require all producer prerequisites, database matrices and aggregate to succeed. Website MUST authenticate successful terminal completion of a supplied triggering producer before proceeding. This explicit correction supersedes unfiltered push and failed-producer dispatch clauses only; preserve the existing manual/final-dispatch executor, independent site push path, optional metrics, bounded waits, immutable provenance, all qualification/freshness checks and least privileges.
+
+## Native benchmark methodology, owner direction 2026-10-09
+
+- ADR-122 and Methodology require only actual native1/3 topology, one shared pure/mixed/ingestion inventory and exclusive measurements. Active two-node dispatch/admission is removed; RF3 majority remains2. Preserve authenticated original failure/null and resource/source/provenance gates; new document-family originals are separately admitted before any numerical website rendering. Executed source closure reflects every new transitive input, with unchanged80/70/90 coverage thresholds.

.github/workflows/Features/BenchmarkComparisons/QualifySite/action.yml

@@ -82,7 +82,7 @@ runs:
         git -C control rev-parse HEAD > "$EVIDENCE_DIR/control-revision.txt"
         closure="scripts/Features/BenchmarkComparisons/site-isolated-dependencies.txt"
         cmp "control/$closure" "website/$closure"
-        [[ "$(wc -l < "control/$closure")" == 80 ]]
+        [[ "$(wc -l < "control/$closure")" == 84 ]]
         LC_ALL=C sort -u "control/$closure" > "$EVIDENCE_DIR/isolated-dependencies.txt"
         cmp "control/$closure" "$EVIDENCE_DIR/isolated-dependencies.txt"
         sha256sum "website/$closure" >> "$EVIDENCE_DIR/executed-evidence-tools.sha256"

.github/workflows/benchmarks.yml

@@ -61,7 +61,7 @@ jobs:
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
       - name: Plan all benchmark scenarios
         id: plan
-        run: node scripts/Features/BenchmarkComparisons/isolated-plan.mjs --output="$RUNNER_TEMP/isolated-plan.json" --scale-output="$RUNNER_TEMP/scaled-plan.json" --vector-output="$RUNNER_TEMP/vector-plan.json" --composite-output="$RUNNER_TEMP/composite-plan.json" --open-loop-output="$RUNNER_TEMP/open-loop-plan.json" --github-output="$GITHUB_OUTPUT"
+        run: node scripts/Features/BenchmarkComparisons/isolated-plan.mjs --output="$RUNNER_TEMP/isolated-plan.json" --scale-output="$RUNNER_TEMP/scaled-plan.json" --vector-output="$RUNNER_TEMP/vector-plan.json" --composite-output="$RUNNER_TEMP/composite-plan.json" --open-loop-output="$RUNNER_TEMP/open-loop-plan.json" --document-output="$RUNNER_TEMP/document-plan.json" --github-output="$GITHUB_OUTPUT"
       - name: Save benchmark plan
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
         with:
@@ -72,6 +72,7 @@ jobs:
             ${{ runner.temp }}/vector-plan.json
             ${{ runner.temp }}/composite-plan.json
             ${{ runner.temp }}/open-loop-plan.json
+            ${{ runner.temp }}/document-plan.json
           if-no-files-found: error
           retention-days: 90
   comparison-images:
@@ -157,6 +158,10 @@ jobs:
         run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/NativeBenchmarkProgressEntryTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/live-entry
       - name: Test benchmark phase and attempt counters
         run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/ComparisonLiveProgress*/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/live-counters
+      - name: Test document resource budgets and current native topology
+        run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(DocumentResourceBudgetTests)|(NativeTopologyAdmissionTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/document-admission --KeyLoadTests:ReportTrx=true
+      - name: Test actual native document schedules and cancellation
+        run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/DocumentComparisonNativeTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/document-native --KeyLoadTests:ReportTrx=true
       - name: Test KeyLoad replay admission
         run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedKeyLoadReplayAdmissionTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/replay-admission
       - name: Test Redis replica error reporting
@@ -264,7 +269,7 @@ jobs:
         if: ${{ !cancelled() }}
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
         with:
-          name: ${{ matrix.kind == 'preflight' && 'comparison-preflight-' || matrix.kind == 'proof' && 'comparison-open-loop-proof-' || matrix.kind == 'open-loop' && 'comparison-open-loop-worker-' || 'comparison-worker-' }}${{ matrix.id }}
+          name: ${{ matrix.kind == 'documents' && 'comparison-document-worker-' || matrix.kind == 'preflight' && 'comparison-preflight-' || matrix.kind == 'proof' && 'comparison-open-loop-proof-' || matrix.kind == 'open-loop' && 'comparison-open-loop-worker-' || 'comparison-worker-' }}${{ matrix.id }}
           path: artifacts/comparisons/isolated/workers/${{ matrix.id }}/*
           if-no-files-found: error
           retention-days: 90
@@ -273,7 +278,7 @@ jobs:
         uses: ./.github/workflows/Features/BenchmarkComparisons/IsolatedCellTeardown
         with:
           setup-outcome: ${{ steps.images.outcome }}
-          artifact-name: ${{ matrix.kind == 'preflight' && 'comparison-preflight-qualification-' || matrix.kind == 'proof' && 'comparison-open-loop-proof-qualification-' || matrix.kind == 'open-loop' && 'comparison-open-loop-case-qualification-' || 'comparison-case-qualification-' }}${{ matrix.id }}
+          artifact-name: ${{ matrix.kind == 'documents' && 'comparison-document-qualification-' || matrix.kind == 'preflight' && 'comparison-preflight-qualification-' || matrix.kind == 'proof' && 'comparison-open-loop-proof-qualification-' || matrix.kind == 'open-loop' && 'comparison-open-loop-case-qualification-' || 'comparison-case-qualification-' }}${{ matrix.id }}
   comparison-postgresql:
     name: ${{ matrix.jobName }}
     needs: [comparison-plan, comparison-images]
@@ -384,9 +389,57 @@ jobs:
       matrix: ${{ fromJSON(needs.comparison-plan.outputs.databases).helixdb }}
     env: *database-environment
     steps: *database-steps
+  keyload-functional-heavy-load:
+    name: KeyLoad functional RF3 million-record ingestion
+    needs: [comparison-images]
+    runs-on: ubuntu-latest
+    timeout-minutes: 180
+    steps:
+      - name: Download source code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+      - name: Set up .NET
+        uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68
+        with:
+          global-json-file: global.json
+      - name: Check Docker
+        run: docker version
+      - name: Download exact source Docker images
+        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131
+        with:
+          artifact-ids: ${{ needs.comparison-images.outputs.bundle-artifact-id }}
+          path: ${{ runner.temp }}/keyload-image-bundle
+      - name: Load and verify Docker images
+        id: images
+        run: node scripts/Features/BenchmarkComparisons/import-images.mjs --bundle="$RUNNER_TEMP/keyload-image-bundle"
+      - name: Configure exact source KeyLoad image
+        env:
+          KEYLOAD_SERVER_IMAGE: ${{ steps.images.outputs.server-image }}
+        run: |
+          test -n "$KEYLOAD_SERVER_IMAGE"
+          printf 'KeyLoad__ContainerImages__Server=%s\nKEYLOAD_IMAGE_RECEIPT=%s\n' "$KEYLOAD_SERVER_IMAGE" "$RUNNER_TEMP/keyload-images/image-receipt.json" >> "$GITHUB_ENV"
+      - name: Restore .NET packages
+        run: dotnet restore KeyLoad.slnx
+      - name: Build functional RF3 tests
+        run: dotnet build tests/KeyLoad.IntegrationTests --no-restore --configuration Release
+      - name: Test ingestion while SDK and MCP reads are active
+        run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=rf3 '--KeyLoadTests:Filter=/*/*/HeavyDocumentLoadRf3Tests/*' --KeyLoadTests:HeavyLoad:Enabled=true --KeyLoadTests:TimeoutMinutes=140 --KeyLoadTests:ResultsDirectory=TestResults/rf3-heavy-load --KeyLoadTests:ReportTrx=true
+      - name: Clean up this job's Docker registry
+        if: always()
+        run: node scripts/Features/BenchmarkComparisons/cleanup-images.mjs
+      - name: Save original functional load results and image receipts
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
+        with:
+          name: keyload-functional-heavy-load
+          path: |
+            TestResults/rf3-heavy-load/**
+            ${{ runner.temp }}/keyload-images/**
+            artifacts/code-quality/**
+          if-no-files-found: error
+          retention-days: 90
   comparison-aggregate:
     name: Combine benchmark results
-    needs: [comparison-build, comparison-plan, comparison-images, comparison-keyload, comparison-postgresql, comparison-qdrant, comparison-rabbitmq, comparison-redis, comparison-neo4j, comparison-mongodb, comparison-opensearch, comparison-kurrentdb, comparison-surrealdb, comparison-helixdb]
+    needs: [comparison-build, comparison-plan, comparison-images, comparison-keyload, comparison-postgresql, comparison-qdrant, comparison-rabbitmq, comparison-redis, comparison-neo4j, comparison-mongodb, comparison-opensearch, comparison-kurrentdb, comparison-surrealdb, comparison-helixdb, keyload-functional-heavy-load]
     if: ${{ always() && !cancelled() && needs.comparison-plan.result == 'success' && needs.comparison-images.result == 'success' }}
     runs-on: ubuntu-latest
     timeout-minutes: 150
@@ -398,7 +451,7 @@ jobs:
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
       - name: Verify benchmark plan
         id: plan
-        run: node scripts/Features/BenchmarkComparisons/isolated-plan.mjs --output="$RUNNER_TEMP/isolated-plan.json" --scale-output="$RUNNER_TEMP/scaled-plan.json" --vector-output="$RUNNER_TEMP/vector-plan.json" --composite-output="$RUNNER_TEMP/composite-plan.json" --open-loop-output="$RUNNER_TEMP/open-loop-plan.json"
+        run: node scripts/Features/BenchmarkComparisons/isolated-plan.mjs --output="$RUNNER_TEMP/isolated-plan.json" --scale-output="$RUNNER_TEMP/scaled-plan.json" --vector-output="$RUNNER_TEMP/vector-plan.json" --composite-output="$RUNNER_TEMP/composite-plan.json" --open-loop-output="$RUNNER_TEMP/open-loop-plan.json" --document-output="$RUNNER_TEMP/document-plan.json"
       - name: Download benchmark results
         run: node scripts/Features/BenchmarkComparisons/isolated-github-collect.mjs --plan="$RUNNER_TEMP/isolated-plan.json" --scale-plan="$RUNNER_TEMP/scaled-plan.json" --vector-plan="$RUNNER_TEMP/vector-plan.json" --input="$RUNNER_TEMP/isolated-capture"
       - name: Check control and complete scale accounting
@@ -452,9 +505,33 @@ jobs:
           path: ${{ runner.temp }}/open-loop-aggregate/**
           if-no-files-found: warn
           retention-days: 90
+      - name: Download authenticated document workload results
+        id: document-intake
+        if: ${{ always() && !cancelled() && steps.plan.outcome == 'success' }}
+        run: node scripts/Features/BenchmarkComparisons/document-github-collect.mjs --capture="$RUNNER_TEMP/isolated-capture/github" --plan="$RUNNER_TEMP/document-plan.json" --output="$RUNNER_TEMP/document-intake"
+      - name: Aggregate complete pure mixed and ingestion results
+        id: document-aggregate
+        if: ${{ always() && !cancelled() && steps.document-intake.outcome == 'success' }}
+        run: node scripts/Features/BenchmarkComparisons/document-aggregate-cli.mjs --input="$RUNNER_TEMP/document-intake" --output="$RUNNER_TEMP/document-aggregate"
+      - name: Save original document workload evidence
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
+        with:
+          name: comparison-document-intake
+          path: ${{ runner.temp }}/document-intake/**
+          if-no-files-found: warn
+          retention-days: 90
+      - name: Save complete document workload cohort
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
+        with:
+          name: comparison-document-cohort
+          path: ${{ runner.temp }}/document-aggregate/**
+          if-no-files-found: warn
+          retention-days: 90
   website-trigger:
     name: Trigger Website
-    needs: [comparison-build, comparison-plan, comparison-images, comparison-keyload, comparison-postgresql, comparison-qdrant, comparison-rabbitmq, comparison-redis, comparison-neo4j, comparison-mongodb, comparison-opensearch, comparison-kurrentdb, comparison-surrealdb, comparison-helixdb, comparison-aggregate]
+    needs: [comparison-build, comparison-plan, comparison-images, comparison-keyload, comparison-postgresql, comparison-qdrant, comparison-rabbitmq, comparison-redis, comparison-neo4j, comparison-mongodb, comparison-opensearch, comparison-kurrentdb, comparison-surrealdb, comparison-helixdb, keyload-functional-heavy-load, comparison-aggregate]
     if: >-
       always() && !cancelled() &&
       needs.comparison-aggregate.result == 'success' &&