Isolate benchmark registry fixtures with owned ephemeral ports

Author: KSemenenko

docs/ADR/ADR-080-benchmark-failure-isolation.md

@@ -41,6 +41,16 @@ upload; cancellation/timeouts that prevent artifacts remain explicit blockers.
    Run these actual loopback fixture tests before `prepare-images.mjs` owns the
    same registry port; restore/build the native test runners first. Keep the
    actual pinned image export/import checks after image construction.
+   The readiness helper accepts an optional numeric loopback port with default
+   registry.port; validate integer1..65535 before URL construction, HTTP or
+   evidence I/O. Both production callers retain their no-argument default5000.
+   Actual HTTP fixtures listen on kernel-assigned port0, retain that listener
+   until cleanup and pass only its observed port. Extend the existing bounded
+   evidence case with invalid-number/type checks, zero HTTP requests and absent
+   evidence assertions; do not change OS settings/processes or accept arbitrary
+   hosts/URLs/environment overrides. Root integrates this port-isolation follow-up,
+   reviews the unchanged production callers and reruns the10 original scenarios
+   through freshly built Aspire/TUnit before the next scoped checkpoint.
    Under FAIL-PREP-KURRENT, `KurrentTarget.InitializeAsync` constructs the actual
    SDK writer only after `KurrentClusterVerifier.VerifyAsync` proves membership.
    Three SDK/resource-model tests belong to ComparisonTests and run in common

docs/Features/BenchmarkComparisons.md

@@ -688,7 +688,7 @@ KeyLoad engine repair and concurrent series-codec work are outside this task.
 | REQ-BC-FAIL-003 authenticated partial results | AC-BC-FAIL-003 failed job accepted only with failed workload, successful result upload and matching failed envelope; missing/malformed/expired/mixed evidence rejected | producer/aggregate/site negative and positive TUnit regressions |
 | REQ-BC-FAIL-004 honest site | AC-BC-FAIL-004 successful competitors retain values; failed cells have no numeric values and expose actual job link | independent numeric oracle, projection validation and real Chrome |
 | REQ-BC-FAIL-005 repair shared preparation | AC-BC-FAIL-005 diagnose exact failed logs, repair benchmark setup/build invocation, retain native isolated topology and Aspire ownership | exact failed-source log, focused regression, delivered-source GitHub rerun |
-| REQ-BC-FAIL-006 bounded registry readiness | AC-BC-FAIL-006 each native HTTP probe has at most2s within the unchanged30s total; only settled non-aborted HTTP200 succeeds; private no-follow diagnostics remain at most121 records/64KiB and evidence-write failures propagate | `ImageRegistryReadinessTests`:10 actual loopback HTTP/error/bounds cases, plus genuine pinned Docker image export/import in GitHub |
+| REQ-BC-FAIL-006 bounded registry readiness | AC-BC-FAIL-006 each native HTTP probe has at most2s within the unchanged30s total; only settled non-aborted HTTP200 succeeds; private no-follow diagnostics remain at most121 records/64KiB and evidence-write failures propagate. Actual fixture listeners use kernel-assigned loopback ports; optional numeric port accepts only integers1..65535 before HTTP/evidence, while both production callers keep the default5000 | `ImageRegistryReadinessTests`:10 actual loopback HTTP/error/bounds cases, including rejected port inputs before I/O, plus genuine pinned Docker image export/import in GitHub |
 | REQ-BC-FAIL-007 Kurrent writer starts after membership | AC-BC-FAIL-007 verify all native1/2/3-member views before constructing the SDK writer; retain native DNS seeds, TLS verification, leader preference, NoStream semantics, acknowledgements, replica-copy oracle and cleanup | `IsolatedKurrentDiscoverySettingsTests`:3 actual SDK/resource-model cases; genuine Aspire-owned StreamAppend preflights for1/2/3 nodes |
 | REQ-BC-FAIL-008 explicit cancellation stops owned work | AC-BC-FAIL-008 workload, finalization and result upload use `!cancelled()` so ordinary failure still finalizes while cancellation stops execution/publication; `always()` cleanup retains bounded diagnostics and safely removes owned registries whose setup was cancelled | `WorkflowBenchmarkFailureTests`, unchanged canceled-job/producer rejection regressions and actual GitHub lifecycle |
 
@@ -775,6 +775,31 @@ Full Release, scoped Unit build and formatter were blocked by concurrent
 KeyCodec/CRUD source/test diagnostics. New workflow C# tests and actual complete
 publication remain pending; none of that engine work is staged in this repair.
 
+The clean88e2cec snapshot passed full Release build, formatter and governance.
+Its actual full unit report has2821/2867 passes,46 failures, no skips/cancellation:
+workflow cancellation3/3 and all selected workflow-layout groups passed. One
+registry case failed with original `EADDRINUSE`; concurrent local tests and an
+observed system ControlCenter listener share port5000. FAIL-PREP-REGISTRY therefore
+uses an OS-owned ephemeral port for each actual HTTP fixture. Only a numeric port
+on the fixed loopback URL is configurable in the readiness helper; preparation,
+bundle publication, Docker mapping, manifest fetch and cleanup keep their existing
+default5000. The evidence-bounds case verifies invalid numbers/types produce no
+requests or evidence directory. Other full-unit failures include temp-path link
+rejection and a native allocation assertion; those are not repaired here. Reruns
+use an owned canonical temporary directory and retain the original failed report.
+Clean-snapshot failed-envelope producer/finalizer2/2 passed through Aspire. The
+chunk development child failed because that archive has no Git metadata; its
+original stderr proves the failed `git check-ignore`, rather than a codec defect.
+
+The subsequent benchmark-only port-isolation candidate (archive of88e2cec plus
+its scoped patch) passed full Release build with0warnings/errors, formatter and
+governance. Its freshly built actual Aspire/TUnit registry scenarios passed10/10
+normal and10/10 scalar, no skips/cancellation/timeouts, using an owned canonical
+temporary directory. The production callers still omit the optional port.
+Source/module/fixture/runner and original report hashes are retained in the local
+development receipt and status. Complete delivered-source GitHub publication is
+pending; these development results do not qualify a full cohort or website.
+
 ```mermaid
 flowchart LR
   Plan[Complete native cell plan] --> Jobs[Independent Aspire workloads]

docs/implementation/status.json

@@ -3190,15 +3190,15 @@
         "scope": "preparation and native preflight only;4096-record development workload; no scale, full cohort, winner or power-loss qualification"
       }
     },
-    "localDevelopmentVerification": "Earlier integrated Release build/formatter and Aspire registry10/10, SDK settings3/3, producer/finalizer2/2, GitHub evidence84/84 and original report probes48/48 passed. Delivered6ec Linux clean build, formatter, workflow inventory and native image checks passed. Follow-up current JS registry fixtures passed10/10 normal and10/10 scalar through the existing Aspire/TUnit bridge (not a newly compiled C# runner). Governance and diff checks pass. Current full Release, scoped Unit build and formatter are blocked by concurrent KeyCodec/CRUD source/test diagnostics outside benchmark scope; new workflow C# tests and complete delivered-source native/site/Pages qualification remain pending.",
+    "localDevelopmentVerification": "Delivered6ec Linux clean build, formatter, workflow inventory, registry10/10 and SDK/resource settings3/3 passed. Clean88e2cec snapshot passed full Release build, formatter and governance; actual full unit2821/2867 passed with46 failures, no skips/cancellation (including a registry port conflict, macOS temp-link rejections, missing Git metadata for the chunk child and an allocation assertion). Workflow cancellation3/3 and failed producer/finalizer2/2 passed through Aspire. Benchmark-only ephemeral-port follow-up freshly built clean with0warnings/errors; formatter and governance passed; actual registry10/10 normal and10/10 scalar passed through Aspire/TUnit with canonical owned TMPDIR. Complete delivered-source native/site/Pages qualification remains pending; no KeyLoad engine repair in this stage.",
     "runtimeQualified": false,
     "publicationQualified": false,
     "keyLoadEngineChanges": false,
     "cancellationRepair": {
       "requirement": "REQ-BC-FAIL-008",
       "ordinaryFailure": "preserves original failed workload and authenticated null-report upload",
       "explicitCancellation": "stops workload/finalization/result upload; bounded always cleanup validates owned partial setup",
-      "qualification": "source reviewed; focused Aspire regression and genuine delivered-source lifecycle pending"
+      "qualification": "Clean88e2cec actual Aspire/TUnit workflow cancellation3/3 passed; genuine delivered-source lifecycle pending."
     },
     "registryDeadlineRegression": {
       "ciRunId": 37161833095,
@@ -3230,6 +3230,53 @@
         "normalReportSha256": "fbd9c1894275e594821a547fbf8a3278a504cc7b1ea2dbacc4a25f41c9dd99a4",
         "scalarReportSha256": "723d3a6c7211b30fe09e45a962e2a82274bed38fb1573c00eff43003b455c8de"
       }
+    },
+    "registryPortIsolation": {
+      "requirement": "REQ-BC-FAIL-006",
+      "production": "both no-argument callers retain fixed loopback port5000",
+      "fixture": "actual listeners own kernel-assigned ports through cleanup; invalid integer/type inputs rejected before HTTP or evidence I/O",
+      "developmentSource": "88e2cec05d31008e9887fe8e970756b7691b7734",
+      "candidatePatchSha256": "f46d5eadfdf74aa61e785383e610454925ccc08c2afdcae3af93eb4b65539c89",
+      "fullReleaseBuild": "passed;0warnings;0errors",
+      "formatter": "passed",
+      "governance": "passed",
+      "invoker": "Aspire-owned freshly built TUnit; owned canonical TMPDIR; actual ephemeral HTTP listeners",
+      "sourceFiles": {
+        "scripts/Features/BenchmarkComparisons/image-manifest.mjs": "3d15894b8427e7190ecd494548ae59edb9151b565a5dba74ea6be474a97202a7",
+        "tests/KeyLoad.UnitTests/Features/BenchmarkComparisons/ImageRegistryReadinessFixture.mjs": "dac1b3b67da8dc48dc1b3c82baaf133f278e126a19e2f5e147022287e6fec076",
+        "tests/KeyLoad.UnitTests/bin/Release/net10.0/KeyLoad.UnitTests.dll": "eb86c57f759baf08567200f318f63b97d5c10bd8bd191a1bd0ccb4efaea1477a"
+      },
+      "reports": {
+        "normal": {
+          "summary": {
+            "total": 10,
+            "passed": 10,
+            "failed": 0,
+            "skipped": 0,
+            "cancelled": 0,
+            "timedOut": 0,
+            "flaky": 0
+          },
+          "sha256": "a211bc3a675c2c5a4b89134c080496962cfd42effefe48bd86741c5b6377e543",
+          "operatingSystem": "macOS 27.0.1",
+          "runtimeVersion": ".NET 10.0.12"
+        },
+        "scalar": {
+          "summary": {
+            "total": 10,
+            "passed": 10,
+            "failed": 0,
+            "skipped": 0,
+            "cancelled": 0,
+            "timedOut": 0,
+            "flaky": 0
+          },
+          "sha256": "749ed17a17a1318fd216e822ca478976dfd49c182720c3aba1f24a2250e3b6da",
+          "operatingSystem": "macOS 27.0.1",
+          "runtimeVersion": ".NET 10.0.12"
+        }
+      },
+      "githubQualified": false
     }
   }
 }

scripts/Features/BenchmarkComparisons/image-manifest.mjs

@@ -8,6 +8,8 @@ const metadataSeparator = '|';
 const mediaTypeParameterSeparator = ';';
 const wait = milliseconds => new Promise(resolve => setTimeout(resolve, milliseconds));
 const invalidDeadline = 'The owned image HTTP deadline is invalid.';
+const invalidRegistryPort = 'The owned loopback registry port is invalid.';
+const maxRegistryPort = 65535;
 const timeoutMessage = 'The owned image HTTP operation exceeded its time bound.';
 const cleanupErrorGroups = new WeakMap();
 
@@ -90,22 +92,27 @@ export function parseManifestEvidence(bytes, digestHeader, contentTypeHeader, ex
   });
 }
 
-export async function waitForRegistry(context) {
+export async function waitForRegistry(context, registryPort = registry.port) {
+  if (!Number.isInteger(registryPort) || registryPort <= 0 || registryPort > maxRegistryPort) {
+    throw new RangeError(invalidRegistryPort);
+  }
+  const url = new URL(registryProtocol.path, registry.url);
+  url.port = String(registryPort);
   const deadline = Date.now() + processLimit.readinessTimeoutMs;
   let sequence = 0;
   while (Date.now() < deadline) {
     const remaining = deadline - Date.now();
     if (remaining <= 0) break;
     sequence++;
-    const result = await probeRegistry(context, sequence, Math.min(processLimit.readinessProbeTimeoutMs, remaining));
+    const result = await probeRegistry(context, sequence, Math.min(processLimit.readinessProbeTimeoutMs, remaining), url);
     if (result.outcome === registryReadinessTokens.outcome.ready && Date.now() < deadline) return;
     const backoff = Math.min(processLimit.readinessIntervalMs, deadline - Date.now());
     if (backoff > 0) await wait(backoff);
   }
   throw new Error(message.registryTimeout);
 }
 
-async function probeRegistry(context, sequence, timeoutMs) {
+async function probeRegistry(context, sequence, timeoutMs, url) {
   const started = Date.now();
   let signal;
   let status = null;
@@ -115,7 +122,7 @@ async function probeRegistry(context, sequence, timeoutMs) {
   try {
     await withHttpDeadline(timeoutMs, async value => {
       signal = value;
-      const response = await fetch(`${registry.url}${registryProtocol.path}`, { signal, redirect: 'error' });
+      const response = await fetch(url, { signal, redirect: 'error' });
       status = response.status;
       phase = registryReadinessTokens.phase.bodyCancel;
       await response.body?.cancel();

tests/KeyLoad.UnitTests/Features/BenchmarkComparisons/ImageRegistryReadinessFixture.mjs

@@ -24,7 +24,7 @@ async function runReadinessScenario() {
     if (scenario === 'evidence-unsafe') await prepareUnsafeEvidence();
     const started = Date.now();
     let failure;
-    try { await waitForRegistry(context); } catch (error) { failure = error; }
+    try { await waitForRegistry(context, fixture.port); } catch (error) { failure = error; }
     const elapsedMs = Date.now() - started;
     if (scenario === 'evidence-unsafe') {
       assert.equal(failure?.message, message.unsafeEvidencePath);
@@ -77,8 +77,10 @@ function serveRequest(fixture, request, response) {
 async function listenFixture(fixture) {
   await new Promise((resolve, reject) => {
     fixture.server.once('error', reject);
-    fixture.server.listen(registry.port, registry.host, () => {
+    fixture.server.listen(0, registry.host, () => {
       fixture.server.removeListener('error', reject);
+      fixture.port = fixture.server.address().port;
+      assert(Number.isInteger(fixture.port) && fixture.port > 0 && fixture.port <= 65535);
       fixture.listening = true;
       resolve();
     });
@@ -153,6 +155,7 @@ function assertSuccessfulReadiness(failure, elapsedMs, rows, fixture) {
 }
 
 async function assertEvidenceBounds() {
+  await assertRejectedRegistryPorts();
   const valid = Object.freeze({ sequence: 1, startedAt: '2026-10-04T00:00:00.000Z', durationMs: 12,
     timeoutMs: 2000, status: 200, aborted: false, phase: 'body-cancel', outcome: 'ready', errorCode: null });
   for (const input of invalidEvidence(valid)) {
@@ -171,6 +174,23 @@ async function assertEvidenceBounds() {
   await assertEvidenceByteLimit(valid);
 }
 
+async function assertRejectedRegistryPorts() {
+  const fixture = makeFixture();
+  try {
+    await listenFixture(fixture);
+    const invalid = [0, -1, 65536, 1.5, NaN, Infinity, null, {}, String(fixture.port)];
+    for (const port of invalid) {
+      await assert.rejects(waitForRegistry(context, port), {
+        name: 'RangeError', message: 'The owned loopback registry port is invalid.',
+      });
+    }
+    assert.equal(fixture.requests, 0);
+    await assert.rejects(stat(context.evidenceDirectory), { code: 'ENOENT' });
+  } finally {
+    await closeFixture(fixture);
+  }
+}
+
 async function assertEvidenceByteLimit(valid) {
   const overflowTemp = path.join(context.runnerTemp, 'evidence-bytes');
   await mkdir(overflowTemp, { recursive: true, mode: 0o700 });