Fix comparison callers and add portable SIMD validation

Author: KSemenenko

.github/workflows/ci.yml

@@ -53,6 +53,10 @@ jobs:
       - name: Verify source-owned Roslyn rules
         run: dotnet test --project tests/KeyLoad.Analyzers.Tests --no-build --no-restore --configuration Release
       - run: dotnet test --project tests/KeyLoad.UnitTests --no-build --no-restore --configuration Release
+      - name: Qualify the full unit suite with hardware intrinsics disabled
+        run: dotnet test --project tests/KeyLoad.UnitTests --no-build --no-restore --configuration Release --results-directory TestResults/unit-scalar
+        env:
+          DOTNET_EnableHWIntrinsic: 0
       - name: Qualify real process recovery after a successful build
         if: ${{ !cancelled() && steps.build.outcome == 'success' }}
         run: dotnet test --project tests/KeyLoad.RecoveryTests --no-build --no-restore --configuration Release

AGENTS.md

@@ -17,9 +17,10 @@ ManagedCode packages are our projects. Fix dependency defects in their owning si
 - When the user requests committing all current changes, commit the full requested working-tree scope on the currently checked-out branch; do not silently omit existing changes or move them to another branch.
 
 ## System-critical SMID and operation efficiency
-- Treat the owner's SMID work as a first-priority product workstream and keep it visible in implementation planning and status. The term is not defined in the current repository or v0.3 architecture documents; do not guess its expansion or begin SMID-specific implementation until its canonical slice, REQ/AC criteria and required ADRs are mapped with the owner.
+- Treat the owner's SIMD work (earlier written as SMID) as a first-priority product workstream and keep it visible in implementation planning and status. On 2026-10-02 the owner confirmed SIMD means vectorized CPU operations and directed .NET intrinsics first, with Rust considered only after profiling. Map its canonical slices, REQ/AC criteria, scalar correctness/portability contract and required ADRs before implementation; never claim acceleration without comparable GitHub measurements.
 - Orleans provides request isolation, cluster routing and activation movement. Keep the one-grain-per-request boundary and RF3 topology; treat Orleans Streams as a first-class architecture workstream to specify and qualify alongside persisted KeyLoad EventStreams. Keep their distinct delivery/restart contracts explicit, and define bounded resource, concurrency, backpressure, recovery and performance behavior for every operation.
 - Optimize each operation only against its correctness and fault contracts, using representative multi-node GitHub qualification to measure latency, throughput, allocations, memory, contention and backlog where applicable. Architecture choices or local builds alone do not prove maximum scalability or performance.
+- Use ZoneTree's native storage APIs correctly and Orleans for bounded parallel execution of independent operation work. Preserve node-local storage ownership, the ordered atomic commit/apply gate, scoped read cuts, cancellation and backpressure; qualify the resulting performance in real multi-node GitHub runs (owner direction 2026-10-02).
 
 ## MCAF repository workflow
 

README.md

@@ -14,7 +14,9 @@ This is an early implementation of the clustered kernel. The default server topo
 
 Implemented surfaces include document CRUD and field patches, partition-scoped unique/composite equality indexes, command deduplication, stream expected-revision append/read, retained topics, per-source durable subscriptions with bounded delivery windows and contiguous checkpoints, scheduled work queues with fenced leases and retry/DLQ, atomic inbox completion, a committed projection outbox with generation pins, protected resumable document changes and scalar live queries, bounded property-graph traversal, ordered samples, exact vector search, BM25 and weighted reciprocal-rank fusion, a bounded read-only Q1 SQL dialect, API keys, field omission and field-use policies, verified backups, native Raft snapshots and empty-replica catch-up, offline journal compaction, a .NET SDK and CLI.
 
-The additive [TimeSeries reads](docs/Features/TimeSeries.md) provide latest samples, complete half-open raw aggregates and dense UTC windows through typed SDK, HTTP and MCP operations using published `ManagedCode.TimeSeries` 10.0.0. They preserve the existing inclusive sample range API and persisted authorization. The first complete new-source CI exposed test fixture/catalog/cancellation expectations; their [source corrections](docs/implementation/qualification-source-r9.json) require their own delivered-SHA CI.
+The additive [TimeSeries reads](docs/Features/TimeSeries.md) provide latest samples, complete half-open raw aggregates and dense UTC windows through typed SDK, HTTP and MCP operations using published `ManagedCode.TimeSeries` 10.0.0. They preserve the existing inclusive sample range API and persisted authorization. Their fixture/catalog/cancellation corrections are exercised in [exact c486 CI](docs/implementation/runtime-qualification-37060131271.json): 871/871 unit cases on each OS and 46/46 genuine RF3 SDK/MCP cases pass. Retention, rollups, SQL integration and resource qualification remain open.
+
+That complete workflow still fails: Windows recovery is 116/136 and native comparison completion is 2/4; measured profiles were skipped. [The performance diagnosis](docs/implementation/performance-diagnosis-37060131271.json) records missing RF3/Rabbit caller bindings and PostgreSQL stream readback, plus unqualified 48-sample TimeSeries observations. These do not establish a performance winner. The next source stage repairs those callers and adds portable .NET SIMD finite-value validation with unchanged scoring, plus a full unit-suite invocation with intrinsics disabled. Its own delivered-SHA tests and measured benefit are pending; Rust requires profiling evidence first.
 
 The [104-task implementation tracker](docs/implementation/status.json) records the remaining work. Automatic canonical journal maintenance, large and interrupted snapshot transfer qualification, shard movement, distributed multi-shard query planning, managed HNSW, subscription coverage discovery across partitions and rebalance, automatic retention and generation cleanup, schema migrations, external backup stores and full release qualification remain under development. The current server uses one replicated physical shard which contains many independent atomic partitions.
 

benchmarks/KeyLoad.ComparisonHost/Features/BenchmarkComparisons/ComparisonEndpointBindings.cs

@@ -0,0 +1,116 @@
+using System.Collections.Immutable;
+using System.Globalization;
+using System.Net.Http.Headers;
+using System.Text;
+using Microsoft.Extensions.Configuration;
+
+namespace KeyLoad.ComparisonHost.Features.BenchmarkComparisons;
+
+/// <summary>Validates endpoints and credentials required by the comparison host callers.</summary>
+internal static class ComparisonEndpointBindings
+{
+    private const string HttpScheme = "http";
+    private const string HttpsScheme = "https";
+    private const string UriRootPath = "/";
+    private const int PrimaryEndpointIndex = 0;
+    private const string FirstEndpointIndex = "0";
+    private const string SecondEndpointIndex = "1";
+    private const string ThirdEndpointIndex = "2";
+
+    internal static ImmutableArray<Uri> ReadKeyLoadEndpoints(IConfiguration configuration, Uri primary)
+    {
+        var endpoints = ImmutableArray.CreateBuilder<Uri>(ComparisonHostConstants.KeyLoadEndpointCount);
+        for (var index = 0; index < ComparisonHostConstants.KeyLoadEndpointCount; index++)
+        {
+            var key = ComparisonHostConstants.KeyLoadEndpoints + ":" + index.ToString(CultureInfo.InvariantCulture);
+            if (!TryParseHttpOrigin(ComparisonHostSettings.RequiredValue(configuration, key), out var endpoint))
+            {
+                throw InvalidKeyLoadEndpoints();
+            }
+
+            endpoints.Add(endpoint);
+        }
+
+        var children = configuration.GetSection(ComparisonHostConstants.KeyLoadEndpoints).GetChildren().ToArray();
+        if (children.Length != ComparisonHostConstants.KeyLoadEndpointCount ||
+            children.Any(child => child.Key is not (FirstEndpointIndex or SecondEndpointIndex or ThirdEndpointIndex)))
+        {
+            throw InvalidKeyLoadEndpoints();
+        }
+
+        var result = endpoints.MoveToImmutable();
+        var seenOrigins = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
+        foreach (var endpoint in result)
+        {
+            if (!seenOrigins.Add(endpoint.GetLeftPart(UriPartial.Authority)))
+            {
+                throw InvalidKeyLoadEndpoints();
+            }
+        }
+
+        if (!IsHttpOrigin(primary) || !SameOrigin(primary, result[PrimaryEndpointIndex]))
+        {
+            throw InvalidKeyLoadEndpoints();
+        }
+
+        return result;
+    }
+
+    internal static Uri ReadRabbitManagementEndpoint(IConfiguration configuration)
+    {
+        var value = ComparisonHostSettings.RequiredValue(configuration,
+            ComparisonHostConstants.RabbitManagementEndpoint);
+        if (!TryParseHttpOrigin(value, out var endpoint))
+        {
+            throw new InvalidOperationException(ComparisonHostConstants.RabbitManagementEndpointInvalidCode);
+        }
+
+        return endpoint;
+    }
+
+    internal static void ValidateRabbitCredentials(string user, string password)
+    {
+        if (string.IsNullOrWhiteSpace(user) || string.IsNullOrWhiteSpace(password) ||
+            user.Contains(ComparisonHostConstants.UserPasswordSeparator, StringComparison.Ordinal) ||
+            ContainsLineBreak(user) || ContainsLineBreak(password))
+        {
+            throw new InvalidOperationException(ComparisonHostConstants.RabbitManagementCredentialsInvalidCode);
+        }
+    }
+
+    internal static AuthenticationHeaderValue CreateRabbitAuthorization(string user, string password)
+    {
+        var credential = user + ComparisonHostConstants.UserPasswordSeparator + password;
+        var token = Convert.ToBase64String(Encoding.UTF8.GetBytes(credential));
+        return new AuthenticationHeaderValue(ComparisonHostConstants.BasicAuthenticationScheme, token);
+    }
+
+    private static bool TryParseHttpOrigin(string value, out Uri endpoint)
+    {
+        if (Uri.TryCreate(value, UriKind.Absolute, out var parsed) && IsHttpOrigin(parsed))
+        {
+            endpoint = parsed;
+            return true;
+        }
+
+        endpoint = null!;
+        return false;
+    }
+
+    private static bool IsHttpOrigin(Uri endpoint)
+        => (string.Equals(endpoint.Scheme, HttpScheme, StringComparison.OrdinalIgnoreCase) ||
+            string.Equals(endpoint.Scheme, HttpsScheme, StringComparison.OrdinalIgnoreCase)) &&
+            !string.IsNullOrWhiteSpace(endpoint.Host) && endpoint.UserInfo.Length == 0 &&
+            endpoint.Query.Length == 0 && endpoint.Fragment.Length == 0 &&
+            string.Equals(endpoint.AbsolutePath, UriRootPath, StringComparison.Ordinal);
+
+    private static bool SameOrigin(Uri left, Uri right)
+        => string.Equals(left.GetLeftPart(UriPartial.Authority), right.GetLeftPart(UriPartial.Authority),
+            StringComparison.OrdinalIgnoreCase);
+
+    private static bool ContainsLineBreak(string value)
+        => value.Contains('\r', StringComparison.Ordinal) || value.Contains('\n', StringComparison.Ordinal);
+
+    private static InvalidOperationException InvalidKeyLoadEndpoints()
+        => new(ComparisonHostConstants.KeyLoadEndpointInvalidCode);
+}

benchmarks/KeyLoad.ComparisonHost/Features/BenchmarkComparisons/ComparisonHostConstants.cs

@@ -3,6 +3,8 @@ namespace KeyLoad.ComparisonHost.Features.BenchmarkComparisons;
 internal static class ComparisonHostConstants
 {
     internal const string KeyLoadEndpoint = "Benchmarks:KeyLoadEndpoint";
+    internal const string KeyLoadEndpoints = "Benchmarks:KeyLoadEndpoints";
+    internal const string KeyLoadEndpointInvalidCode = "KeyLoadComparisonEndpointsInvalid";
     internal const string Profile = "Benchmarks:Profile";
     internal const string TimeSeriesProfile = "timeseries";
     internal const string TimescaleConnection = "ConnectionStrings:benchmark-timescale";
@@ -14,6 +16,11 @@ internal static class ComparisonHostConstants
     internal const string AdminKey = "Benchmarks:AdminKey";
     internal const string PostgresConnection = "ConnectionStrings:benchmark-postgres";
     internal const string RabbitConnection = "ConnectionStrings:benchmark-rabbit";
+    internal const string RabbitManagementEndpoint = "Benchmarks:RabbitManagementEndpoint";
+    internal const string RabbitUser = "Benchmarks:RabbitUser";
+    internal const string RabbitPassword = "Benchmarks:RabbitPassword";
+    internal const string RabbitManagementEndpointInvalidCode = "RabbitManagementEndpointInvalid";
+    internal const string RabbitManagementCredentialsInvalidCode = "RabbitManagementCredentialsInvalid";
     internal const string RedisConnection = "ConnectionStrings:benchmark-redis";
     internal const string PostgresImage = "Benchmarks:Images:Postgres";
     internal const string QdrantImage = "Benchmarks:Images:Qdrant";
@@ -38,6 +45,7 @@ internal static class ComparisonHostConstants
     internal const int FailedExitCode = 1;
     internal const int LifetimeHours = 2;
     internal const int TargetCount = 6;
-    internal const int HttpClientCount = 3;
+    internal const int KeyLoadEndpointCount = 3;
+    internal const int HttpClientCount = 6;
     internal static readonly TimeSpan InfiniteTimeout = Timeout.InfiniteTimeSpan;
 }

benchmarks/KeyLoad.ComparisonHost/Features/BenchmarkComparisons/ComparisonHostSettings.cs

@@ -1,3 +1,4 @@
+using System.Collections.Immutable;
 using KeyLoad.Comparisons;
 using Microsoft.Extensions.Configuration;
 
@@ -7,13 +8,17 @@ namespace KeyLoad.ComparisonHost.Features.BenchmarkComparisons;
 /// <param name="Options">Validated workload options.</param>
 /// <param name="RunId">Unique identifier used to isolate target data.</param>
 /// <param name="KeyLoadEndpoint">KeyLoad endpoint.</param>
+/// <param name="KeyLoadEndpoints">Immutable KeyLoad RF3 peer endpoints, including the primary endpoint.</param>
 /// <param name="QdrantEndpoint">Qdrant endpoint.</param>
 /// <param name="QdrantApiKey">Qdrant API key.</param>
 /// <param name="Neo4jEndpoint">Neo4j endpoint.</param>
 /// <param name="Neo4jPassword">Neo4j password.</param>
 /// <param name="AdminKey">KeyLoad administrator key.</param>
 /// <param name="PostgresConnection">PostgreSQL connection string.</param>
 /// <param name="RabbitConnection">RabbitMQ connection string.</param>
+/// <param name="RabbitManagementEndpoint">RabbitMQ management API endpoint.</param>
+/// <param name="RabbitUser">RabbitMQ management API user.</param>
+/// <param name="RabbitPassword">RabbitMQ management API password.</param>
 /// <param name="RedisConnection">Redis connection string.</param>
 /// <param name="PostgresImage">PostgreSQL image label.</param>
 /// <param name="QdrantImage">Qdrant image label.</param>
@@ -27,13 +32,17 @@ internal sealed record ComparisonHostSettings(
     ComparisonOptions Options,
     string RunId,
     Uri KeyLoadEndpoint,
+    ImmutableArray<Uri> KeyLoadEndpoints,
     Uri QdrantEndpoint,
     string QdrantApiKey,
     Uri Neo4jEndpoint,
     string Neo4jPassword,
     string AdminKey,
     string PostgresConnection,
     string RabbitConnection,
+    Uri RabbitManagementEndpoint,
+    string RabbitUser,
+    string RabbitPassword,
     string RedisConnection,
     string PostgresImage,
     string QdrantImage,
@@ -53,6 +62,7 @@ internal static ComparisonHostSettings Read(IConfiguration configuration)
         var options = ComparisonOptions.Read(configuration);
         var keyLoadEndpoint = ReadEndpoint(configuration, ComparisonHostConstants.KeyLoadEndpoint);
         var qdrantEndpoint = ReadEndpoint(configuration, ComparisonHostConstants.QdrantEndpoint);
+        var keyLoadEndpoints = ComparisonEndpointBindings.ReadKeyLoadEndpoints(configuration, keyLoadEndpoint);
         var qdrantApiKey = Required(configuration, ComparisonHostConstants.QdrantApiKey);
         var neo4jEndpoint = ReadEndpoint(configuration, ComparisonHostConstants.Neo4jEndpoint);
         var neo4jPassword = Required(configuration, ComparisonHostConstants.Neo4jPassword);
@@ -61,6 +71,10 @@ internal static ComparisonHostSettings Read(IConfiguration configuration)
         var postgresImage = Required(configuration, ComparisonHostConstants.PostgresImage);
         var qdrantImage = Required(configuration, ComparisonHostConstants.QdrantImage);
         var rabbitConnection = Required(configuration, ComparisonHostConstants.RabbitConnection);
+        var rabbitManagementEndpoint = ComparisonEndpointBindings.ReadRabbitManagementEndpoint(configuration);
+        var rabbitUser = Required(configuration, ComparisonHostConstants.RabbitUser);
+        var rabbitPassword = Required(configuration, ComparisonHostConstants.RabbitPassword);
+        ComparisonEndpointBindings.ValidateRabbitCredentials(rabbitUser, rabbitPassword);
         var rabbitImage = Required(configuration, ComparisonHostConstants.RabbitImage);
         var redisConnection = Required(configuration, ComparisonHostConstants.RedisConnection);
         var redisImage = Required(configuration, ComparisonHostConstants.RedisImage);
@@ -69,14 +83,17 @@ internal static ComparisonHostSettings Read(IConfiguration configuration)
         var output = Path.GetFullPath(Required(configuration, ComparisonHostConstants.Output));
         var sourceRevision = configuration[ComparisonHostConstants.SourceRevision];
         return new(options, Guid.NewGuid().ToString(ComparisonHostConstants.GuidFormat), keyLoadEndpoint,
-            qdrantEndpoint, qdrantApiKey, neo4jEndpoint, neo4jPassword, adminKey, postgresConnection,
-            rabbitConnection, redisConnection, postgresImage, qdrantImage, rabbitImage, redisImage, neo4jImage,
+            keyLoadEndpoints, qdrantEndpoint, qdrantApiKey, neo4jEndpoint, neo4jPassword, adminKey, postgresConnection,
+            rabbitConnection, rabbitManagementEndpoint, rabbitUser, rabbitPassword, redisConnection,
+            postgresImage, qdrantImage, rabbitImage, redisImage, neo4jImage,
             output, storage, sourceRevision);
     }
 
     private static Uri ReadEndpoint(IConfiguration configuration, string key)
         => new(Required(configuration, key), UriKind.Absolute);
 
-    private static string Required(IConfiguration configuration, string key)
+    internal static string RequiredValue(IConfiguration configuration, string key)
         => configuration[key] ?? throw new InvalidOperationException(ComparisonHostConstants.MissingSettingPrefix + key);
+
+    private static string Required(IConfiguration configuration, string key) => RequiredValue(configuration, key);
 }