perf(discovery): pipeline MMS model discovery with smart enrichment

.github/workflows/dotnet-ci.yml

@@ -66,6 +66,51 @@ jobs:
             Tee-Object -FilePath artifacts/test-results/dotnet-test.log
           if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
 
+      - name: Package test build
+        if: success()
+        shell: pwsh
+        run: |
+          $root = "artifacts/ariec61850-test-build"
+          $binary = Join-Path $root "lib/net8.0"
+          $nuget = Join-Path $root "nuget"
+          $builtBinary = ".\source\.artifacts\bin\AR.Iec61850\Release\net8.0"
+          New-Item -ItemType Directory -Path $binary -Force | Out-Null
+          New-Item -ItemType Directory -Path $nuget -Force | Out-Null
+
+          if (-not (Test-Path $builtBinary)) {
+            throw "Expected Release output was not found at $builtBinary"
+          }
+          Copy-Item (Join-Path $builtBinary "*") $binary -Recurse -Force
+
+          $packageVersion = "0.0.0-pr134.$env:GITHUB_RUN_NUMBER"
+          dotnet pack .\source\src\AR.Iec61850\AR.Iec61850.csproj `
+            -c Release `
+            --no-build `
+            -o $nuget `
+            -p:PackageVersion=$packageVersion
+          if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+          @(
+            "ARIEC61850 PR #134 smart-discovery test build",
+            "Commit: $env:GITHUB_SHA",
+            "Branch: $env:GITHUB_HEAD_REF",
+            "Run: $env:GITHUB_RUN_NUMBER",
+            "TargetFramework: net8.0",
+            "NuGetVersion: $packageVersion",
+            "",
+            "lib/net8.0 contains the direct Release build for DLL-reference testing.",
+            "nuget contains the installable local .nupkg for PackageReference testing."
+          ) | Set-Content -Path (Join-Path $root "BUILD.txt") -Encoding utf8
+
+      - name: Upload PR test build
+        if: success()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: ariec61850-pr134-test-build
+          path: artifacts/ariec61850-test-build
+          if-no-files-found: error
+          retention-days: 14
+
       - name: Upload test diagnostics
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7

CHANGELOG.md

@@ -6,6 +6,7 @@ All notable public changes to ARIEC61850 are recorded here. The project is still
 
 ### Added
 
+- Added zero-traffic smart-discovery KPI evidence across structural `GetNameList`, DataSet directory, hierarchy GVA, and bounded FC-root initial Read phases, including per-phase latency, duplicate semantic-request detection, peak outstanding requests, canonical completeness counters, deterministic repeat-run signatures, and explicit partial wire-accounting status for unobserved enrichment paths.
 - Added live-authoritative SCL ReportControl family reconciliation for static reporting: indexed declarative controls are matched only to concrete live MMS RCB instances in the same domain/logical-node/report-FC with decimal instance suffixes, while non-indexed controls remain exact-name only and no runtime RCB name is synthesized.
 - Added an SCL-aware static report planner that scopes the existing safe RCB selector to the reconciled live family, allowing a busy first instance to fall through to another proven-free instance without escaping to an unrelated control block.
 - Added registered-monitor initial GI bootstrap for persistent reporting: the monitor is installed before the one-shot GI request, initial mapped DataSet reports are returned explicitly, and GI is attempted only when the live RCB attribute inventory proves the field exists.

docs/SMART_DISCOVERY_KPI.md

@@ -0,0 +1,140 @@
+# Smart Discovery KPI / Capture Convergence
+
+Status: **implemented + unit tested; live capture convergence not yet re-validated after this patch**.
+
+This document defines the P0-4 observability contract for smart IEC 61850 discovery. The KPI layer is diagnostic evidence only. It does not create a second IEC 61850 model, does not alter canonical semantics, and does not add MMS traffic.
+
+## Architecture
+
+```text
+existing smart discovery requests
+        |
+        +-- GetNameList
+        +-- GetNamedVariableListAttributes
+        +-- GetVariableAccessAttributes
+        `-- bounded FC-root Read
+        |
+        v
+existing confirmed-service execution
+        |
+        +------------------------+
+        |                        |
+        v                        v
+canonical discovery/model   zero-traffic KPI recorder
+                             counts / latency / duplicate keys
+                             peak outstanding / completeness
+                             deterministic signature
+```
+
+The canonical IED model remains the only semantic source of truth. KPI fields are an evidence overlay and are never used to manufacture LD/LN/DO/DA, DataSet, RCB, value, type, or SCL semantics.
+
+## Zero-traffic invariant
+
+KPI collection must not issue any additional:
+
+- `GetNameList`;
+- `GetVariableAccessAttributes`;
+- `Read`;
+- `GetNamedVariableListAttributes`;
+- association request;
+- report/control request;
+- transport probe.
+
+Observations are opened only around confirmed requests that the smart path already intends to send.
+
+## Observed phases
+
+The optimized default path records these wire request classes:
+
+| Phase | MMS service | Logical duplicate key |
+|---|---|---|
+| `structure` | `GetNameList` | object class + domain/VMD + continuation token |
+| `dataset-directory` | `GetNamedVariableListAttributes` | exact DataSet reference |
+| `type-enrichment` | `GetVariableAccessAttributes` | exact domain/item reference |
+| `initial-read` | `Read` | ordered FC-root reference batch |
+
+A repeated normalized logical key in one generation increments `DuplicateRequests`. Legitimate GetNameList continuation pages are distinct because the continuation token is part of the key.
+
+## Snapshot contract
+
+`MmsClientSession.LastSmartDiscoveryKpi` exposes a point-in-time `MmsSmartDiscoveryKpiSnapshot` containing:
+
+- total / successful / failed observed confirmed requests;
+- duplicate request count;
+- peak outstanding request count;
+- per-phase request count and latency statistics;
+- LD, LN, raw-variable and FC-point completeness;
+- DataSet / DataSet-directory / ordered member completeness;
+- RCB / BRCB / URCB counts;
+- `WireAccountingComplete` plus explicit `AccountingNotes`;
+- deterministic signature.
+
+`RefreshSmartDiscoveryModelKpi(directory)` can refresh LD/LN/FC-point completeness after later semantic materialization. It is local bookkeeping and performs no network I/O.
+
+## Deterministic signature
+
+The signature intentionally excludes:
+
+- wall-clock timestamps;
+- measured request latency;
+- worker completion order;
+- invoke ID allocation order.
+
+It includes normalized request identity/attempt outcomes, accounting coverage, and semantic completeness counts. Equivalent evidence should therefore produce the same signature even when concurrent workers finish in a different order.
+
+This signature is a convergence diagnostic, not an IEC 61850 semantic fingerprint and not an SCL identity.
+
+## Accounting coverage
+
+The default optimized smart-discovery path keeps report attribute probing off the structural critical path. For that path, all currently instrumented smart request loops can report complete wire accounting.
+
+If `ProbeReportAttributes=true` and report controls are eligible for probing, the current legacy report-enrichment Read fallback path is not yet observed at individual wire-attempt granularity. The snapshot therefore sets:
+
+```text
+WireAccountingComplete = false
+```
+
+and adds an explicit accounting note. This prevents an undercount from being presented as complete evidence.
+
+## Current controlled convergence target
+
+For the current controlled regression device/session, the project-level target supplied to P0 is:
+
+```text
+LD                    32
+LN                   119
+semantic leaves     4,925
+DataSet                2
+ordered FCDA           58
+logical ReportControl  32
+```
+
+These values are capture-specific regression evidence, not IEC 61850 limits and must never be hard-coded into protocol behavior.
+
+P0-4 live acceptance requires a new controlled run to demonstrate, for the same evidence set:
+
+1. expected canonical completeness is retained;
+2. `DuplicateRequests == 0` unless a documented retry/fallback is expected;
+3. `WireAccountingComplete == true` for the optimized default path;
+4. repeated runs yield the same deterministic signature when semantic evidence and outcomes are unchanged;
+5. request count and per-phase latency are recorded without additional MMS traffic;
+6. DataSet member order and RCB semantics remain unchanged.
+
+## What is validated now
+
+- **Implemented:** zero-traffic KPI recorder and smart call-site observations.
+- **Unit tested:** deterministic signature independent of completion order; duplicate detection; peak outstanding count; model-completeness refresh; explicit partial accounting.
+- **CI validated:** build/test/source-verification must pass on the exact PR head before this phase is called code-complete.
+- **Not yet laboratory re-validated:** final request counts, latency distribution, duplicate count, and canonical completeness against a fresh controlled packet capture after this patch.
+
+## Next phase: P0-5 Request Budget & Adaptive Convergence
+
+P0-5 may consume P0-4 evidence to reduce redundant requests and tune bounded concurrency. It must not optimize from guesses. Any request removal or concurrency change must preserve canonical completeness, DataSet ordering, report/control safety, deterministic publication order, and negotiated `maxOutstandingCalling` limits.
+
+P0-5 candidate gates:
+
+- establish per-phase request budget from controlled repeat runs;
+- identify only proven duplicate/redundant semantic requests;
+- adapt window conservatively from negotiated peer limits and observed latency/failure evidence;
+- compare before/after request count and wall time with identical canonical signature/completeness;
+- fall back to the conservative path when evidence is insufficient.

docs/SMART_MMS_DISCOVERY.md

@@ -0,0 +1,137 @@
+# Smart MMS discovery
+
+`MmsClientSession.DiscoverSmartAsync` is the capture-informed discovery path for fast, deterministic IEC 61850 model construction. It follows the repository's canonical contract: live MMS provides online evidence, SCL may prioritize and validate that evidence, and both feed the same canonical model rather than separate semantic trees.
+
+## Design invariants
+
+- One TCP/COTP/MMS association and one receive pump per session.
+- Confirmed responses are correlated by invoke ID; discovery never starts a second raw receive loop.
+- The VMD domain list is enumerated once per smart discovery operation.
+- Independent `(domain, object-class)` GetNameList chains may be outstanding concurrently.
+- Continuation pages inside one GetNameList chain are always sequential.
+- The effective discovery window is capped by the peer's negotiated `maxOutstandingCalling` when available. Unknown peers use a conservative fallback cap.
+- A fixed-size worker pool is used instead of creating one Task per domain/service pair, keeping scheduling and allocation bounded on large IEDs.
+- Published dictionaries are rebuilt from live evidence in deterministic domain order, so concurrent completion and SCL priority hints never change model semantics.
+- Valid evidence from completed pages/chains is retained when a later page or another chain fails.
+- Expected receive-pump/association loss during concurrent work becomes a partial result instead of an unhandled `Task.WhenAll` failure.
+- Initial smart discovery does not perform an eager per-leaf Read or GetVariableAccessAttributes sweep.
+- Report attribute reads and DataSet directory reads are deferred by default because they are enrichment, not prerequisites for the first usable LD/LN/DO/DA tree.
+
+## Bounded pagination
+
+The smart pager is intentionally separate from the compatibility pager. It uses an O(n) `HashSet` boundary de-duplication path and has explicit guards for:
+
+- maximum pages per GetNameList chain;
+- maximum names per chain;
+- no-new-name pages while `moreFollows` remains true;
+- empty continuation tokens;
+- repeated continuation tokens/cycles.
+
+If a guard fires, already-decoded names remain available and the chain is marked incomplete. The engine does not keep asking the same IED question indefinitely.
+
+## Type enrichment
+
+Canonical type enrichment is planned by `LiveIedVariableTypeProbePlanner` and executed by `LiveIedVariableTypeProbeExecutor.ProbeSmartAsync`.
+
+The adaptive ladder is:
+
+1. Probe each MMS logical-node root (`LN`) once.
+2. Use the returned `TypeSpecification` tree to prove every descendant path that is actually present in that hierarchy.
+3. Only unresolved branches fall back to distinct `LN$FC$DO` roots.
+4. Only descendants still unresolved after the DO-root result fall back to exact leaf GVA.
+
+This is deliberately coverage-aware: a successful but shallow parent response does not suppress required child probes. Normal structured IEC 61850 servers therefore approach one GVA per logical node, while unusual servers still retain an exact fallback path.
+
+Type-tree projection into the canonical live model is indexed by MMS Domain/LogicalNode before results are mapped. A type result therefore scans only the relevant LN instead of repeatedly scanning every point in the IED.
+
+## Initial value reads
+
+Initial values use the existing canonical/SCL/live `InitialFcReadPlan`; the smart path does not invent a second read model. The planner groups leaf points into FC roots such as `XCBR1$ST` or `MMXU1$MX` and batches up to the bounded MMS request limit.
+
+`ExecuteInitialFcReadPlanSmartAsync` changes only the scheduling layer:
+
+- default maximum four outstanding Read batches when association limits are known;
+- conservative two-batch window when `maxOutstandingCalling` is unknown;
+- always capped by the negotiated association limit;
+- fixed worker count instead of Task-per-batch;
+- invoke-ID response correlation remains authoritative;
+- per-batch timeout remains explicit;
+- one timeout/transport fault resets the association once so late responses cannot become stale evidence;
+- completed batches and projections are preserved when another batch fails;
+- no automatic retry/split storm and no fallback to thousands of leaf Reads.
+
+Example:
+
+```csharp
+var initialPlan = InitialFcReadPlanner.FromLiveDirectory(discovery.IedDirectory);
+var initialValues = await session.ExecuteInitialFcReadPlanSmartAsync(
+    initialPlan,
+    new MmsSmartInitialFcReadOptions
+    {
+        MaxOutstandingBatches = 4
+    },
+    cancellationToken);
+```
+
+For an opened SCL model, use the canonical SCL initial-read planner already provided by the engine and execute that same plan through the smart executor.
+
+## SCL-assisted scheduling
+
+SCL is a scheduling/validation hint, never a substitute for online evidence. `MmsSmartDiscoveryOptions.PriorityDomains` can be populated from the exact expected MMS domains of the trusted SCL IED/AP/Server selection. The engine still performs live VMD `GetNameList`, keeps extra live domains, and publishes the live-selected domain set unchanged.
+
+```csharp
+var options = new MmsSmartDiscoveryOptions
+{
+    MaxConcurrentChains = 8,
+    // Optional, when trusted SCL context is already selected:
+    PriorityDomains = sclDomainInventory?.ExpectedDomains ?? Array.Empty<string>()
+};
+
+var discovery = await session.DiscoverSmartAsync(options, cancellationToken);
+
+var exactTypes = await LiveIedVariableTypeProbeExecutor.ProbeSmartAsync(
+    session,
+    discovery.IedDirectory,
+    options,
+    cancellationToken);
+```
+
+Expensive runtime enrichment can be requested explicitly when needed:
+
+```csharp
+var deepOptions = new MmsSmartDiscoveryOptions
+{
+    MaxConcurrentChains = 8,
+    ProbeReportAttributes = true,
+    ReadDataSetDirectories = true
+};
+```
+
+The legacy `DiscoverAsync`, `GetVariableAccessAttributesBatchAsync`, and sequential `ExecuteInitialFcReadPlanAsync` APIs remain unchanged for compatibility. Consumers can migrate deliberately and compare model completeness before making smart paths their default.
+
+## Transport safety
+
+Pipelining requires multiple confirmed requests to be outstanding. `TpktClient` therefore serializes writers around each complete TPKT frame. The frame is allocated inside that single-writer gate, keeping peak outbound-frame allocation bounded while still allowing multiple request/response lifecycles to remain outstanding. The existing single receive pump remains the only association reader.
+
+## Consumer anti-lag rule
+
+The engine smart methods use asynchronous I/O and `ConfigureAwait(false)` internally, so MMS waits, response decoding, and canonical model materialization do not require the caller's UI synchronization context. A UI consumer should publish immutable snapshots/results back to the UI in coarse batches rather than adding thousands of tree nodes one by one. Discovery/type/value phases are intentionally separate so the first structural model can be shown before optional metadata/value enrichment finishes.
+
+## Capture-informed target
+
+The reference capture used during this refactor showed the existing consumer issuing roughly 30.7k confirmed MMS requests, including roughly 23.7k GetVariableAccessAttributes and 6.6k Read requests, while the comparison tool used a much smaller, pipelined request set. These values are benchmark evidence, not protocol requirements. The smart path targets the independently observed scheduling pattern—single association, bounded outstanding requests, structural discovery first, hierarchy-aware metadata, FC-root selective reads—without copying vendor code or vendor-specific implementation details.
+
+For acceptance, compare the same IED and capture conditions using:
+
+- time to first visible model item;
+- time to usable LD/LN/DO/DA tree;
+- final LD/LN/FC-point and dataset counts;
+- confirmed request count by service;
+- peak outstanding confirmed requests;
+- failed/partial domain chains;
+- exact type coverage after smart enrichment;
+- FC-root initial-value coverage and Read request count;
+- managed allocations and task count during discovery;
+- UI-thread stalls / long frames in the consuming application.
+
+A performance result is accepted only when the final model remains semantically equivalent for the required scope.