Skip to content

Add end-to-end workflow correlation - #194

Merged
chullybun merged 3 commits into
mainfrom
feature/workflow-correlation
Nov 9, 2026
Merged

chullybun merged 3 commits into
mainfrom
feature/workflow-correlation

Conversation

@chullybun

Copy link
Copy Markdown
Collaborator

Summary

  • Capture optional X-Correlation-Id into ExecutionContext at ASP.NET Core and ASP.NET-integrated Functions boundaries, validate supplied tokens, and echo the effective value on responses.
  • Propagate correlation through outgoing HTTP requests, newly created events and Service Bus receive processing, preserving explicit overrides, clearing and persisted absence.
  • Add dynamic logging scopes and the custom correlation.id activity attribute without changing trace identity or baggage; omit correlation on batch publishing spans.
  • Keep current-request correlation authoritative during idempotency replay while honoring explicit header configuration.
  • Enable appropriate middleware ordering and logging scopes in host templates and samples; update documentation and the v4.4.0 changelog.

Verification

  • CoreEx correlation tests: 24 passed; correlation plus HTTP request extension tests: 29 passed.
  • ASP.NET correlation/idempotency regressions: 23 passed; final HTTP helper relocation checks: 20 passed.
  • Functions HTTP/raw callback/idempotency regressions: 35 passed; final HTTP helper relocation checks: 15 passed.
  • Events full unit suite: 230 passed; Service Bus correlation receive tests: 11 passed.
  • Functions Aspire runtime test project compiled successfully, with the existing ASPIRE010 warning.
  • Real Functions/Aspire runtime execution remains deferred because of local Podman startup failures.
  • Subsequent template/sample changes were statically inspected only; builds, related tests and template-pack validation were explicitly deferred to CI.

Propagate optional correlation across HTTP, execution contexts, events and Service Bus processing. Add dynamic logging scopes and tracing enrichment, preserve current correlation during idempotency replay, and enable host defaults in templates and samples.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Copilot AI balanced review requested due to automatic review settings November 8, 2026 21:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved correlation-handling defects can corrupt scoped span tags, skip broker correlation in typed subscribers, misattribute exported logs, and attach unrelated workflow IDs to nested publishing spans.

4 open issues
What changed in this PR

Adds optional workflow correlation across CoreEx HTTP, Functions and messaging boundaries, connecting related operations without changing trace identity or baggage.

Changes:

  • Adds validated correlation capture, response echo, outgoing HTTP propagation and event defaults, while preserving explicit overrides and stored absence.
  • Adds logging scopes and tracing attributes, plus current-request correlation handling during idempotency replay.
  • Updates templates, samples and documentation, and adds correlation regression tests. Real Functions runtime execution and template/sample validation remain deferred as described in the PR.
File Description
tests/​CoreEx.Test.Unit/​GlobalUsing.cs Adds diagnostic and logging test imports.
tests/​CoreEx.Test.Unit/​CoreEx.Test.Unit.csproj Adds console logging test dependency.
tests/​CoreEx.Events.Test.Unit/​CorrelationEventTests.cs Tests event defaults, reconstruction and publishing spans.
tests/​CoreEx.Azure.Messaging.ServiceBus.Test.Unit/​CorrelationReceiveTests.cs Tests receive capture, clearing and restoration.
tests/​CoreEx.Azure.Functions.TestApp/​ServiceBusTraceProbe.cs Records correlation during message processing.
tests/​CoreEx.Azure.Functions.TestApp/​NativeMessageSubscriber.cs Captures processing-boundary observations.
tests/​CoreEx.Azure.Functions.TestApp/​HttpFunctions.cs Supports response override and clearing scenarios.
tests/​CoreEx.Azure.Functions.Test.Unit/​HttpFunctionsTests.cs Tests deferred response correlation and invalid headers.
tests/​CoreEx.Azure.Functions.Test.Unit/​GlobalUsing.cs Adds logging imports.
tests/​CoreEx.Azure.Functions.Test.Unit/​FunctionsServiceBusExtensionsTests.cs Tests raw callback correlation and settlement.
tests/​CoreEx.AspNetCore.Test.Unit/​CorrelationHostTests.cs Tests HTTP capture, tracing, errors and replay.
tests/​aspire/​CoreEx.Azure.Functions.Test.Aspire/​FunctionsRuntimeTests.cs Adds real-host correlation assertions.
src/​CoreEx/​README.md Documents workflow correlation semantics.
src/​CoreEx/​Invokers/​InvokerTracer.cs Adds correlation to invoker activities.
src/​CoreEx/​Http/​README.md Documents outgoing HTTP propagation.
src/​CoreEx/​Http/​HttpRequestMessageExtensions.cs Adds request-level override and suppression.
src/​CoreEx/​Http/​HttpRequestBuilder.cs Adds fluent correlation configuration.
src/​CoreEx/​Http/​HttpNames.cs Defines the shared correlation header.
src/​CoreEx/​Http/​CorrelationIdHandler.cs Propagates and validates outgoing correlation.
src/​CoreEx/​GlobalUsing.cs Imports correlation diagnostics.
src/​CoreEx/​ExecutionContext.Scoped.cs Opens logging scopes for scoped execution.
src/​CoreEx/​ExecutionContext.cs Adds validated correlation state and copying.
src/​CoreEx/​Diagnostics/​Correlation.cs Adds validation, logging and receive scopes.
src/​CoreEx/​CoreExExtensions.OpenTelemetry.cs Tags HTTP spans from outgoing headers.
src/​CoreEx/​CoreExExtensions.DependencyInjection.cs Adds HTTP handler registration helper.
src/​CoreEx/​CoreExExtensions.ApplicationBuilder.cs Registers correlation on CoreEx HTTP clients.
src/​CoreEx/​AGENTS.md Documents correlation usage and boundaries.
src/​CoreEx.Template/​README.md Describes template correlation defaults.
src/​CoreEx.Template/​content/​CoreEx.Subscribe/​src/​app-name.Subscribe/​Program.cs Updates middleware order and logging options.
src/​CoreEx.Template/​content/​CoreEx.Subscribe/​src/​app-name.Subscribe/​appsettings.json Enables console scopes.
src/​CoreEx.Template/​content/​CoreEx.Subscribe/​src/​app-name.Subscribe/​AGENTS.md Documents subscriber correlation behavior.
src/​CoreEx.Template/​content/​CoreEx.Relay/​src/​app-name.Relay/​Program.cs Updates middleware order and logging options.
src/​CoreEx.Template/​content/​CoreEx.Relay/​src/​app-name.Relay/​appsettings.json Enables console scopes.
src/​CoreEx.Template/​content/​CoreEx.Relay/​src/​app-name.Relay/​AGENTS.md Documents stored correlation preservation.
src/​CoreEx.Template/​content/​CoreEx.Functions/​src/​app-name.Functions/​Program.cs Enables OpenTelemetry logging scopes.
src/​CoreEx.Template/​content/​CoreEx.Functions/​src/​app-name.Functions/​appsettings.json Enables console scopes.
src/​CoreEx.Template/​content/​CoreEx.Functions/​src/​app-name.Functions/​AGENTS.md Documents invocation coverage and limitations.
src/​CoreEx.Template/​content/​CoreEx.Api/​src/​app-name.Api/​Program.cs Updates middleware order and logging options.
src/​CoreEx.Template/​content/​CoreEx.Api/​src/​app-name.Api/​appsettings.json Enables console scopes.
src/​CoreEx.Template/​content/​CoreEx.Api/​src/​app-name.Api/​AGENTS.md Documents API correlation defaults.
src/​CoreEx.Events/​Subscribing/​EventSubscriberBase.cs Imports envelope correlation within receive scopes.
src/​CoreEx.Events/​README.md Documents event defaults and receive semantics.
src/​CoreEx.Events/​Publishing/​EventPublisherBase.cs Clears correlation on the outer publishing span.
src/​CoreEx.Events/​GlobalUsing.cs Imports correlation diagnostics.
src/​CoreEx.Events/​EventFormatter.cs Preserves absent correlation during reconstruction.
src/​CoreEx.Events/​EventData.With.cs Adds fluent correlation override and clearing.
src/​CoreEx.Events/​EventData.cs Snapshots ambient correlation and preserves JSON absence.
src/​CoreEx.Events/​AGENTS.md Documents event correlation invariants.
src/​CoreEx.Azure.Messaging.ServiceBus/​ServiceBusSubscriberBase.cs Adds transport-level correlation scopes.
src/​CoreEx.Azure.Messaging.ServiceBus/​README.md Documents received correlation policy.
src/​CoreEx.Azure.Messaging.ServiceBus/​GlobalUsing.cs Imports correlation diagnostics.
src/​CoreEx.Azure.Messaging.ServiceBus/​AGENTS.md Records correlation ownership and validation rules.
src/​CoreEx.Azure.Functions/​README.md Documents invocation, receive and replay behavior.
src/​CoreEx.Azure.Functions/​GlobalUsing.cs Adds correlation and logging imports.
src/​CoreEx.Azure.Functions/​FunctionsServiceBusReceiveArgs.cs Adds logging and correlation-ownership options.
src/​CoreEx.Azure.Functions/​FunctionsServiceBusExtensions.cs Captures raw callback correlation through settlement.
src/​CoreEx.Azure.Functions/​FunctionsExecutionContextMiddleware.cs Adds HTTP capture, response echo and diagnostics.
src/​CoreEx.Azure.Functions/​AGENTS.md Documents supported correlation boundaries.
src/​CoreEx.AspNetCore/​README.md Documents HTTP correlation and middleware ordering.
src/​CoreEx.AspNetCore/​Idempotency/​IdempotencyKey.cs Clarifies replay comments and simplifies braces.
src/​CoreEx.AspNetCore/​Idempotency/​HybridCacheIdempotencyProvider.cs Excludes correlation from cached headers by default.
src/​CoreEx.AspNetCore/​Http/​HttpCorrelation.cs Shares HTTP validation and response-header handling.
src/​CoreEx.AspNetCore/​GlobalUsing.cs Adds correlation imports and reorders declarations.
src/​CoreEx.AspNetCore/​ExecutionContextMiddleware.cs Captures correlation and enriches request diagnostics.
src/​CoreEx.AspNetCore/​ExceptionHandlingMiddleware.cs Normalizes the final newline only.
src/​CoreEx.AspNetCore/​AGENTS.md Adds HTTP correlation and ordering guidance.
samples/​src/​Contoso.Shopping.Subscribe/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Shopping.Subscribe/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Shopping.Relay/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Shopping.Relay/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Shopping.Api/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Shopping.Api/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Products.Subscribe/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Products.Subscribe/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Products.Relay/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Products.Relay/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Products.Api/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Products.Api/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Orders.Api/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Orders.Api/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Customers.Relay/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Customers.Relay/​appsettings.json Enables console scopes.
samples/​src/​Contoso.Customers.Api/​Program.cs Updates middleware order and logging options.
samples/​src/​Contoso.Customers.Api/​appsettings.json Enables console scopes.
samples/​docs/​hosts-layer.md Explains host correlation and revised ordering.
CHANGELOG.md Records correlation capabilities and adoption requirements.
.github/​instructions/​coreex-host-setup.instructions.md Updates host guidance and middleware examples.
.github/​instructions/​coreex-functions.instructions.md Adds Functions correlation and settlement guidance.

Review effort: Balanced

Comment thread src/CoreEx/Diagnostics/Correlation.cs Resolved
Comment thread src/CoreEx/ExecutionContext.Scoped.cs Resolved
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 15:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Native subscribers still invoke CloudEvent detection when correlation is disabled, allowing malformed content types to throw before native deserialization and error classification.

1 open issue
4 resolved since last review

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 16:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Explicitly configured structured receives can incorrectly restore the broker correlation ID over the authoritative envelope value on the completed receive span; scope ownership needs correction and regression coverage.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Use configured CloudEvent correlation ownership for structured receives

src/​CoreEx.Azure.Messaging.ServiceBus/​ServiceBusSubscriberBase.cs:46

With ContentMode = ContentMode.Structured, a valid envelope without ce_specversion and with application/json or no content type can decode successfully while IsCloudEvent() returns false. This creates an outer broker-correlation scope, so disposing the inner envelope scope restores the broker's correlation.id instead of retaining the authoritative envelope value, including absence, on the completed receive span. Choose ownership solely from UsesCloudEventCorrelation, and add a regression asserting the final activity tag for an explicitly configured structured receive.

1 resolved since last review

Review effort: Balanced

@chullybun
chullybun merged commit 52ba023 into main Nov 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants