Skip to content

fix(eventhubs): safely clean up uAMQP producer sessions after discovery failure - #7511

Open
Johnathan W (j7nw4r) wants to merge 9 commits into
mainfrom
fix/eventhubs-discovery-cleanup
Open

Johnathan W (j7nw4r) wants to merge 9 commits into
mainfrom
fix/eventhubs-discovery-cleanup

Conversation

@j7nw4r

@j7nw4r Johnathan W (j7nw4r) commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Summary

Fix Event Hubs producer cleanup after partition discovery fails with uAMQP, so releasing the producer does not abort the process.

cleanup
  cancel the active uAMQP properties query and wait for its lock
  release management/link ownership
  end an acquired session, even when management creation failed
  release native resources and polling ownership at final destruction after failed END

Track session initialization independently of management-client creation. Explicit close and destruction share the cleanup path, repeated initialization failures do not acquire duplicate polling references, and both session End overloads preserve shutdown errors while permitting retry. Successful shutdown and the assertion for omitted shutdown remain supported.

Properties-query synchronization, child-context cancellation, and stale-reference rejection apply to uAMQP. Rust retains its initialization-only lock; Rust cancellation, recovery, and concurrent cleanup are deferred to azure-sdk-for-rust#5527. Public API signatures are unchanged.

Refresh the two Blob Checkpoint Store recordings for current Storage SDK request headers and Arrow listing responses.

Evidence

  • Before: Failure after session BEGIN could skip session cleanup; failed native END could leave polling ownership unreleased and trigger Session was not ended before destruction. Concurrent uAMQP query/cleanup could release resources still in use.
  • After: All 20 uAMQP session, properties, and concurrent-cleanup regression cases pass locally on Linux ARM64. Another 63 offline Event Hubs tests pass with each backend.
  • Rust Core AMQP and Event Hubs test targets, and the uAMQP Event Hubs test target, build with warnings treated as errors. The uAMQP Core test build needs -Wno-error=unused-result for an existing warning in unchanged message_sender_receiver.cpp:929.
  • Spelling passes for all 14 text files changed by the PR; formatting and diff checks pass. Windows/macOS CI and live-service validation remain separate.

Merge Danger

Door: Two-way. The change can be reverted without a data migration or public API change.

Blast Radius: AMQP. Changes affect discovery cleanup, uAMQP session ownership, and uAMQP properties-query synchronization. Rust management cancellation and bounded session teardown remain deferred.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@j7nw4r

Copy link
Copy Markdown
Member Author

/azp run cpp - event hubs

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

@j7nw4r

Copy link
Copy Markdown
Member Author

/azp run cpp - eventhubs

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Management-client cleanup can race with concurrent properties queries during gateway recovery, causing a null dereference or use-after-free.

1 open issue
What changed in this PR

This PR improves Event Hubs producer cleanup after partition discovery fails, with supporting changes to shared uAMQP session lifetime handling. Public API signatures remain unchanged.

Changes:

  • Track session initialization separately from management-client creation and unify explicit close with destructor cleanup.
  • Defer native cleanup after failed session shutdown until the last owner releases it.
  • Add cleanup regression tests and update the Event Hubs recording asset reference.
File Description
sdk/​eventhubs/​azure-messaging-eventhubs/​test/​ut/​properties_cleanup_test.cpp Tests discovery failures, partial initialization, and repeated close.
sdk/​eventhubs/​azure-messaging-eventhubs/​test/​ut/​CMakeLists.txt Registers cleanup tests and their uAMQP dependencies.
sdk/​eventhubs/​azure-messaging-eventhubs/​src/​private/​eventhubs_utilities.hpp Tracks session ownership and consolidates properties-client cleanup.
sdk/​eventhubs/​azure-messaging-eventhubs/​CHANGELOG.md Documents the producer cleanup fix.
sdk/​eventhubs/​assets.json Updates the recording asset reference.
sdk/​core/​azure-core-amqp/​test/​ut/​session_cleanup_tests.cpp Tests failed shutdown, shared ownership, retries, and missing shutdown.
sdk/​core/​azure-core-amqp/​test/​ut/​CMakeLists.txt Registers session cleanup tests and dependencies.
sdk/​core/​azure-core-amqp/​test/​ut/​cleanup_test_listener.hpp Provides a minimal TCP listener for cleanup tests.
sdk/​core/​azure-core-amqp/​src/​impl/​uamqp/​amqp/​session.cpp Defers cleanup after failed shutdown and shares shutdown logic.
sdk/​core/​azure-core-amqp/​src/​impl/​uamqp/​amqp/​private/​session_impl.hpp Adds shutdown-failure state and a test injection point.
sdk/​core/​azure-core-amqp/​inc/​azure/​core/​amqp/​internal/​session.hpp Documents ownership behavior after failed shutdown.
sdk/​core/​azure-core-amqp/​CHANGELOG.md Documents the uAMQP session cleanup fix.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The default Rust backend ignores management-call cancellation, allowing a withheld response to keep the mutex locked and block ConsumerClient::Close() even after the close context expires.

1 open issue
1 resolved since last review

Review effort: Balanced

Bridge the operation context into the synchronous Rust wait so cancellation releases the properties mutex before close. Reject null results and replace failed management links before another query. Bound consuming management teardown and add real loopback regressions for both getters and public consumer close.
Revert c86152e and keep the properties-query lock and child-context cancellation specific to uAMQP. Rust keeps its initialization-only lock until management cancellation is implemented.

Track the Rust implementation, recovery behavior, and regression tests in Azure/azure-sdk-for-rust#5527. Preserve the uAMQP discovery/session cleanup fix and checkpoint recordings.
@j7nw4r Johnathan W (j7nw4r) changed the title fix(eventhubs): safely clean up producer sessions after discovery failure fix(eventhubs): safely clean up uAMQP producer sessions after discovery failure Oct 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants