fix(agentserver): isolate live response runtime state

[Shanmukha Pasumarthy (shanmukha1200)]

Summary

• Partition live execution, pending, deletion, and history state by the platform user key.
• Partition SSE streams, resilient task IDs, multi-turn input/precondition IDs, and process-local recovery references by user without changing public response IDs.
• Allow different users to execute the same public response ID independently; reject duplicate active or retained-replay IDs within the same user partition with 409 response_id_conflict (not POST idempotency).
• Require provider authorization before persisted SSE replay; not-found and provider errors never fall through to cached events.
• Preserve reservations until request cleanup, including store=False requests before record publication.
• Reconstruct stream ownership from durable user context during recovery.

Compatibility and companion fix

Persistent provider partitioning is implemented separately in #49025. This PR requires a provider that enforces the supplied PlatformContext; it does not independently fix the existing built-in persistent providers. Deploy both fixes for complete built-in provider and runtime isolation.

Identified users do not adopt legacy shared task/replay state. Drain active tasks and retained replay state before upgrading. Anonymous identifier format is preserved. Identity headers must come from trusted platform authentication.

Validation

• 1,574 tests passed, 6 skipped across unit, contract, integration, conformance, and recovery reconstruction suites, including a rerun after rebasing onto current main.
• Coverage includes concurrent same-ID user streams, cancellation/deletion independence, replay after eviction, persisted-owner versus foreign-live access, reservation-only creates, fail-closed provider errors, file replay rehydration, and reconstructed streaming tasks with durable user identity.
• Pylint, MyPy, Black, and diff whitespace checks passed.
• Bounded security review found no new vulnerabilities in the lifecycle changes; this is not hosted end-to-end or actual process-crash verification.

Restart-safe retained replay

• Core stream lookup now restores existing persisted logs under the same per-ID lock used for creation/deletion, without creating absent replay files. Cold deletion also removes retained logs.
• Same-user POST collisions still return 409 after restart; authorized GET replay and DELETE use the same scoped lookup. Missing, expired, locked, and inaccessible log cases have regression coverage.
• Core is versioned at 2.2.1 (unreleased), and the Responses dependency requires that version so this fix cannot silently run with the old registry.
• Additional validation: 1,581 Responses tests passed (7 skipped), 123 Core streaming tests passed (1 skipped), both packages passed Pylint and Linux-platform MyPy, and the bounded cold-replay security review found no new vulnerabilities. Restart tests reconstruct registries and release file handles; they do not simulate a real process crash.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot review overview

🟡 Changes recommended

Retained replay streams and pre-publication reservations can still permit cross-user provider fallback or event disclosure.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 · 1

Open (3)

• Validate stream ownership before replay fallback · New
• Include reservations in ownership checks · New
• Qualify isolation claims to live runtime state · New

What changed in this PR

Partitions live response runtime state by user and prevents duplicate live response IDs.

Changes:

• Adds user-scoped runtime keys and response-ID reservations.
• Propagates user identity through orchestration and endpoints.
• Adds isolation, collision, replay-retention, and race tests.

File	Description
tests/​unit/​test_runtime_state.py	Tests partitioning and reservations.
tests/​contract/​test_user_isolation_enforcement.py	Tests cross-user conflicts and isolation.
tests/​contract/​test_delete_eviction_race.py	Updates race helpers for user keys.
CHANGELOG.md	Documents the runtime isolation fix.
hosting/​_runtime_state.py	Implements partitioned state and reservations.
hosting/​_orchestrator.py	Propagates user keys through runtime operations.
hosting/​_endpoint_handler.py	Enforces reservations and scoped endpoint access.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Security-sensitive multi-user lifecycle and recovery behavior requires final human validation alongside companion PR #49025.

Review effort: Balanced
Findings: 1

Open (1)

• Clarify response_id_conflict covers active execution and retained replay · New

Resolved since last review (3)

• Include reservations in ownership checks
• Validate stream ownership before replay fallback
• Qualify isolation claims to live runtime state

[Copilot]

Copilot review overview

🔵 Needs a closer look

File-backed replay logs are neither detected for duplicate admission nor rehydrated for replay after a real process restart.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Clarify response_id_conflict covers active execution and retained replay

Previously missed (1)

In code that hasn't changed since last review

Restarted registry misses persisted stream conflicts

sdk/​agentserver/​azure-ai-agentserver-responses/​azure/​ai/​agentserver/​responses/​hosting/​_endpoint_handler.py:758

After an actual process restart, the registry's _slots map is empty even when a file-backed replay log still exists. _StreamsRegistry.get() only checks _slots; rehydration occurs in get_or_create(). Therefore this reservation succeeds for an ID whose replay is still retained on disk, and the subsequent stream creation reopens the old closed log instead of returning the promised 409 response_id_conflict. The registry needs a non-creating persisted-stream probe/rehydration API (or equivalent) so admission can distinguish a retained file from a genuinely new ID.

This issue also appears on line 1466 of the same file.

[Copilot]

Copilot review overview

🟡 Changes recommended

Cold-start file replay is neither detected during ID reservation nor rehydrated by the replay endpoint.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Duplicate POST bypasses conflict after restart · New

[Copilot]

Copilot review overview

🟡 Changes recommended

Public Core streaming documentation still describes the previous get() and recovery behavior.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Update streaming docs for cold file-backed log lookup · New

Resolved since last review (1)

• Duplicate POST bypasses conflict after restart

[Copilot]

Copilot review overview

🟡 Changes recommended

Cold deletion can silently leave replay files recoverable, and internal streaming documentation remains inconsistent with scoped lifecycle IDs.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Verify log deletion before committing tombstones · New

Resolved since last review (1)

• Update streaming docs for cold file-backed log lookup

Previously missed (1)

In code that hasn't changed since last review

Update streaming docs for user-scoped lifecycle keys

sdk/​agentserver/​azure-ai-agentserver-responses/​README.md:75

The package's developer-facing streaming/streaming-internals.md:41-49,64-80 still says files are named from the public response_id and recovery calls get_or_create(response_id). For identified users this change instead requires the hashed lifecycle ID, so extension authors following that document will inspect or open the wrong stream. Update that architecture document to describe anonymous compatibility and user-scoped lifecycle keys.

[Copilot]

Copilot review overview

🟡 Changes recommended

Responses needs a new package version, and missing stream lookups currently cause unbounded lock retention.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Bump Responses version after release and regenerate package metadata · New

Resolved since last review (1)

• Verify log deletion before committing tombstones

Previously missed (1)

In code that hasn't changed since last review

Prevent unbounded growth of per-ID locks

sdk/​agentserver/​azure-ai-agentserver-core/​azure/​ai/​agentserver/​core/​streaming/​_registry.py:218

Every missing get(id) now inserts an asyncio.Lock into _id_locks, but production code never removes these entries. The Responses admission path calls get() for every POST, including non-streaming and store=False requests that never create a stream slot, so ordinary traffic permanently grows this dictionary by one entry per response ID. Use ref-counted/transient per-ID locks (removing an idle lock only after all waiters are done) or another bounded synchronization strategy.