Invalidate subject caches after deletion - #2359
Open
Shubham Padkonde (Shubham-Padkonde) wants to merge 2 commits into
Open
Shubham Padkonde (Shubham-Padkonde) wants to merge 2 commits into
Shubham Padkonde (Shubham-Padkonde) wants to merge 2 commits into
Conversation
Shubham Padkonde (Shubham-Padkonde)
requested review from
a team and
Matthew Seal (MSeal)
as code owners
September 16, 2026 07:24
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
Copilot started reviewing on behalf of
Shubham Padkonde (Shubham-Padkonde)
September 16, 2026 07:24
View session
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved cache-race and TTL-expiry failures can leave stale data or make successful deletions fail.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Updates sync and async schema registry clients to invalidate subject schema and latest-result caches after successful soft or permanent deletion.
Changes:
- Updates deletion cache invalidation.
- Adds parameterized sync/async regression tests.
- Documents the fix in the changelog.
File summaries
| File | Summary |
|---|---|
tests/schema_registry/_sync/test_api_client.py |
Sync regression coverage for cache invalidation. |
tests/schema_registry/_async/test_api_client.py |
Async regression coverage for cache invalidation. |
src/confluent_kafka/schema_registry/_sync/schema_registry_client.py |
Critical (1 vote): in-flight reads can repopulate stale caches after deletion. Moderate (1 vote): TTL expiry can cause deletion invalidation to raise KeyError. |
src/confluent_kafka/schema_registry/_async/schema_registry_client.py |
Critical (1 vote): in-flight reads can repopulate stale caches after deletion. Moderate (1 vote): TTL expiry can cause deletion invalidation to raise KeyError. |
CHANGELOG.md |
Documents the customer-facing cache invalidation fix. |
Review details
Suppressed comments (2)
src/confluent_kafka/schema_registry/_async/schema_registry_client.py:1094
TTLCachecan expire a matching entry after the key snapshot is built but before this deletion runs; cachetools can raiseKeyErrorwhendeltargets an expired entry. That makes a successful subject deletion fail during cache invalidation for short TTLs. Usepop(cache_key, None)(or otherwise tolerate expiry) while holding the lock.
del self._latest_with_metadata_cache[cache_key]
src/confluent_kafka/schema_registry/_sync/schema_registry_client.py:1089
TTLCachecan expire a matching entry after the key snapshot is built but before this deletion runs; cachetools can raiseKeyErrorwhendeltargets an expired entry. That makes a successful subject deletion fail during cache invalidation for short TTLs. Usepop(cache_key, None)(or otherwise tolerate expiry) while holding the lock.
del self._latest_with_metadata_cache[cache_key]
- Files reviewed: 5/5 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+1089
to
+1093
| self._cache.remove_by_subject(subject_name) | ||
| async with self._latest_lock: | ||
| self._latest_version_cache.pop(subject_name, None) | ||
| for cache_key in list(self._latest_with_metadata_cache): | ||
| if cache_key[0] == subject_name: |
Comment on lines
+1084
to
+1088
| self._cache.remove_by_subject(subject_name) | ||
| with self._latest_lock: | ||
| self._latest_version_cache.pop(subject_name, None) | ||
| for cache_key in list(self._latest_with_metadata_cache): | ||
| if cache_key[0] == subject_name: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A successful soft delete left registered schemas cached, so registering the same schema again returned the cached ID without recreating the subject. Both soft and permanent deletion also left latest-version and metadata-filtered latest results cached.
Clear the deleted subject's schema cache after either successful deletion, and remove its latest-cache entries under the existing lock. Preserve other subjects' entries and keep caches unchanged if DELETE fails. Update the async implementation, generated sync client/tests, and changelog.
Checklist
References
Related cache discussion: #1950. This does not address its global schema-by-ID cache issue. #2333 covers cache invalidation after registration; this change covers whole-subject deletion.
Test & Review
Added 40 parameterized regression cases across sync/async clients: soft/permanent deletion, LRU/TTL latest caches, subsequent reads and re-registration, unrelated subjects, and failed deletion.
Tested with Python 3.12 and the 2.15.1 native extension against this checkout's Python source. No live Kafka/Schema Registry integration tests or full project suite were run.
Prepared with Codex assistance.