Fix stale TID errors in VLE path materialization - #2551
Open
jrgemignani wants to merge 1 commit into
Open
Conversation
Four problems with the TID-based VLE cache and its version counters, all from 798917c ("VLE cache + performance improvements"). 1. A path bound by MATCH could not be projected once the same statement deleted its endpoints (apache#2549): MATCH p = (n0)<-[:R*..2]-(n1) DETACH DELETE n0, n1 RETURN p ERROR: get_vertex_entry_properties: stale TID - ... That commit replaced the properties Datum in vertex_entry and edge_entry with a TID fetched lazily at projection. cypher_delete() advances es_snapshot->curcid past every delete, so a path's own endpoints fail the visibility test by the time they are read; before, properties were captured at cache build and a later delete could not affect them. Such a tuple is still physically present, the deleting transaction having not committed, so it is read anyway and the path reports the properties it was matched with. The relaxation is narrow: only a tuple deleted by our own transaction qualifies, and only while the row still carries the cached entity, so a recycled line pointer cannot be substituted. Any other unreachable TID still raises the error, keeping a real invalidation bug visible. Properties are detoasted under the buffer pin, and the buffer is now released on the failing path too, since heap_fetch is called with keep_buf, which leaves it pinned when only visibility fails. 2. VACUUM FULL and CLUSTER rewrite the heap, moving every cached TID, and announce themselves through no trigger and no version counter. A cached context then resolved stale TIDs against the new file, giving the stale TID error or "could not read block", which never reaches AGE's guard. Both are now intercepted in ag_ProcessUtility_hook as TRUNCATE already was, and the database-wide forms invalidate every tracked graph. Plain VACUUM and ANALYZE do not move tuples and are ignored. 3. NULL properties were reported as a stale TID. Label tables are created with properties NOT NULL, so a NULL means the table was altered out from under AGE; that is now said plainly rather than blamed on the cache. 4. Version counter slots were never released, making the table a tally of every graph ever mutated rather than of those that exist. A server cycling graphs filled it, then warned on every mutation and fell back to snapshot invalidation. drop_graph() now releases its slot and a freed slot is reused; a new occupant seeds its version above every value the table has issued, so a context cached for the previous occupant cannot compare equal. The cap moves 128 -> 256, about 4 KB of shared memory at 16 bytes per entry; lookups scan only the entries in use, so unused slots cost nothing. Both accessors now share one helper, and hardcoded column numbers give way to the Anum_ag_label_* constants. cypher_vle gains 14 cases: the reported query, a fan-out that fails if the result depends on which row is projected first, partial and edge-only deletes, the edge-list projection, an edge property constraint reaching the accessor during traversal, zero-length bounds, self-loops, labelled vertices, a multi-hop chain, a delete from an earlier statement that must not be resurrected, savepoint and transaction rollback, and out-of-line TOAST asserted set-identical to a live read. age_global_graph covers CLUSTER and all three VACUUM FULL spellings (named, parenthesised, database-wide), plus plain VACUUM, ANALYZE and FULL false which must not invalidate; NULL properties on a vertex and an edge; 260 create/drop cycles that must stay silent; a rolled-back drop; and a graph recreated under a dropped name. Verified on PostgreSQL 18.4 and 18.6: clean build, no warnings; installcheck 43/43 on 18.4 before and after, run twice; and 43/43 on 18.6 with --enable-cassert, reporting no assertion failure or resource leak. Fixes apache#2549 Co-authored-by: GitHub Copilot (Claude Opus 5) <noreply@github.com> modified: regress/expected/age_global_graph.out modified: regress/expected/cypher_vle.out modified: regress/sql/age_global_graph.sql modified: regress/sql/cypher_vle.sql modified: src/backend/catalog/ag_catalog.c modified: src/backend/commands/graph_commands.c modified: src/backend/utils/adt/age_global_graph.c modified: src/include/utils/age_global_graph.h
jrgemignani
requested review from
MuhammadTahaNaveed,
gregfelice and
muhammadshoaib
and
a lite review from Copilot
August 25, 2026 00:10
There was a problem hiding this comment.
Pull request overview
This PR fixes several correctness issues in AGE’s TID-backed VLE cache and graph-version invalidation, addressing stale-TID failures when projecting paths after in-transaction deletes (Issue #2549) and improving cache invalidation behavior for heap-rewriting utilities and graph lifecycle churn.
Changes:
- Relax property fetch visibility rules narrowly to allow projecting entities deleted by the same transaction while still detecting genuinely stale cache entries.
- Invalidate cached graph contexts on heap-rewrite utilities (VACUUM FULL / CLUSTER), including database-wide forms.
- Improve diagnostics for NULL properties (schema violation) and allow graph-version counter slots to be released/reused on drop.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/include/utils/age_global_graph.h | Exposes new APIs for global version invalidation and slot release. |
| src/backend/utils/adt/age_global_graph.c | Implements property fetch helper with narrow “own-delete” relaxation, NULL-property errors, and improved version-slot management (+ cap increase). |
| src/backend/commands/graph_commands.c | Releases the dropped graph’s version slot after drop_graph completes. |
| src/backend/catalog/ag_catalog.c | Hooks VACUUM FULL / CLUSTER to invalidate graph contexts when heaps are rewritten. |
| regress/sql/cypher_vle.sql | Adds regression coverage for projecting VLE paths after deletes and related edge cases (14 cases). |
| regress/expected/cypher_vle.out | Expected output updates for the new cypher_vle regression cases. |
| regress/sql/age_global_graph.sql | Adds regression coverage for heap rewrite invalidation, NULL properties, and version-slot churn/reuse behavior. |
| regress/expected/age_global_graph.out | Expected output updates for the new age_global_graph regression cases. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+1415
to
+1416
| * original. Anything else leaves *found false and the caller reports a stale | ||
| * entry, which is what keeps a genuine cache-invalidation bug visible. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four problems with the TID-based VLE cache and its version counters, all from 798917c ("VLE cache + performance improvements").
A path bound by MATCH could not be projected once the same statement deleted its endpoints (stale TID` after deleting a path-bound vertex #2549):
That commit replaced the properties Datum in vertex_entry and edge_entry with a TID fetched lazily at projection. cypher_delete() advances es_snapshot->curcid past every delete, so a path's own endpoints fail the visibility test by the time they are read; before, properties were captured at cache build and a later delete could not affect them.
Such a tuple is still physically present, the deleting transaction having not committed, so it is read anyway and the path reports the properties it was matched with. The relaxation is narrow: only a tuple deleted by our own transaction qualifies, and only while the row still carries the cached entity, so a recycled line pointer cannot be substituted. Any other unreachable TID still raises the error, keeping a real invalidation bug visible. Properties are detoasted under the buffer pin, and the buffer is now released on the failing path too, since heap_fetch is called with keep_buf, which leaves it pinned when only visibility fails.
VACUUM FULL and CLUSTER rewrite the heap, moving every cached TID, and announce themselves through no trigger and no version counter. A cached context then resolved stale TIDs against the new file, giving the stale TID error or "could not read block", which never reaches AGE's guard. Both are now intercepted in ag_ProcessUtility_hook as TRUNCATE already was, and the database-wide forms invalidate every tracked graph. Plain VACUUM and ANALYZE do not move tuples and are ignored.
NULL properties were reported as a stale TID. Label tables are created with properties NOT NULL, so a NULL means the table was altered out from under AGE; that is now said plainly rather than blamed on the cache.
Version counter slots were never released, making the table a tally of every graph ever mutated rather than of those that exist. A server cycling graphs filled it, then warned on every mutation and fell back to snapshot invalidation. drop_graph() now releases its slot and a freed slot is reused; a new occupant seeds its version above every value the table has issued, so a context cached for the previous occupant cannot compare equal. The cap moves 128 -> 256, about 4 KB of shared memory at 16 bytes per entry; lookups scan only the entries in use, so unused slots cost nothing.
Both accessors now share one helper, and hardcoded column numbers give way to the Anum_ag_label_* constants.
cypher_vle gains 14 cases: the reported query, a fan-out that fails if the result depends on which row is projected first, partial and edge-only deletes, the edge-list projection, an edge property constraint reaching the accessor during traversal, zero-length bounds, self-loops, labelled vertices, a multi-hop chain, a delete from an earlier statement that must not be resurrected, savepoint and transaction rollback, and out-of-line TOAST asserted set-identical to a live read.
age_global_graph covers CLUSTER and all three VACUUM FULL spellings (named, parenthesised, database-wide), plus plain VACUUM, ANALYZE and FULL false which must not invalidate; NULL properties on a vertex and an edge; 260 create/drop cycles that must stay silent; a rolled-back drop; and a graph recreated under a dropped name.
Verified on PostgreSQL 18.4 and 18.6: clean build, no warnings; installcheck 43/43 on 18.4 before and after, run twice; and 43/43 on 18.6 with --enable-cassert, reporting no assertion failure or resource leak.
Fixes #2549
modified: regress/expected/age_global_graph.out
modified: regress/expected/cypher_vle.out
modified: regress/sql/age_global_graph.sql
modified: regress/sql/cypher_vle.sql
modified: src/backend/catalog/ag_catalog.c
modified: src/backend/commands/graph_commands.c
modified: src/backend/utils/adt/age_global_graph.c
modified: src/include/utils/age_global_graph.h