From eb0b55d55b4186a251f1918cc8e33862605b3238 Mon Sep 17 00:00:00 2001 From: John Gemignani Date: Mon, 24 Aug 2026 17:06:55 -0700 Subject: [PATCH] Fix stale TID errors in VLE path materialization Four problems with the TID-based VLE cache and its version counters, all from 798917c2 ("VLE cache + performance improvements"). 1. A path bound by MATCH could not be projected once the same statement deleted its endpoints (#2549): MATCH p = (n0)<-[:R*..2]-(n1) DETACH DELETE n0, n1 RETURN p ERROR: get_vertex_entry_properties: stale TID - ... That commit replaced the properties Datum in vertex_entry and edge_entry with a TID fetched lazily at projection. cypher_delete() advances es_snapshot->curcid past every delete, so a path's own endpoints fail the visibility test by the time they are read; before, properties were captured at cache build and a later delete could not affect them. Such a tuple is still physically present, the deleting transaction having not committed, so it is read anyway and the path reports the properties it was matched with. The relaxation is narrow: only a tuple deleted by our own transaction qualifies, and only while the row still carries the cached entity, so a recycled line pointer cannot be substituted. Any other unreachable TID still raises the error, keeping a real invalidation bug visible. Properties are detoasted under the buffer pin, and the buffer is now released on the failing path too, since heap_fetch is called with keep_buf, which leaves it pinned when only visibility fails. 2. VACUUM FULL and CLUSTER rewrite the heap, moving every cached TID, and announce themselves through no trigger and no version counter. A cached context then resolved stale TIDs against the new file, giving the stale TID error or "could not read block", which never reaches AGE's guard. Both are now intercepted in ag_ProcessUtility_hook as TRUNCATE already was, and the database-wide forms invalidate every tracked graph. Plain VACUUM and ANALYZE do not move tuples and are ignored. 3. NULL properties were reported as a stale TID. Label tables are created with properties NOT NULL, so a NULL means the table was altered out from under AGE; that is now said plainly rather than blamed on the cache. 4. Version counter slots were never released, making the table a tally of every graph ever mutated rather than of those that exist. A server cycling graphs filled it, then warned on every mutation and fell back to snapshot invalidation. drop_graph() now releases its slot and a freed slot is reused; a new occupant seeds its version above every value the table has issued, so a context cached for the previous occupant cannot compare equal. The cap moves 128 -> 256, about 4 KB of shared memory at 16 bytes per entry; lookups scan only the entries in use, so unused slots cost nothing. Both accessors now share one helper, and hardcoded column numbers give way to the Anum_ag_label_* constants. cypher_vle gains 14 cases: the reported query, a fan-out that fails if the result depends on which row is projected first, partial and edge-only deletes, the edge-list projection, an edge property constraint reaching the accessor during traversal, zero-length bounds, self-loops, labelled vertices, a multi-hop chain, a delete from an earlier statement that must not be resurrected, savepoint and transaction rollback, and out-of-line TOAST asserted set-identical to a live read. age_global_graph covers CLUSTER and all three VACUUM FULL spellings (named, parenthesised, database-wide), plus plain VACUUM, ANALYZE and FULL false which must not invalidate; NULL properties on a vertex and an edge; 260 create/drop cycles that must stay silent; a rolled-back drop; and a graph recreated under a dropped name. Verified on PostgreSQL 18.4 and 18.6: clean build, no warnings; installcheck 43/43 on 18.4 before and after, run twice; and 43/43 on 18.6 with --enable-cassert, reporting no assertion failure or resource leak. Fixes #2549 Co-authored-by: GitHub Copilot (Claude Opus 5) modified: regress/expected/age_global_graph.out modified: regress/expected/cypher_vle.out modified: regress/sql/age_global_graph.sql modified: regress/sql/cypher_vle.sql modified: src/backend/catalog/ag_catalog.c modified: src/backend/commands/graph_commands.c modified: src/backend/utils/adt/age_global_graph.c modified: src/include/utils/age_global_graph.h --- regress/expected/age_global_graph.out | 384 +++++++++++++++ regress/expected/cypher_vle.out | 603 +++++++++++++++++++++++ regress/sql/age_global_graph.sql | 231 +++++++++ regress/sql/cypher_vle.sql | 267 ++++++++++ src/backend/catalog/ag_catalog.c | 104 ++++ src/backend/commands/graph_commands.c | 8 + src/backend/utils/adt/age_global_graph.c | 342 ++++++++++--- src/include/utils/age_global_graph.h | 2 + 8 files changed, 1877 insertions(+), 64 deletions(-) diff --git a/regress/expected/age_global_graph.out b/regress/expected/age_global_graph.out index 3a5f46060..6f269e6ab 100644 --- a/regress/expected/age_global_graph.out +++ b/regress/expected/age_global_graph.out @@ -732,6 +732,390 @@ NOTICE: graph "vle_trigger_test" has been dropped (1 row) +----------------------------------------------------------------------------------------------------------------------------- +-- +-- Heap rewrites must invalidate cached graph contexts +-- +-- A rewrite moves every TID a cached context holds and announces itself through +-- no trigger and no version counter. Left untracked, the context keeps +-- resolving stale TIDs against the new file: it reads the wrong tuple, or a +-- block past the end of a now shorter relation. +-- +-- Every rewrite below is preceded by a read that caches the context and +-- followed by a read in the SAME session; a fresh session would rebuild and +-- prove nothing. The low ids are deleted first so the survivors sit at high +-- offsets and a rewrite has to move them. Ground truth throughout is the +-- 59 -> 60 path. +-- +----------------------------------------------------------------------------------------------------------------------------- +SELECT create_graph('rewrite_inval'); +NOTICE: graph "rewrite_inval" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('rewrite_inval', $$ + UNWIND range(1, 60) AS i CREATE ({id: i}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH (a), (b) WHERE a.id = 59 AND b.id = 60 CREATE (a)-[:R]->(b) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH (n) WHERE n.id < 51 DETACH DELETE n +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +-- CLUSTER rewrites in index order, moving the survivors to the front +CLUSTER rewrite_inval."_ag_label_vertex" USING "_ag_label_vertex_pkey"; +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +-- make dead space again so VACUUM FULL has something to compact away +SELECT * FROM cypher('rewrite_inval', $$ + UNWIND range(100, 119) AS i CREATE ({id: i}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH (n) WHERE n.id >= 100 AND n.id < 110 DETACH DELETE n +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +VACUUM FULL rewrite_inval."_ag_label_vertex"; +VACUUM FULL rewrite_inval."R"; +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +-- plain VACUUM and ANALYZE leave tuples in place, so they are not rewrites and +-- the cached context stays usable +VACUUM rewrite_inval."_ag_label_vertex"; +ANALYZE rewrite_inval."_ag_label_vertex"; +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +-- the parenthesised spelling is a rewrite too +VACUUM (FULL) rewrite_inval."_ag_label_vertex"; +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +-- and FULL turned off explicitly is not +VACUUM (FULL false) rewrite_inval."_ag_label_vertex"; +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +-- naming no relation rewrites the whole database, which has to invalidate +-- every tracked graph rather than a named one +VACUUM FULL; +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710715, "label": "_ag_label_vertex", "properties": {"id": 59}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710716, "start_id": 281474976710715, "properties": {}}::edge, {"id": 281474976710716, "label": "_ag_label_vertex", "properties": {"id": 60}}::vertex]::path +(1 row) + +SELECT drop_graph('rewrite_inval', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table rewrite_inval._ag_label_vertex +drop cascades to table rewrite_inval._ag_label_edge +drop cascades to table rewrite_inval."R" +NOTICE: graph "rewrite_inval" has been dropped + drop_graph +------------ + +(1 row) + +----------------------------------------------------------------------------------------------------------------------------- +-- +-- NULL properties are a schema problem, not a stale cache +-- +-- Label tables are created with properties NOT NULL, so a NULL there means the +-- table was altered out from under AGE. Reporting that as a stale TID sends the +-- reader after the wrong thing. +-- +----------------------------------------------------------------------------------------------------------------------------- +SELECT create_graph('null_props'); +NOTICE: graph "null_props" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('null_props', $$ + CREATE ({id: 1})-[:R {w: 1}]->({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +ALTER TABLE null_props."_ag_label_vertex" + ALTER COLUMN properties DROP NOT NULL; +UPDATE null_props."_ag_label_vertex" SET properties = NULL; +SELECT * FROM cypher('null_props', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); +ERROR: get_vertex_entry_properties: vertex 281474976710657 has null properties +UPDATE null_props."_ag_label_vertex" + SET properties = ag_catalog.agtype_build_map(); +-- the edge constraint is inherited from _ag_label_edge, so it is dropped there +ALTER TABLE null_props."_ag_label_edge" + ALTER COLUMN properties DROP NOT NULL; +UPDATE null_props."R" SET properties = NULL; +SELECT * FROM cypher('null_props', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); +ERROR: get_edge_entry_properties: edge 844424930131969 has null properties +-- restored: projection works again +UPDATE null_props."R" SET properties = ag_catalog.agtype_build_map(); +SELECT * FROM cypher('null_props', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {}}::vertex]::path +(1 row) + +SELECT drop_graph('null_props', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table null_props._ag_label_vertex +drop cascades to table null_props._ag_label_edge +drop cascades to table null_props."R" +NOTICE: graph "null_props" has been dropped + drop_graph +------------ + +(1 row) + +----------------------------------------------------------------------------------------------------------------------------- +-- +-- A dropped graph releases its version counter slot +-- +-- The counter table is a fixed size. While slots were never released it was a +-- tally of every graph ever mutated, so a server that cycles graphs filled it, +-- warned on every later mutation, and fell back to snapshot invalidation -- +-- correct, but far more eager. Cycling well past the cap must now be silent. +-- +-- NOTICEs are muted so the loop does not bury the output; a WARNING from a full +-- table would still come through and fail this test, which is the assertion. +-- +----------------------------------------------------------------------------------------------------------------------------- +SET client_min_messages = warning; +DO $slots$ +DECLARE + i int; +BEGIN + FOR i IN 1..260 LOOP + PERFORM ag_catalog.create_graph('slot_churn_' || i); + EXECUTE format('SELECT * FROM ag_catalog.cypher(%L, $c$ CREATE ({id: 1}) $c$) AS (v ag_catalog.agtype)', + 'slot_churn_' || i); + PERFORM ag_catalog.drop_graph('slot_churn_' || i, true); + END LOOP; +END +$slots$; +RESET client_min_messages; +SELECT count(*) AS churn_graphs_left +FROM ag_catalog.ag_graph WHERE name::text LIKE 'slot_churn_%'; + churn_graphs_left +------------------- + 0 +(1 row) + +-- a drop that is rolled back must leave the graph tracked and usable +SELECT create_graph('drop_rollback'); +NOTICE: graph "drop_rollback" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('drop_rollback', $$ + CREATE ({id: 1})-[:R]->({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('drop_rollback', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +BEGIN; +SELECT drop_graph('drop_rollback', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table drop_rollback._ag_label_vertex +drop cascades to table drop_rollback._ag_label_edge +drop cascades to table drop_rollback."R" +NOTICE: graph "drop_rollback" has been dropped + drop_graph +------------ + +(1 row) + +ROLLBACK; +SELECT * FROM cypher('drop_rollback', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +-- and a later mutation still invalidates correctly +SELECT * FROM cypher('drop_rollback', $$ CREATE ({id: 3}) $$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('drop_rollback', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +SELECT drop_graph('drop_rollback', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table drop_rollback._ag_label_vertex +drop cascades to table drop_rollback._ag_label_edge +drop cascades to table drop_rollback."R" +NOTICE: graph "drop_rollback" has been dropped + drop_graph +------------ + +(1 row) + +-- a graph recreated under the same name must not inherit the old one's +-- context. The freed slot may be handed straight back, so the new occupant's +-- version has to be distinguishable from whatever was cached for the previous +-- one. +SELECT create_graph('slot_reuse'); +NOTICE: graph "slot_reuse" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('slot_reuse', $$ + CREATE ({id: 111})-[:R]->({id: 222}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('slot_reuse', $$ + MATCH p = (a)-[:R*..2]->(b) RETURN p +$$) AS (p agtype); + p +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 111}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 222}}::vertex]::path +(1 row) + +SELECT drop_graph('slot_reuse', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table slot_reuse._ag_label_vertex +drop cascades to table slot_reuse._ag_label_edge +drop cascades to table slot_reuse."R" +NOTICE: graph "slot_reuse" has been dropped + drop_graph +------------ + +(1 row) + +SELECT create_graph('slot_reuse'); +NOTICE: graph "slot_reuse" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('slot_reuse', $$ + CREATE ({id: 333})-[:R]->({id: 444}) +$$) AS (v agtype); + v +--- +(0 rows) + +-- 333 -> 444, never the dropped graph's 111 -> 222 +SELECT * FROM cypher('slot_reuse', $$ + MATCH p = (a)-[:R*..2]->(b) RETURN p +$$) AS (p agtype); + p +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 333}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 444}}::vertex]::path +(1 row) + +SELECT drop_graph('slot_reuse', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table slot_reuse._ag_label_vertex +drop cascades to table slot_reuse._ag_label_edge +drop cascades to table slot_reuse."R" +NOTICE: graph "slot_reuse" has been dropped + drop_graph +------------ + +(1 row) + ----------------------------------------------------------------------------------------------------------------------------- -- -- End of tests diff --git a/regress/expected/cypher_vle.out b/regress/expected/cypher_vle.out index de85176b6..fac61e5f3 100644 --- a/regress/expected/cypher_vle.out +++ b/regress/expected/cypher_vle.out @@ -1334,6 +1334,609 @@ NOTICE: graph "issue_2382" has been dropped (1 row) +-- +-- Issue #2549: a VLE path bound before a DETACH DELETE must still project. +-- +-- Entries in the VLE cache hold a TID and fetch properties lazily when the path +-- is materialized. cypher_delete() advances es_snapshot->curcid past every +-- delete, so by projection time a path's own endpoints are no longer visible +-- under the active snapshot. They are still readable, because the deleting +-- transaction has not committed, so the path reports the properties it was +-- matched with. +-- +-- Row order here is scan order, so every query that can return more than one +-- path orders on a scalar key: the point of several of these cases is that the +-- result does NOT depend on which row is projected first. +-- +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +-- 2549.1 the reported case: both endpoints deleted, path projected +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + k | p +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 281474976710658 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path + 281474976710659 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131970, "label": "R", "end_id": 281474976710657, "start_id": 281474976710659, "properties": {}}::edge, {"id": 281474976710659, "label": "_ag_label_vertex", "properties": {"id": 3}}::vertex]::path +(2 rows) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.2 fan-out: the shared vertex must read the same in every row, whichever +-- row the executor projects first +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}), + (a)<-[:R]-({id: 4}), (a)<-[:R]-({id: 5}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + k | p +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 281474976710658 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path + 281474976710659 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131970, "label": "R", "end_id": 281474976710657, "start_id": 281474976710659, "properties": {}}::edge, {"id": 281474976710659, "label": "_ag_label_vertex", "properties": {"id": 3}}::vertex]::path + 281474976710660 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131971, "label": "R", "end_id": 281474976710657, "start_id": 281474976710660, "properties": {}}::edge, {"id": 281474976710660, "label": "_ag_label_vertex", "properties": {"id": 4}}::vertex]::path + 281474976710661 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131972, "label": "R", "end_id": 281474976710657, "start_id": 281474976710661, "properties": {}}::edge, {"id": 281474976710661, "label": "_ag_label_vertex", "properties": {"id": 5}}::vertex]::path +(4 rows) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.3 only one endpoint deleted; the survivor is unaffected +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n1 + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + k | p +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 281474976710658 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path + 281474976710659 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131970, "label": "R", "end_id": 281474976710657, "start_id": 281474976710659, "properties": {}}::edge, {"id": 281474976710659, "label": "_ag_label_vertex", "properties": {"id": 3}}::vertex]::path +(2 rows) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.4 edge deleted rather than a vertex: exercises the edge accessor +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R {w: 7}]-({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[e:R*..2]-(n1) + DELETE e + RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {"w": 7}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.5 edge-list projection (build_edge_list) after the endpoints are gone +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE ({id: 1})-[:R {k: 1}]->({id: 2})-[:R {k: 2}]->({id: 3}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH (n0)-[e:R*2]->(n1) + DETACH DELETE n0, n1 + RETURN e +$$) AS (e agtype); + e +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ + [{"id": 844424930131970, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {"k": 1}}::edge, {"id": 844424930131969, "label": "R", "end_id": 281474976710659, "start_id": 281474976710658, "properties": {"k": 2}}::edge] +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.6 edge property constraint: the accessor is also reached during +-- traversal, not only when the path is built +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R {w: 7}]-({id: 2}), (a)<-[:R {w: 9}]-({id: 3}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R* {w: 7}]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {"w": 7}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.7 zero-length bound: the zero-hop paths carry the deleted vertex too +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*0..2]-(n1) + DETACH DELETE n0, n1 + RETURN id(n0) AS k0, id(n1) AS k1, length(p) AS len, p +$$) AS (k0 agtype, k1 agtype, len agtype, p agtype) ORDER BY k0, k1, len; + k0 | k1 | len | p +-----------------+-----------------+-----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 281474976710657 | 281474976710657 | 0 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex]::path + 281474976710657 | 281474976710658 | 1 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path + 281474976710658 | 281474976710658 | 0 | [{"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(3 rows) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.8 self-loop +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ CREATE (a {id: 1}) $$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH (a {id: 1}) CREATE (a)-[:R {s: 1}]->(a) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)-[:R*..2]->(n1) + DETACH DELETE n0 + RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710657, "properties": {"s": 1}}::edge, {"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex]::path +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.9 labelled vertices keep their labels +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a:Person {id: 1})<-[:R]-(b:Company {id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + p +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 844424930131969, "label": "Person", "properties": {"id": 1}}::vertex, {"id": 1125899906842625, "label": "R", "end_id": 844424930131969, "start_id": 1407374883553281, "properties": {}}::edge, {"id": 1407374883553281, "label": "Company", "properties": {"id": 2}}::vertex]::path +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 5 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."Person" +drop cascades to table issue_2549."R" +drop cascades to table issue_2549."Company" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.10 multi-hop chain: every vertex and edge on the path is deleted +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE ({id: 1})-[:R]->({id: 2})-[:R]->({id: 3})-[:R]->({id: 4}) +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)-[:R*3]->(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + p +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131971, "label": "R", "end_id": 281474976710658, "start_id": 281474976710657, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex, {"id": 844424930131970, "label": "R", "end_id": 281474976710659, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710659, "label": "_ag_label_vertex", "properties": {"id": 3}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710660, "start_id": 281474976710659, "properties": {}}::edge, {"id": 281474976710660, "label": "_ag_label_vertex", "properties": {"id": 4}}::vertex]::path +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.11 an entity deleted by an EARLIER statement must stay gone: the +-- relaxation must not resurrect it on a later read +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}) +$$) AS (v agtype); + v +--- +(0 rows) + +BEGIN; +SELECT * FROM cypher('issue_2549', $$ + MATCH (n) WHERE n.id = 2 DETACH DELETE n +$$) AS (v agtype); + v +--- +(0 rows) + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + k | p +-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 281474976710659 | [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131970, "label": "R", "end_id": 281474976710657, "start_id": 281474976710659, "properties": {}}::edge, {"id": 281474976710659, "label": "_ag_label_vertex", "properties": {"id": 3}}::vertex]::path +(1 row) + +COMMIT; +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.12 a delete undone by ROLLBACK TO leaves the entity live +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +BEGIN; +SAVEPOINT s1; +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +ROLLBACK TO s1; +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +COMMIT; +SELECT count(*) AS vertices_still_present +FROM issue_2549."_ag_label_vertex"; + vertices_still_present +------------------------ + 2 +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.13 projecting a deleted path does not persist anything on rollback +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +BEGIN; +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + p +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + [{"id": 281474976710657, "label": "_ag_label_vertex", "properties": {"id": 1}}::vertex, {"id": 844424930131969, "label": "R", "end_id": 281474976710657, "start_id": 281474976710658, "properties": {}}::edge, {"id": 281474976710658, "label": "_ag_label_vertex", "properties": {"id": 2}}::vertex]::path +(1 row) + +ROLLBACK; +SELECT count(*) AS vertices_after_rollback +FROM issue_2549."_ag_label_vertex"; + vertices_after_rollback +------------------------- + 2 +(1 row) + +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + +-- 2549.14 properties stored out of line (TOAST) must come back intact: the +-- value is read from a tuple that is no longer visible, so it is +-- detoasted while the buffer is still pinned +SELECT create_graph('issue_2549'); +NOTICE: graph "issue_2549" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + v +--- +(0 rows) + +-- deterministic and poorly compressible, so it is pushed out of line +UPDATE issue_2549."_ag_label_vertex" + SET properties = ('{"id": 1, "blob": "' || + (SELECT string_agg(md5(g::text), '' ORDER BY g) + FROM generate_series(1, 2000) g) || '"}')::agtype; +UPDATE issue_2549."R" + SET properties = ('{"blob": "' || + (SELECT string_agg(md5(g::text), '' ORDER BY g) + FROM generate_series(1, 2000) g) || '"}')::agtype; +-- if this ever reports false the case below stops proving anything +SELECT pg_relation_size(reltoastrelid) > 0 AS vertex_properties_out_of_line + FROM pg_class WHERE oid = 'issue_2549."_ag_label_vertex"'::regclass; + vertex_properties_out_of_line +------------------------------- + t +(1 row) + +BEGIN; +CREATE TEMP TABLE vle_live AS + SELECT p FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) RETURN p + $$) AS (p agtype); +CREATE TEMP TABLE vle_deleted AS + SELECT p FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) DETACH DELETE n0, n1 RETURN p + $$) AS (p agtype); +SELECT (SELECT count(*) FROM vle_live) AS live_rows, + (SELECT count(*) FROM vle_deleted) AS deleted_rows, + (SELECT count(*) FROM (SELECT p FROM vle_live + EXCEPT SELECT p FROM vle_deleted) d) AS only_live, + (SELECT count(*) FROM (SELECT p FROM vle_deleted + EXCEPT SELECT p FROM vle_live) d) AS only_deleted; + live_rows | deleted_rows | only_live | only_deleted +-----------+--------------+-----------+-------------- + 1 | 1 | 0 | 0 +(1 row) + +ROLLBACK; +SELECT drop_graph('issue_2549', true); +NOTICE: drop cascades to 3 other objects +DETAIL: drop cascades to table issue_2549._ag_label_vertex +drop cascades to table issue_2549._ag_label_edge +drop cascades to table issue_2549."R" +NOTICE: graph "issue_2549" has been dropped + drop_graph +------------ + +(1 row) + -- -- End -- diff --git a/regress/sql/age_global_graph.sql b/regress/sql/age_global_graph.sql index 65b748484..9852edcf4 100644 --- a/regress/sql/age_global_graph.sql +++ b/regress/sql/age_global_graph.sql @@ -354,6 +354,237 @@ $$) AS (name agtype); -- Cleanup SELECT * FROM drop_graph('vle_trigger_test', true); +----------------------------------------------------------------------------------------------------------------------------- +-- +-- Heap rewrites must invalidate cached graph contexts +-- +-- A rewrite moves every TID a cached context holds and announces itself through +-- no trigger and no version counter. Left untracked, the context keeps +-- resolving stale TIDs against the new file: it reads the wrong tuple, or a +-- block past the end of a now shorter relation. +-- +-- Every rewrite below is preceded by a read that caches the context and +-- followed by a read in the SAME session; a fresh session would rebuild and +-- prove nothing. The low ids are deleted first so the survivors sit at high +-- offsets and a rewrite has to move them. Ground truth throughout is the +-- 59 -> 60 path. +-- +----------------------------------------------------------------------------------------------------------------------------- + +SELECT create_graph('rewrite_inval'); + +SELECT * FROM cypher('rewrite_inval', $$ + UNWIND range(1, 60) AS i CREATE ({id: i}) +$$) AS (v agtype); + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH (a), (b) WHERE a.id = 59 AND b.id = 60 CREATE (a)-[:R]->(b) +$$) AS (v agtype); + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH (n) WHERE n.id < 51 DETACH DELETE n +$$) AS (v agtype); + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- CLUSTER rewrites in index order, moving the survivors to the front +CLUSTER rewrite_inval."_ag_label_vertex" USING "_ag_label_vertex_pkey"; + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- make dead space again so VACUUM FULL has something to compact away +SELECT * FROM cypher('rewrite_inval', $$ + UNWIND range(100, 119) AS i CREATE ({id: i}) +$$) AS (v agtype); + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH (n) WHERE n.id >= 100 AND n.id < 110 DETACH DELETE n +$$) AS (v agtype); + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +VACUUM FULL rewrite_inval."_ag_label_vertex"; +VACUUM FULL rewrite_inval."R"; + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- plain VACUUM and ANALYZE leave tuples in place, so they are not rewrites and +-- the cached context stays usable +VACUUM rewrite_inval."_ag_label_vertex"; +ANALYZE rewrite_inval."_ag_label_vertex"; + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- the parenthesised spelling is a rewrite too +VACUUM (FULL) rewrite_inval."_ag_label_vertex"; + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- and FULL turned off explicitly is not +VACUUM (FULL false) rewrite_inval."_ag_label_vertex"; + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- naming no relation rewrites the whole database, which has to invalidate +-- every tracked graph rather than a named one +VACUUM FULL; + +SELECT * FROM cypher('rewrite_inval', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +SELECT drop_graph('rewrite_inval', true); + +----------------------------------------------------------------------------------------------------------------------------- +-- +-- NULL properties are a schema problem, not a stale cache +-- +-- Label tables are created with properties NOT NULL, so a NULL there means the +-- table was altered out from under AGE. Reporting that as a stale TID sends the +-- reader after the wrong thing. +-- +----------------------------------------------------------------------------------------------------------------------------- + +SELECT create_graph('null_props'); + +SELECT * FROM cypher('null_props', $$ + CREATE ({id: 1})-[:R {w: 1}]->({id: 2}) +$$) AS (v agtype); + +ALTER TABLE null_props."_ag_label_vertex" + ALTER COLUMN properties DROP NOT NULL; +UPDATE null_props."_ag_label_vertex" SET properties = NULL; + +SELECT * FROM cypher('null_props', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +UPDATE null_props."_ag_label_vertex" + SET properties = ag_catalog.agtype_build_map(); + +-- the edge constraint is inherited from _ag_label_edge, so it is dropped there +ALTER TABLE null_props."_ag_label_edge" + ALTER COLUMN properties DROP NOT NULL; +UPDATE null_props."R" SET properties = NULL; + +SELECT * FROM cypher('null_props', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- restored: projection works again +UPDATE null_props."R" SET properties = ag_catalog.agtype_build_map(); + +SELECT * FROM cypher('null_props', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +SELECT drop_graph('null_props', true); + +----------------------------------------------------------------------------------------------------------------------------- +-- +-- A dropped graph releases its version counter slot +-- +-- The counter table is a fixed size. While slots were never released it was a +-- tally of every graph ever mutated, so a server that cycles graphs filled it, +-- warned on every later mutation, and fell back to snapshot invalidation -- +-- correct, but far more eager. Cycling well past the cap must now be silent. +-- +-- NOTICEs are muted so the loop does not bury the output; a WARNING from a full +-- table would still come through and fail this test, which is the assertion. +-- +----------------------------------------------------------------------------------------------------------------------------- + +SET client_min_messages = warning; + +DO $slots$ +DECLARE + i int; +BEGIN + FOR i IN 1..260 LOOP + PERFORM ag_catalog.create_graph('slot_churn_' || i); + EXECUTE format('SELECT * FROM ag_catalog.cypher(%L, $c$ CREATE ({id: 1}) $c$) AS (v ag_catalog.agtype)', + 'slot_churn_' || i); + PERFORM ag_catalog.drop_graph('slot_churn_' || i, true); + END LOOP; +END +$slots$; + +RESET client_min_messages; + +SELECT count(*) AS churn_graphs_left +FROM ag_catalog.ag_graph WHERE name::text LIKE 'slot_churn_%'; + +-- a drop that is rolled back must leave the graph tracked and usable +SELECT create_graph('drop_rollback'); + +SELECT * FROM cypher('drop_rollback', $$ + CREATE ({id: 1})-[:R]->({id: 2}) +$$) AS (v agtype); + +SELECT * FROM cypher('drop_rollback', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +BEGIN; +SELECT drop_graph('drop_rollback', true); +ROLLBACK; + +SELECT * FROM cypher('drop_rollback', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +-- and a later mutation still invalidates correctly +SELECT * FROM cypher('drop_rollback', $$ CREATE ({id: 3}) $$) AS (v agtype); + +SELECT * FROM cypher('drop_rollback', $$ + MATCH p = (n0)-[:R*..2]->(n1) RETURN p +$$) AS (p agtype); + +SELECT drop_graph('drop_rollback', true); + +-- a graph recreated under the same name must not inherit the old one's +-- context. The freed slot may be handed straight back, so the new occupant's +-- version has to be distinguishable from whatever was cached for the previous +-- one. +SELECT create_graph('slot_reuse'); + +SELECT * FROM cypher('slot_reuse', $$ + CREATE ({id: 111})-[:R]->({id: 222}) +$$) AS (v agtype); + +SELECT * FROM cypher('slot_reuse', $$ + MATCH p = (a)-[:R*..2]->(b) RETURN p +$$) AS (p agtype); + +SELECT drop_graph('slot_reuse', true); + +SELECT create_graph('slot_reuse'); + +SELECT * FROM cypher('slot_reuse', $$ + CREATE ({id: 333})-[:R]->({id: 444}) +$$) AS (v agtype); + +-- 333 -> 444, never the dropped graph's 111 -> 222 +SELECT * FROM cypher('slot_reuse', $$ + MATCH p = (a)-[:R*..2]->(b) RETURN p +$$) AS (p agtype); + +SELECT drop_graph('slot_reuse', true); + ----------------------------------------------------------------------------------------------------------------------------- -- -- End of tests diff --git a/regress/sql/cypher_vle.sql b/regress/sql/cypher_vle.sql index 5f3f54ed2..12359f4de 100644 --- a/regress/sql/cypher_vle.sql +++ b/regress/sql/cypher_vle.sql @@ -487,6 +487,273 @@ $$) AS (person agtype, friend agtype); SELECT drop_graph('issue_2382', true); +-- +-- Issue #2549: a VLE path bound before a DETACH DELETE must still project. +-- +-- Entries in the VLE cache hold a TID and fetch properties lazily when the path +-- is materialized. cypher_delete() advances es_snapshot->curcid past every +-- delete, so by projection time a path's own endpoints are no longer visible +-- under the active snapshot. They are still readable, because the deleting +-- transaction has not committed, so the path reports the properties it was +-- matched with. +-- +-- Row order here is scan order, so every query that can return more than one +-- path orders on a scalar key: the point of several of these cases is that the +-- result does NOT depend on which row is projected first. +-- +SELECT create_graph('issue_2549'); + +-- 2549.1 the reported case: both endpoints deleted, path projected +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + +SELECT drop_graph('issue_2549', true); + +-- 2549.2 fan-out: the shared vertex must read the same in every row, whichever +-- row the executor projects first +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}), + (a)<-[:R]-({id: 4}), (a)<-[:R]-({id: 5}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + +SELECT drop_graph('issue_2549', true); + +-- 2549.3 only one endpoint deleted; the survivor is unaffected +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n1 + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; + +SELECT drop_graph('issue_2549', true); + +-- 2549.4 edge deleted rather than a vertex: exercises the edge accessor +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R {w: 7}]-({id: 2}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[e:R*..2]-(n1) + DELETE e + RETURN p +$$) AS (p agtype); + +SELECT drop_graph('issue_2549', true); + +-- 2549.5 edge-list projection (build_edge_list) after the endpoints are gone +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE ({id: 1})-[:R {k: 1}]->({id: 2})-[:R {k: 2}]->({id: 3}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH (n0)-[e:R*2]->(n1) + DETACH DELETE n0, n1 + RETURN e +$$) AS (e agtype); + +SELECT drop_graph('issue_2549', true); + +-- 2549.6 edge property constraint: the accessor is also reached during +-- traversal, not only when the path is built +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R {w: 7}]-({id: 2}), (a)<-[:R {w: 9}]-({id: 3}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R* {w: 7}]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + +SELECT drop_graph('issue_2549', true); + +-- 2549.7 zero-length bound: the zero-hop paths carry the deleted vertex too +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*0..2]-(n1) + DETACH DELETE n0, n1 + RETURN id(n0) AS k0, id(n1) AS k1, length(p) AS len, p +$$) AS (k0 agtype, k1 agtype, len agtype, p agtype) ORDER BY k0, k1, len; + +SELECT drop_graph('issue_2549', true); + +-- 2549.8 self-loop +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ CREATE (a {id: 1}) $$) AS (v agtype); +SELECT * FROM cypher('issue_2549', $$ + MATCH (a {id: 1}) CREATE (a)-[:R {s: 1}]->(a) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)-[:R*..2]->(n1) + DETACH DELETE n0 + RETURN p +$$) AS (p agtype); + +SELECT drop_graph('issue_2549', true); + +-- 2549.9 labelled vertices keep their labels +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a:Person {id: 1})<-[:R]-(b:Company {id: 2}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + +SELECT drop_graph('issue_2549', true); + +-- 2549.10 multi-hop chain: every vertex and edge on the path is deleted +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE ({id: 1})-[:R]->({id: 2})-[:R]->({id: 3})-[:R]->({id: 4}) +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)-[:R*3]->(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); + +SELECT drop_graph('issue_2549', true); + +-- 2549.11 an entity deleted by an EARLIER statement must stay gone: the +-- relaxation must not resurrect it on a later read +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}), (a)<-[:R]-({id: 3}) +$$) AS (v agtype); + +BEGIN; +SELECT * FROM cypher('issue_2549', $$ + MATCH (n) WHERE n.id = 2 DETACH DELETE n +$$) AS (v agtype); + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + RETURN id(n1) AS k, p +$$) AS (k agtype, p agtype) ORDER BY k; +COMMIT; + +SELECT drop_graph('issue_2549', true); + +-- 2549.12 a delete undone by ROLLBACK TO leaves the entity live +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + +BEGIN; +SAVEPOINT s1; +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); +ROLLBACK TO s1; + +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + RETURN p +$$) AS (p agtype); +COMMIT; + +SELECT count(*) AS vertices_still_present +FROM issue_2549."_ag_label_vertex"; + +SELECT drop_graph('issue_2549', true); + +-- 2549.13 projecting a deleted path does not persist anything on rollback +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + +BEGIN; +SELECT * FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) + DETACH DELETE n0, n1 + RETURN p +$$) AS (p agtype); +ROLLBACK; + +SELECT count(*) AS vertices_after_rollback +FROM issue_2549."_ag_label_vertex"; + +SELECT drop_graph('issue_2549', true); + +-- 2549.14 properties stored out of line (TOAST) must come back intact: the +-- value is read from a tuple that is no longer visible, so it is +-- detoasted while the buffer is still pinned +SELECT create_graph('issue_2549'); +SELECT * FROM cypher('issue_2549', $$ + CREATE (a {id: 1})<-[:R]-({id: 2}) +$$) AS (v agtype); + +-- deterministic and poorly compressible, so it is pushed out of line +UPDATE issue_2549."_ag_label_vertex" + SET properties = ('{"id": 1, "blob": "' || + (SELECT string_agg(md5(g::text), '' ORDER BY g) + FROM generate_series(1, 2000) g) || '"}')::agtype; +UPDATE issue_2549."R" + SET properties = ('{"blob": "' || + (SELECT string_agg(md5(g::text), '' ORDER BY g) + FROM generate_series(1, 2000) g) || '"}')::agtype; + +-- if this ever reports false the case below stops proving anything +SELECT pg_relation_size(reltoastrelid) > 0 AS vertex_properties_out_of_line + FROM pg_class WHERE oid = 'issue_2549."_ag_label_vertex"'::regclass; + +BEGIN; +CREATE TEMP TABLE vle_live AS + SELECT p FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) RETURN p + $$) AS (p agtype); + +CREATE TEMP TABLE vle_deleted AS + SELECT p FROM cypher('issue_2549', $$ + MATCH p = (n0)<-[:R*..2]-(n1) DETACH DELETE n0, n1 RETURN p + $$) AS (p agtype); + +SELECT (SELECT count(*) FROM vle_live) AS live_rows, + (SELECT count(*) FROM vle_deleted) AS deleted_rows, + (SELECT count(*) FROM (SELECT p FROM vle_live + EXCEPT SELECT p FROM vle_deleted) d) AS only_live, + (SELECT count(*) FROM (SELECT p FROM vle_deleted + EXCEPT SELECT p FROM vle_live) d) AS only_deleted; +ROLLBACK; + +SELECT drop_graph('issue_2549', true); + -- -- End -- diff --git a/src/backend/catalog/ag_catalog.c b/src/backend/catalog/ag_catalog.c index 107de370d..79e3fd455 100644 --- a/src/backend/catalog/ag_catalog.c +++ b/src/backend/catalog/ag_catalog.c @@ -52,6 +52,8 @@ void ag_ProcessUtility_hook(PlannedStmt *pstmt, const char *queryString, bool re QueryCompletion *qc); static bool is_age_drop(DropStmt *drop_stmt); +static bool vacuum_rewrites_heap(VacuumStmt *vacuum_stmt); +static void invalidate_graph_for_rangevar(RangeVar *relation); static void invalidate_extension_cache_callback(Datum argument, Oid relationId) @@ -121,6 +123,60 @@ void process_utility_hook_fini(void) ProcessUtility_hook = prev_process_utility_hook; } +/* + * True when this VACUUM rewrites the heap, which only FULL does. Plain VACUUM + * and ANALYZE leave live tuples where they are, so cached TIDs stay correct. + */ +static bool vacuum_rewrites_heap(VacuumStmt *vacuum_stmt) +{ + ListCell *lc; + + foreach(lc, vacuum_stmt->options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "full") == 0) + { + return defGetBoolean(def); + } + } + + return false; +} + +/* + * Invalidate the graph owning this relation, if it owns one. + * + * Resolved without taking a lock: the caller is about to run a command that + * wants AccessExclusiveLock on the same relation, and the OID is only used to + * find a graph and bump a counter. If the name no longer resolves we simply do + * not bump, which is what would happen if the command failed anyway. + */ +static void invalidate_graph_for_rangevar(RangeVar *relation) +{ + Oid rel_oid; + Oid graph_oid; + + if (relation == NULL) + { + return; + } + + rel_oid = RangeVarGetRelid(relation, NoLock, true); + + if (!OidIsValid(rel_oid)) + { + return; + } + + graph_oid = get_graph_oid_for_table(rel_oid); + + if (OidIsValid(graph_oid)) + { + increment_graph_version(graph_oid); + } +} + /* * When Postgres tries to drop AGE using the standard logic, two issues occur: * @@ -200,6 +256,54 @@ void ag_ProcessUtility_hook(PlannedStmt *pstmt, const char *queryString, } } break; + case T_VacuumStmt: + { + /* + * VACUUM FULL rewrites the heap, so every TID the VLE + * cache holds moves. Nothing else reports that: it fires + * no trigger and touches no version counter, so a cached + * context would keep resolving stale TIDs against the new + * file and read the wrong tuple, or a block past the end + * of a now shorter relation. + */ + VacuumStmt *vstmt = (VacuumStmt *) parsetree; + + if (vacuum_rewrites_heap(vstmt)) + { + if (vstmt->rels == NIL) + { + increment_all_graph_versions(); + } + else + { + ListCell *lc; + + foreach(lc, vstmt->rels) + { + VacuumRelation *vrel = + (VacuumRelation *) lfirst(lc); + + invalidate_graph_for_rangevar(vrel->relation); + } + } + } + } + break; + case T_ClusterStmt: + { + /* CLUSTER rewrites the heap for the same reason. */ + ClusterStmt *cstmt = (ClusterStmt *) parsetree; + + if (cstmt->relation == NULL) + { + increment_all_graph_versions(); + } + else + { + invalidate_graph_for_rangevar(cstmt->relation); + } + } + break; default: break; } diff --git a/src/backend/commands/graph_commands.c b/src/backend/commands/graph_commands.c index f7e8d070b..c30c027b2 100644 --- a/src/backend/commands/graph_commands.c +++ b/src/backend/commands/graph_commands.c @@ -31,6 +31,7 @@ #include "catalog/ag_label.h" #include "commands/label_commands.h" #include "commands/graph_commands.h" +#include "utils/age_global_graph.h" #include "utils/name_validation.h" /* @@ -194,6 +195,7 @@ Datum drop_graph(PG_FUNCTION_ARGS) Name graph_name; char *graph_name_str; bool cascade; + Oid graph_oid; if (PG_ARGISNULL(0)) { @@ -210,11 +212,17 @@ Datum drop_graph(PG_FUNCTION_ARGS) errmsg("graph \"%s\" does not exist", graph_name_str))); } + /* read while the ag_graph row is still there */ + graph_oid = get_graph_oid(graph_name_str); + drop_schema_for_graph(graph_name_str, cascade); delete_graph(graph_name); CommandCounterIncrement(); + /* the graph is gone, so let another one reuse its version counter slot */ + release_graph_version(graph_oid); + ereport(NOTICE, (errmsg("graph \"%s\" has been dropped", graph_name_str))); PG_RETURN_VOID(); diff --git a/src/backend/utils/adt/age_global_graph.c b/src/backend/utils/adt/age_global_graph.c index 397e0511c..984d0fd01 100644 --- a/src/backend/utils/adt/age_global_graph.c +++ b/src/backend/utils/adt/age_global_graph.c @@ -19,7 +19,11 @@ #include "postgres.h" +#include "fmgr.h" + #include "access/heapam.h" +#include "access/htup_details.h" +#include "access/xact.h" #include "catalog/namespace.h" #include "commands/trigger.h" #include "common/hashfn.h" @@ -51,8 +55,17 @@ #define VERTEX_HTAB_INITIAL_SIZE 10000 #define EDGE_HTAB_INITIAL_SIZE 10000 -/* Maximum number of graphs tracked for version counting */ -#define AGE_MAX_GRAPHS 128 +/* + * Maximum number of graphs tracked for version counting. + * + * There is no hard limit behind this number. An entry is 16 bytes, so the whole + * table is about 4 KB of shared memory, and it is sized for headroom. Because a + * slot is released when its graph is dropped, this bounds the graphs that exist + * at one time rather than the graphs ever created. Lookups are linear scans of + * the slots in use, which is the reason not to raise it much further without + * replacing the scan with a hash. + */ +#define AGE_MAX_GRAPHS 256 /* * Graph version counter entry. Stored in shared memory (DSM or shmem) @@ -1372,52 +1385,133 @@ Oid get_vertex_entry_label_table_oid(vertex_entry *ve) } /* - * Fetch vertex properties on demand from the heap via stored TID. + * Outcome of fetch_entry_properties(). A missing row and a row whose properties + * are NULL are different failures: the first means the cache is stale, the + * second means the label table no longer satisfies the NOT NULL that AGE + * creates it with. Reporting them apart keeps a schema problem from being + * described as a cache problem. + */ +typedef enum entry_fetch_status +{ + ENTRY_FETCH_OK, + ENTRY_FETCH_GONE, + ENTRY_FETCH_NULL_PROPS +} entry_fetch_status; + +/* + * Read one cached entry's properties out of the heap. * - * Returns a datumCopy of the properties in the current memory context. - * The caller does not need to free the result explicitly — it will be - * freed when the memory context is reset (typically the SRF multi-call - * context for VLE, which is cleaned up when the SRF completes). + * The stored TID normally resolves under the active snapshot. It does not once + * this statement has deleted the tuple: cypher_delete() advances + * es_snapshot->curcid past every delete, so a path bound by an earlier MATCH + * can no longer see its own endpoints by the time it is projected (issue + * #2549). That tuple is still physically present -- our transaction has not + * committed, so nothing may prune it -- and the properties it carried when the + * path was matched are what the path should report, so it is read anyway. + * + * The relaxation is deliberately narrow: only a tuple deleted by our own + * transaction qualifies, and only while the row still holds the entity that was + * cached, so a line pointer recycled by vacuum cannot be mistaken for the + * original. Anything else leaves *found false and the caller reports a stale + * entry, which is what keeps a genuine cache-invalidation bug visible. * - * If the tuple is no longer visible (e.g., concurrent mutation between - * cache build and fetch), the version counter should have invalidated - * the cache. If we get here with a stale TID, it indicates a bug in - * the invalidation logic. + * The value is detoasted here, under the buffer pin, so no caller is left + * holding an external pointer into a tuple that is logically gone. */ -Datum get_vertex_entry_properties(vertex_entry *ve) +static Datum fetch_entry_properties(Oid label_table_oid, ItemPointer tid, + graphid expected_id, AttrNumber id_attnum, + AttrNumber props_attnum, + entry_fetch_status *status) { Relation rel; + TupleDesc tupdesc; HeapTupleData tuple; - Buffer buffer; + Buffer buffer = InvalidBuffer; Datum result = (Datum) 0; + bool usable; + bool isnull; - rel = table_open(ve->vertex_label_table_oid, AccessShareLock); - tuple.t_self = ve->tid; + *status = ENTRY_FETCH_GONE; + + rel = table_open(label_table_oid, AccessShareLock); + tupdesc = RelationGetDescr(rel); + tuple.t_self = *tid; + + /* + * keep_buf leaves the tuple readable when the fetch fails on visibility + * alone; a line pointer that is gone clears t_data and returns no buffer. + */ + usable = heap_fetch(rel, GetActiveSnapshot(), &tuple, &buffer, true); - if (heap_fetch(rel, GetActiveSnapshot(), &tuple, &buffer, true)) + if (!usable && BufferIsValid(buffer)) { - TupleDesc tupdesc = RelationGetDescr(rel); - bool isnull; - Datum props; + TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple.t_data); - /* properties is column 2 (1-indexed) */ - props = heap_getattr(&tuple, 2, tupdesc, &isnull); - if (!isnull) + if (TransactionIdIsValid(xmax) && + TransactionIdIsCurrentTransactionId(xmax)) { - result = datumCopy(props, false, -1); + Datum id = heap_getattr(&tuple, id_attnum, tupdesc, &isnull); + + usable = !isnull && DATUM_GET_GRAPHID(id) == expected_id; + } + } + + if (usable) + { + Datum props = heap_getattr(&tuple, props_attnum, tupdesc, &isnull); + + if (isnull) + { + *status = ENTRY_FETCH_NULL_PROPS; + } + else + { + result = PointerGetDatum(PG_DETOAST_DATUM_COPY(props)); + *status = ENTRY_FETCH_OK; } + } + if (BufferIsValid(buffer)) + { ReleaseBuffer(buffer); } table_close(rel, AccessShareLock); - /* - * If heap_fetch failed, the tuple is no longer visible. This should - * not happen under normal operation because the version counter - * invalidates the cache when the graph is mutated. - */ - if (result == (Datum) 0) + return result; +} + +/* + * Fetch vertex properties on demand from the heap via stored TID. + * + * Returns a detoasted copy of the properties in the current memory context. + * The caller does not need to free the result explicitly — it will be + * freed when the memory context is reset (typically the SRF multi-call + * context for VLE, which is cleaned up when the SRF completes). + * + * A tuple this transaction has already deleted is still reported, carrying the + * properties it held when the path was matched; see fetch_entry_properties. + * Any other unreachable TID means the version counter failed to invalidate the + * cache, and is raised as an error. + */ +Datum get_vertex_entry_properties(vertex_entry *ve) +{ + Datum result; + entry_fetch_status status; + + result = fetch_entry_properties(ve->vertex_label_table_oid, &ve->tid, + ve->vertex_id, + Anum_ag_label_vertex_table_id, + Anum_ag_label_vertex_table_properties, + &status); + + if (status == ENTRY_FETCH_NULL_PROPS) + { + elog(ERROR, "get_vertex_entry_properties: vertex " INT64_FORMAT + " has null properties", ve->vertex_id); + } + + if (status != ENTRY_FETCH_OK) { elog(ERROR, "get_vertex_entry_properties: stale TID - " "vertex entry references a tuple that is no longer visible"); @@ -1451,33 +1545,22 @@ Oid get_edge_entry_label_table_oid(edge_entry *ee) */ Datum get_edge_entry_properties(edge_entry *ee) { - Relation rel; - HeapTupleData tuple; - Buffer buffer; - Datum result = (Datum) 0; + Datum result; + entry_fetch_status status; - rel = table_open(ee->edge_label_table_oid, AccessShareLock); - tuple.t_self = ee->tid; + result = fetch_entry_properties(ee->edge_label_table_oid, &ee->tid, + get_edge_entry_id(ee), + Anum_ag_label_edge_table_id, + Anum_ag_label_edge_table_properties, + &status); - if (heap_fetch(rel, GetActiveSnapshot(), &tuple, &buffer, true)) + if (status == ENTRY_FETCH_NULL_PROPS) { - TupleDesc tupdesc = RelationGetDescr(rel); - bool isnull; - Datum props; - - /* properties is column 4 (1-indexed) */ - props = heap_getattr(&tuple, 4, tupdesc, &isnull); - if (!isnull) - { - result = datumCopy(props, false, -1); - } - - ReleaseBuffer(buffer); + elog(ERROR, "get_edge_entry_properties: edge " INT64_FORMAT + " has null properties", get_edge_entry_id(ee)); } - table_close(rel, AccessShareLock); - - if (result == (Datum) 0) + if (status != ENTRY_FETCH_OK) { elog(ERROR, "get_edge_entry_properties: stale TID - " "edge entry references a tuple that is no longer visible"); @@ -1933,32 +2016,163 @@ void increment_graph_version(Oid graph_oid) } } - /* add new entry */ - if (state->num_entries < AGE_MAX_GRAPHS) + /* take a slot, preferring one left behind by a dropped graph */ { - int idx = state->num_entries; + int idx = -1; + bool appending; - state->entries[idx].graph_oid = graph_oid; - pg_atomic_init_u64(&state->entries[idx].version, 1); + for (i = 0; i < state->num_entries; i++) + { + if (state->entries[i].graph_oid == InvalidOid) + { + idx = i; + break; + } + } + + if (idx < 0 && state->num_entries < AGE_MAX_GRAPHS) + { + idx = state->num_entries; + } + + if (idx < 0) + { + elog(WARNING, "AGE: graph version counter table full (%d graphs)", + AGE_MAX_GRAPHS); + LWLockRelease(&state->lock); + return; + } + + appending = (idx == state->num_entries); + + /* + * Seed above every version this table has ever issued, freed slots + * included, so the sequence never repeats a value. A context cached + * against this slot's previous occupant -- or against an earlier graph + * that happened to reuse this OID -- then cannot compare equal by + * coincidence and be mistaken for current. + */ + { + uint64 seed = 0; + int j; + + for (j = 0; j < state->num_entries; j++) + { + uint64 v = pg_atomic_read_u64(&state->entries[j].version); + + if (v > seed) + { + seed = v; + } + } + + if (appending) + { + pg_atomic_init_u64(&state->entries[idx].version, seed + 1); + } + else + { + pg_atomic_write_u64(&state->entries[idx].version, seed + 1); + } + } /* - * Write barrier ensures the entry fields are fully visible to - * other backends before num_entries is incremented. This prevents - * readers on weak memory-ordering architectures (e.g., ARM) from - * seeing the incremented count before the entry is initialized. + * Publish the version before the oid: readers match on the oid without + * the lock and must never find a slot whose version is not yet set. */ pg_write_barrier(); - state->num_entries++; + state->entries[idx].graph_oid = graph_oid; + + if (appending) + { + /* + * Write barrier ensures the entry fields are fully visible to + * other backends before num_entries is incremented. This prevents + * readers on weak memory-ordering architectures (e.g., ARM) from + * seeing the incremented count before the entry is initialized. + */ + pg_write_barrier(); + state->num_entries++; + } } - else + + LWLockRelease(&state->lock); +} + +/* + * Release a dropped graph's slot so another graph can use it. + * + * Without this the table is a tally of every graph ever mutated, and a server + * that cycles graphs eventually fills it: further graphs go untracked, every + * mutation warns, and their contexts fall back to snapshot comparison, which is + * correct but invalidates far more often. + * + * The version is deliberately left in the freed slot. It is part of the + * high-water mark the next occupant seeds above, which is what stops a stale + * context from matching a later graph. + */ +void release_graph_version(Oid graph_oid) +{ + GraphVersionState *state = get_version_state(); + int i; + + if (state == NULL || !OidIsValid(graph_oid)) + { + return; + } + + LWLockAcquire(&state->lock, LW_EXCLUSIVE); + + for (i = 0; i < state->num_entries; i++) { - elog(WARNING, "AGE: graph version counter table full (%d graphs)", - AGE_MAX_GRAPHS); + if (state->entries[i].graph_oid == graph_oid) + { + /* + * Clearing the oid is enough to retire the slot: a lock-free + * reader matches on it, so it stops finding this graph and falls + * back to snapshot comparison until the graph is registered again. + */ + state->entries[i].graph_oid = InvalidOid; + break; + } } LWLockRelease(&state->lock); } +/* + * Bump the version of every tracked graph. + * + * For commands that rewrite a heap without naming one: VACUUM FULL or CLUSTER + * over a whole database. A graph absent from this table has never been mutated + * through the counter, so its contexts are still validated by snapshot + * comparison in is_ggctx_invalid() and need no bump. + */ +void increment_all_graph_versions(void) +{ + GraphVersionState *state = get_version_state(); + int i; + + if (state == NULL) + { + return; + } + + /* + * num_entries only grows, and entries are published with a write barrier + * before it is incremented, so reading it without the lock can miss a + * brand-new graph but never sees a half-built entry. A graph added after + * this read has no cached context to invalidate yet. + */ + for (i = 0; i < state->num_entries; i++) + { + if (state->entries[i].graph_oid != InvalidOid) + { + pg_atomic_fetch_add_u64(&state->entries[i].version, 1); + } + } +} + /* * Helper function to look up the graph OID for a given label table OID. * Uses AGE's label relation cache for fast lookup. diff --git a/src/include/utils/age_global_graph.h b/src/include/utils/age_global_graph.h index d68530a91..b08b0d18f 100644 --- a/src/include/utils/age_global_graph.h +++ b/src/include/utils/age_global_graph.h @@ -96,6 +96,8 @@ graphid get_edge_entry_end_vertex_id(edge_entry *ee); /* Graph version counter functions — shared memory (DSM or shmem) */ uint64 get_graph_version(Oid graph_oid); void increment_graph_version(Oid graph_oid); +void increment_all_graph_versions(void); +void release_graph_version(Oid graph_oid); Oid get_graph_oid_for_table(Oid table_oid); /*