Skip to content

An entity tag a writer cannot read the graph from - #393

Merged
namedgraph merged 1 commit into
developfrom
ft-opaque-validator
Sep 24, 2026
Merged

namedgraph merged 1 commit into
developfrom
ft-opaque-validator

Conversation

@namedgraph

Copy link
Copy Markdown
Member

Follow-up to #392. Making writes conditional meant handing validators to agents who may not read what they describe: HEAD is answered for any access mode, so that an agent with acl:Append and no acl:Read — a dropbox depositor, which is what the ACL vocabulary recommends the mode for — can obtain the tag its writes have to quote.

The tag it gets is an XOR fold of a hash per triple, and XOR is linear.

The problem

hash(G ∪ {t}) = hash(G) ⊕ hashTriple(t)

hashTriple is computable offline, and a set gains nothing when you add a member it already has. So a depositor appends a triple, reads the tag the write answers with, and learns whether that triple was already in a document it cannot read — one bit per request, using only the capability it was granted:

HEAD /inbox/                                     → 200  ETag h0
POST /inbox/  <urn:uuid:n1> … .                  → 204  ETag h_A   calibrate
POST /inbox/  <urn:uuid:n2> … .                  → 204  ETag h_B   calibrate
POST /inbox/  <#review> <#outcome> "approved" .  → 204  ETag h_C

h_C == h_B ⊕ hT(mod_B) ⊕ hT(mod_C) ⊕ hashTriple(t)   ⇒ absent
h_C == h_B ⊕ hT(mod_B) ⊕ hT(mod_C)                    ⇒ present

The calibration steps cancel the dct:modified each POST restamps. On /settings it is cleaner still — PATCH stamps no timestamp, so h₁ == h₀ ⊕ hashTriple(t) exactly.

Limits, stated plainly: one triple per probe, guessed exactly; no interleaving writer without re-calibration; the probe triples stay in the document, so it is noisy and auditable.

The change

SHA-256(graph URI ‖ sorted N-Triples), truncated to 128 bits of lowercase hex. Only the function changes:

before after
derived from content yes yes
strong validator (If-Match needs strong comparison) yes yes
identical content ⇒ identical tag yes yes
detects writes that bypass the resource yes yes
no stored state, no secret yes yes
hash(G ∪ {t}) solvable from hash(G) yes no
identical content in two documents ⇒ same tag yes no
collision-resistant no yes

That last row matters beyond confidentiality: two graphs sharing a tag means If-Match can pass against content that changed — the lost update this mechanism exists to prevent.

Why not a stored revision (CouchDB _rev, Kubernetes resourceVersion), which is the idiomatic choice elsewhere: If-Match requires a strong validator; LDH has writes that bypass the document resource (PackageService, the test harness) so a revision would desynchronise; and there is nowhere to keep one — no store for per-document server state exists.

Two sites, from an audit

AuthorizationFilter is @PreMatching and global, so the broadened HEAD reaches every resource. The question per resource is not where a tag is produced but where a non-reader can move one:

resource mutable by a non-reader changed
DocumentHierarchyGraphStoreImpl yes — POST is acl:Append yes
Settings yes — PATCH is acl:Write, GET is acl:Read yes
acl/Access no — zero write methods no
SPARQLEndpointImpl, ProxyRequestFilter n/a — whoever sees the tag can read the content no

Also here

toSortedNTriples moves out of GraphVersioningService (which delegates, so its behaviour and existing test are unchanged) — a validator depending on a GitHub-mirroring service reads wrong.

PackageService now skolemizes. It writes straight to the graph store, to avoid a self-request deadlock from inside OntologyFilter, and was the one write path not upholding the invariant that the data LDH writes is blank-node-free. That invariant is what makes a sorted serialization canonical: Jena's _:bN labels are not stable across reads, so a stored blank node would give its document a different tag on every read and no conditional request against it could succeed. Measured zero blank-node triples in both stores — but the local package's vocabulary happens to be free of them; one delivering an owl:Restriction or an rdf:List would not have been.

Tests

EntityTagsTest asserts the two properties an HTTP test cannot construct: two graphs with identical triples and different URIs do not share a tag, and

tag(G+a) ⊕ tag(G+b) ≠ tag(G) ⊕ tag(G+a+b)

the identity a linear hash satisfies and a digest does not — so it fails against the old code.

POST-blank-node-skolemized.sh covers the one write path of three that had none. no-blank-nodes-in-store.sh asks the store rather than enumerating write paths, which is how PackageService was missed in the first place; its limitation (it only sees what earlier tests left behind) is in the test.

Verification

All 17 entity-tag http-tests pass unchanged — conditional GET, conditional PUT/POST/DELETE and their 412s, the language dimension, proxied tags byte-identical to direct ones, settings, preconditions. UI suite 104 passed / 0 failed. The http failure set is identical to the pre-change baseline; the tally moved 25 → 27 only because the baseline ran on a branch lacking the accumulator fix and under-reported by exactly the two imports failures.

Deliberate gaps

  • No ETag-opaque.sh: its properties are unconstructible over HTTP (two documents cannot have identical content — their URIs are in their own triples) or need offline hashTriple arithmetic. The unit test asserts them exactly.
  • No test for PackageService's skolemization: the ontology resolves through Jena's document manager, so a blank-node fixture must be served with blank nodes. The fix is in; no-blank-nodes-in-store.sh catches a recurrence once such a package is imported.
  • Core and Web-Client follow. ModelUtils.hashModel and ResultSetUtils.hashResultSet are still the XOR fold, inherited by ExceptionMapperBase and ProxyRequestFilter. Fixing there also changes every deployed consumer's ETags once, so it wants its own release.

Making writes conditional meant handing validators to agents who may not read what they describe: HEAD is answered for any access mode so that an agent with acl:Append and no acl:Read - a dropbox depositor, which is what the ACL vocabulary recommends the mode for - can obtain the tag its writes must quote. The tag it gets is a XOR fold of a hash per triple, and XOR is linear:

    hash(G + t) = hash(G) XOR hashTriple(t)

hashTriple is computable offline, and a set gains nothing when you add a member it already has. So a depositor appends a triple, reads the tag the write answers with, and learns whether that triple was already in a document it cannot read. One bit per request, using only the capability it was granted. On /settings it is cleaner still: PATCH stamps no dct:modified, so there is not even a timestamp to cancel out - h1 == h0 XOR hashTriple(t) exactly.

The tag is now a SHA-256 digest of the graph URI and a sorted N-Triples serialization, truncated to 128 bits of lowercase hex. What changes is only the function:

- the URI is digested with the content, so identical triples in two documents no longer share a tag and a guess cannot be materialized somewhere readable and compared;
- the digest is not linear, so the difference between two tags says nothing about what was added;
- and it no longer collides the way an XOR fold over an overlapping-shift per-triple mix does, which mattered beyond confidentiality: two different graphs sharing a tag means If-Match can pass against content that changed, which is the lost update this mechanism exists to prevent.

Everything that made a content hash the right shape for RDF caching is kept: still derived from the content, still strong as If-Match requires, still identical for identical content, still sensitive to writes that bypass the resource, still no stored state and no secret. A stored revision - CouchDB's _rev, Kubernetes' resourceVersion - would be the idiomatic choice elsewhere, but not here: it would desynchronize the moment PackageService or a test harness writes straight to the graph store, and there is nowhere to keep one.

Two sites, established by audit rather than assumption. AuthorizationFilter is @PreMatching and global, so the broadened HEAD reaches every resource, and the question for each is not where a tag is produced but where a NON-READER can move one. Documents can (POST is acl:Append) and /settings can (PATCH is acl:Write while GET is acl:Read). acl/Access has no write method and SPARQL results are only seen by agents who could read them anyway, so both keep theirs.

toSortedNTriples moves out of GraphVersioningService, which keeps working by delegation: a validator depending on a GitHub-mirroring service reads wrong, and its existing test already pins the property the tag needs - two models with the same triples serialize identically.

PackageService now skolemizes. It writes straight to the graph store, to avoid a self-request deadlock from inside OntologyFilter, and was the one write path not upholding the invariant that the data LDH writes is blank-node-free. That invariant is what makes a sorted serialization canonical at all: Jena's _:bN labels are not stable across reads, so a stored blank node would give its document a different tag on every read and no conditional request against it could ever succeed. Measured zero blank-node triples in both stores, but the local package's vocabulary happens to be free of them - one delivering an owl:Restriction or an rdf:List would not have been.

Tests: EntityTagsTest asserts the two properties an HTTP test cannot construct - two graphs with identical triples and different URIs do not share a tag, and tag(G+a) XOR tag(G+b) != tag(G) XOR tag(G+a+b), the identity a linear hash satisfies and a digest does not. POST-blank-node-skolemized.sh covers the one write path of three that had no test. no-blank-nodes-in-store.sh asks the store instead of enumerating paths, which is how PackageService was missed in the first place.

All 17 entity-tag tests pass unchanged - conditional GET, conditional PUT/POST/DELETE and their 412s, the language dimension, proxied tags byte-identical to direct ones, settings, and the precondition tests - as do the UI suite and the four skolemization tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@namedgraph
namedgraph merged commit 32e074c into develop Sep 24, 2026
2 checks passed
@namedgraph
namedgraph deleted the ft-opaque-validator branch September 24, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant