An entity tag a writer cannot read the graph from - #393
Merged
Merged
Conversation
Making writes conditional meant handing validators to agents who may not read what they describe: HEAD is answered for any access mode so that an agent with acl:Append and no acl:Read - a dropbox depositor, which is what the ACL vocabulary recommends the mode for - can obtain the tag its writes must quote. The tag it gets is a XOR fold of a hash per triple, and XOR is linear:
hash(G + t) = hash(G) XOR hashTriple(t)
hashTriple is computable offline, and a set gains nothing when you add a member it already has. So a depositor appends a triple, reads the tag the write answers with, and learns whether that triple was already in a document it cannot read. One bit per request, using only the capability it was granted. On /settings it is cleaner still: PATCH stamps no dct:modified, so there is not even a timestamp to cancel out - h1 == h0 XOR hashTriple(t) exactly.
The tag is now a SHA-256 digest of the graph URI and a sorted N-Triples serialization, truncated to 128 bits of lowercase hex. What changes is only the function:
- the URI is digested with the content, so identical triples in two documents no longer share a tag and a guess cannot be materialized somewhere readable and compared;
- the digest is not linear, so the difference between two tags says nothing about what was added;
- and it no longer collides the way an XOR fold over an overlapping-shift per-triple mix does, which mattered beyond confidentiality: two different graphs sharing a tag means If-Match can pass against content that changed, which is the lost update this mechanism exists to prevent.
Everything that made a content hash the right shape for RDF caching is kept: still derived from the content, still strong as If-Match requires, still identical for identical content, still sensitive to writes that bypass the resource, still no stored state and no secret. A stored revision - CouchDB's _rev, Kubernetes' resourceVersion - would be the idiomatic choice elsewhere, but not here: it would desynchronize the moment PackageService or a test harness writes straight to the graph store, and there is nowhere to keep one.
Two sites, established by audit rather than assumption. AuthorizationFilter is @PreMatching and global, so the broadened HEAD reaches every resource, and the question for each is not where a tag is produced but where a NON-READER can move one. Documents can (POST is acl:Append) and /settings can (PATCH is acl:Write while GET is acl:Read). acl/Access has no write method and SPARQL results are only seen by agents who could read them anyway, so both keep theirs.
toSortedNTriples moves out of GraphVersioningService, which keeps working by delegation: a validator depending on a GitHub-mirroring service reads wrong, and its existing test already pins the property the tag needs - two models with the same triples serialize identically.
PackageService now skolemizes. It writes straight to the graph store, to avoid a self-request deadlock from inside OntologyFilter, and was the one write path not upholding the invariant that the data LDH writes is blank-node-free. That invariant is what makes a sorted serialization canonical at all: Jena's _:bN labels are not stable across reads, so a stored blank node would give its document a different tag on every read and no conditional request against it could ever succeed. Measured zero blank-node triples in both stores, but the local package's vocabulary happens to be free of them - one delivering an owl:Restriction or an rdf:List would not have been.
Tests: EntityTagsTest asserts the two properties an HTTP test cannot construct - two graphs with identical triples and different URIs do not share a tag, and tag(G+a) XOR tag(G+b) != tag(G) XOR tag(G+a+b), the identity a linear hash satisfies and a digest does not. POST-blank-node-skolemized.sh covers the one write path of three that had no test. no-blank-nodes-in-store.sh asks the store instead of enumerating paths, which is how PackageService was missed in the first place.
All 17 entity-tag tests pass unchanged - conditional GET, conditional PUT/POST/DELETE and their 412s, the language dimension, proxied tags byte-identical to direct ones, settings, and the precondition tests - as do the UI suite and the four skolemization tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #392. Making writes conditional meant handing validators to agents who may not read what they describe:
HEADis answered for any access mode, so that an agent withacl:Appendand noacl:Read— a dropbox depositor, which is what the ACL vocabulary recommends the mode for — can obtain the tag its writes have to quote.The tag it gets is an XOR fold of a hash per triple, and XOR is linear.
The problem
hashTripleis computable offline, and a set gains nothing when you add a member it already has. So a depositor appends a triple, reads the tag the write answers with, and learns whether that triple was already in a document it cannot read — one bit per request, using only the capability it was granted:The calibration steps cancel the
dct:modifiedeach POST restamps. On/settingsit is cleaner still —PATCHstamps no timestamp, soh₁ == h₀ ⊕ hashTriple(t)exactly.Limits, stated plainly: one triple per probe, guessed exactly; no interleaving writer without re-calibration; the probe triples stay in the document, so it is noisy and auditable.
The change
SHA-256(graph URI ‖ sorted N-Triples), truncated to 128 bits of lowercase hex. Only the function changes:If-Matchneeds strong comparison)hash(G ∪ {t})solvable fromhash(G)That last row matters beyond confidentiality: two graphs sharing a tag means
If-Matchcan pass against content that changed — the lost update this mechanism exists to prevent.Why not a stored revision (CouchDB
_rev, KubernetesresourceVersion), which is the idiomatic choice elsewhere:If-Matchrequires a strong validator; LDH has writes that bypass the document resource (PackageService, the test harness) so a revision would desynchronise; and there is nowhere to keep one — no store for per-document server state exists.Two sites, from an audit
AuthorizationFilteris@PreMatchingand global, so the broadenedHEADreaches every resource. The question per resource is not where a tag is produced but where a non-reader can move one:DocumentHierarchyGraphStoreImplPOSTisacl:AppendSettingsPATCHisacl:Write,GETisacl:Readacl/AccessSPARQLEndpointImpl,ProxyRequestFilterAlso here
toSortedNTriplesmoves out ofGraphVersioningService(which delegates, so its behaviour and existing test are unchanged) — a validator depending on a GitHub-mirroring service reads wrong.PackageServicenow skolemizes. It writes straight to the graph store, to avoid a self-request deadlock from insideOntologyFilter, and was the one write path not upholding the invariant that the data LDH writes is blank-node-free. That invariant is what makes a sorted serialization canonical: Jena's_:bNlabels are not stable across reads, so a stored blank node would give its document a different tag on every read and no conditional request against it could succeed. Measured zero blank-node triples in both stores — but the local package's vocabulary happens to be free of them; one delivering anowl:Restrictionor anrdf:Listwould not have been.Tests
EntityTagsTestasserts the two properties an HTTP test cannot construct: two graphs with identical triples and different URIs do not share a tag, andthe identity a linear hash satisfies and a digest does not — so it fails against the old code.
POST-blank-node-skolemized.shcovers the one write path of three that had none.no-blank-nodes-in-store.shasks the store rather than enumerating write paths, which is howPackageServicewas missed in the first place; its limitation (it only sees what earlier tests left behind) is in the test.Verification
All 17 entity-tag http-tests pass unchanged — conditional GET, conditional PUT/POST/DELETE and their 412s, the language dimension, proxied tags byte-identical to direct ones, settings, preconditions. UI suite 104 passed / 0 failed. The http failure set is identical to the pre-change baseline; the tally moved 25 → 27 only because the baseline ran on a branch lacking the accumulator fix and under-reported by exactly the two
importsfailures.Deliberate gaps
ETag-opaque.sh: its properties are unconstructible over HTTP (two documents cannot have identical content — their URIs are in their own triples) or need offlinehashTriplearithmetic. The unit test asserts them exactly.PackageService's skolemization: the ontology resolves through Jena's document manager, so a blank-node fixture must be served with blank nodes. The fix is in;no-blank-nodes-in-store.shcatches a recurrence once such a package is imported.ModelUtils.hashModelandResultSetUtils.hashResultSetare still the XOR fold, inherited byExceptionMapperBaseandProxyRequestFilter. Fixing there also changes every deployed consumer's ETags once, so it wants its own release.