An attribute whose range is xsd:hexBinary is imported as text - #1008
Conversation
Signed-off-by: dada-yan <BinjunYann@gmail.com>
WaylandYang
left a comment
There was a problem hiding this comment.
Approving. Thanks @Maya-Kid for picking this up from the #1001 review.
Reviewed at d14140d. The line decision 20 draws is whether a value can appear in a sentence, and a hash or an address does; base64Binary stays skipped for the reason the comment already gives. The test now pins both sides, and the record carries the revision.
CI did not run on this PR (the CI workflow is not registered on the repository at the moment, which we are looking into), so I ran the backend steps locally on this commit: cargo fmt --check and clippy with -D warnings are clean, the migrations run twice, and cargo test --workspace against Postgres gives 1304 passed, 0 failed. Two chat stop tests timed out on the first run while the machine was compiling and passed on the second; they are not touched by this change.
It will be merged once CI is back and green.
|
Closing and reopening to trigger CI: the workflow did not run when this was opened. Nothing is wrong with the PR. |
Follows the #1001 review, which pointed at the 13 UCO attributes still skipped because their range is
xsd:hexBinaryorxsd:base64Binary.Problem
hexBinaryamong them. The dividing line it states is "can this value appear in a sentence".hexBinaryis written as hex digits. In prose it carries a hash, an address or a flag, for example "The file's SHA-256 is 9f86d0…" or "the region starts at 7FFE0000".types:hashValue,observable:androidID, the region start and end addresses, and the ICMP type and code. On the merged UCO 1.5.0 all 12 are skipped at import asunusable_range, although a forensic report states them in sentences.Change
crates/utopia-ingest/src/ontology_rdf.rs,datatype_ofmapsxsd:hexBinarytotext, alongsideanyURIandtoken: the value is kept as it is written.unusableno longer lists it, and its doc comment says why.base64Binarystays skipped, as doXMLLiteraland the XML identifiers.base64Binaryattribute,observable:byteStringValue("the raw, byte-string representation of the extracted string"), is that kind of payload.docs/decisions/0001-ontology-import-and-governance.md, decision 20 moveshexBinaryout of the skipped list, and a revision entry records the change.Verification
Run in containers on a Linux host: Rust 1.98.1 and PostgreSQL 16 with pgvector.
a_value_we_can_store_is_kept_even_when_we_cannot_type_itnow checks thathexBinarymaps totext, and thatbase64Binary,XMLLiteralandQNamestay unusable. On dev with only the test changed, it fails at thehexBinarycheck.cargo fmt --all --checkandcargo clippy --locked --workspace --all-targets -- -D warningsare clean.UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspacepasses: 1304 passed, 0 failed.UCO 1.5.0 was run through the import plan that the preview returns, on an empty base, as measured for #1001:
byteStringValue)core:informalType)Everything else in the plan is unchanged. None of the five bundled packs uses
hexBinaryorbase64Binary.