fix(honeydew): honor a field's declared datatype in Ossie → Honeydew - #419
Draft
baruchoxman wants to merge 1 commit into
Draft
baruchoxman wants to merge 1 commit into
baruchoxman wants to merge 1 commit into
Conversation
_ossie_field_to_honeydew_datatype derived the Honeydew datatype only from the shape of `dimension`, so a `Date` field became `number` and a `Boolean` became `string`. The heuristic dates back to the converter's first commit, when the spec had no `datatype` field at all; apache#113 added the enum and the converter was never updated. Map the core-spec datatype enum onto Honeydew's vocabulary (bool, date, float, number, string, time, timestamp) and consult it right after the HONEYDEW round-trip extension. `Opaque`, absent and unrecognized values keep falling back to the dimension-shape heuristic, so models that declare no datatype convert exactly as before. Honeydew's `number` is integral and `float` is approximate, so `Float` and `Decimal` both map to `float`; `number` would claim they are integral. Fixes apache#411 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #411.
_ossie_field_to_honeydew_datatypederived the Honeydew datatype only from the shape ofdimension, so aDatefield becamenumberand aBooleanbecamestring— the declareddatatypehad zero effect on the output. The heuristic dates back to the converter's first commit, when the spec had nodatatypefield; #113 added the enum and the converter was never updated.What changed
converter.py: added_OSSIE_TO_HONEYDEW_DATATYPE, mapping the core-specdatatypesenum onto Honeydew's vocabulary._ossie_field_to_honeydew_datatypenow consults, in order: the round-trippedHONEYDEWextension datatype → the field's declareddatatype→ the existingdimension-shape heuristic.Opaque, absent, and unrecognized values fall through to the heuristic, so models that declare no datatype convert exactly as before.README.md: replaced the now-false "Ossie fields have no explicit datatype" limitation with the actual mapping, and documented the two limitations that remain (see below).Opaque, unrecognized, non-string), extension-vs-declared precedence, an end-to-end file-content case, and a_fields_to_honeydewcase covering both source and calculated attributes.Mapping notes
Two deliberate deviations from the mapping suggested in the issue, both per Honeydew's attribute schema, where
numbermeans integer andfloatmeans floating point:Float→float(notnumber) —numberwould claim it is integral.Decimal→float(notnumber) — Honeydew has no exact-decimal type, andDecimalis not integral either, sofloatis the closer approximation.Per the spec,
dimension.is_timeis a role flag independent of type, so a year-grain field (datatype: Integer,is_time: true) now yieldsnumberrather thantimestamp. That is the intended consequence of letting the declared type win.Out of scope
metric.datatypeis still ignored — metrics are hardcoded tonumber. Same bug class, but Ossie->Honeydew converter ignores a field's explicitdatatype#411 scopes to fields, and Honeydew metrics have notimetype. Documented as a limitation in the README.test_ossie_roundtrip_tpcds_examplefails on this branch, and identically on unmodifiedmain— this converter still expects thesemantic_model:wrapper that Define one semantic model per document without a wrapper #383 removed. That is Update remaining converters for the flat (one-model-per-document) Ossie schema #418, and fix(honeydew): read and write flat semantic model documents #420 fixes it for this converter. fix(honeydew): read and write flat semantic model documents #420 is independent of this PR and should merge first; once it lands I will rebase this branch onmainand that failure disappears. Until then this PR's CI shows that one red test, inherited frommain. The two branches merge cleanly in either order — verified by cherry-picking this commit onto fix(honeydew): read and write flat semantic model documents #420's branch: 145 passed.Test plan
cd converters/honeydew && uv sync && uv run pytest— 147 passed, 1 pre-existing failure (test_ossie_roundtrip_tpcds_example, red onmaintoo, tracked by Update remaining converters for the flat (one-model-per-document) Ossie schema #418)🤖 Generated with Claude Code