fix(trino): don't ingest an unset column comment as an empty description - #31626
fix(trino): don't ingest an unset column comment as an empty description#31626harshsoni2024 wants to merge 1 commit into
Conversation
Trino's SHOW COLUMNS reports a column with no comment as the empty string rather than NULL, and reports an explicit COMMENT '' the same way, so the two are indistinguishable at that layer. _get_columns passed the value through unchanged, so every uncommented column was ingested with description: "" instead of no description at all. Normalize '' to None so the sink's exclude_none drops the field, and apply the same normalization to the table comment, since TRINO_TABLE_COMMENTS filters is not null but not COMMENT ''. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Code Review ✅ ApprovedNormalizes empty string column and table comments to None in the Trino connector so unset comments are omitted from the payload instead of ingesting as empty descriptions. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
| TRINO_TABLE_COMMENTS.format(catalog_name=catalog_name, schema_name=schema), | ||
| ) | ||
| return {"text": self.all_table_comments.get((table_name, schema))} | ||
| return {"text": self.all_table_comments.get((table_name, schema)) or None} |
There was a problem hiding this comment.
Table normalization lacks coverage
The new empty-string normalization in get_table_comment is not exercised by the accompanying tests, so a regression that restores blank table descriptions would not be detected alongside the equivalent column cases.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
✅ Playwright Results — workflow succeededValidated commit ✅ 110 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ✅ met · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 46m 15s ⏱️ Max setup 3m 11s · max shard execution 12m 11s · max shard-job elapsed before upload 18m 37s · reporting 4s 🌐 212.99 requests/attempt · 1.79 app boots/UI scenario · 0.00% common-shard skew Optimization targets still in progress:
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
🚦 Removed from the merge queue —
|



Describe your changes:
Fixes #31544
I worked on the connector half of Trino: empty column comments ingested as "", and updated column comments ignored with overrideMetadata=true #31544: Trino columns with no comment were
ingested with an empty-string description.
Trino's
SHOW COLUMNSreports a column with no comment as the empty string,never
NULL— and it reports an explicitCOMMENT ''identically, so the twoare indistinguishable at that layer.
_get_columnspassedrecord.Commentstraight through, and
sql_column_handlermaps it ontodescription, so everyuncommented column landed in OpenMetadata with
"description": ""rather than nodescription at all.
Normalizing
''toNoneis enough to fix it: the sink serializes withexclude_none=True, so the field is now omitted from the payload entirely.Applied the same normalization to the table comment in
get_table_comment.TRINO_TABLE_COMMENTSfilters"comment" is not nullbut notCOMMENT '', so atable commented with an empty string had the same problem.
Why this matters beyond cosmetics: a blank description counts as empty on the
server side, so it is the one state a bot PUT is allowed to overwrite. That made
the sync look functional — an uncommented column would accept its first comment
— while a column with any real description silently stopped updating. The
server-side half of that behaviour is fixed separately on branch
trino_column_description_update_issue; both are needed to close Trino: empty column comments ingested as "", and updated column comments ignored with overrideMetadata=true #31544.Type of change:
High-level design:
N/A — small change.
Tests:
Use cases covered
rather than an empty-string description
COMMENT ''is likewise ingested with nodescription
Unit tests
ingestion/tests/unit/topology/database/test_trino_metadata.py—new
TestTrinoColumnCommentsclass covering_get_columns:test_unset_comment_is_none—''→Nonetest_populated_comment_is_preserved— real comment unchangedtest_null_comment_is_none—None→Nonepytest ingestion/tests/unit/topology/database/test_trino_metadata.py→16 passed.
ruff checkandruff format --checkclean on both changed files.Backend integration tests
Ingestion integration tests
end-to-end run below. Trino integration tests require a live Trino instance.
Playwright (UI) tests
Manual testing performed
Local Trino (
memorycatalog) against a local OpenMetadata stack.metadata ingest -c trino.yamland fetched the columns from the API.ageandstudent_data.idcame back as"description": "".descriptionfield, whileid,created_atand
namekeep their comment text unchanged.UI screen recording / screenshots:
Not applicable.
Checklist:
Fixes <issue-number>: <short explanation>Fixes #31544above.Greptile Summary
The PR normalizes empty Trino column and table comments to
None, preventing blank descriptions from being emitted during ingestion.SHOW COLUMNScomments to absent column descriptions while preserving populated comments.Confidence Score: 4/5
The PR appears safe to merge, with only a non-blocking test-coverage gap for the new table-comment normalization.
The normalization matches Trino’s string-or-null comment contract and existing connector conventions; the only accepted concern is that the parallel table-comment behavior is not directly tested.
Files Needing Attention: ingestion/src/metadata/ingestion/source/database/trino/metadata.py, ingestion/tests/unit/topology/database/test_trino_metadata.py
Important Files Changed
Reviews (1): Last reviewed commit: "fix(trino): don't ingest an unset column..." | Re-trigger Greptile
Context used: