tests: ensure type consistency per spec - #3719
jayceslesar wants to merge 2 commits into
Conversation
| StatsAggregator(iceberg_type, physical_type_string) | ||
|
|
||
|
|
||
| def test_iceberg_types_write_spec_compliant_parquet_types(tmp_path: Path) -> None: |
There was a problem hiding this comment.
You've got a test below to fail if pyarrow introduces any types not in the test. Can you do something similar if we introduce a type and it's not in the list?
| missing = { | ||
| name | ||
| for name in dir(pa.lib) | ||
| if name.startswith("Type_") and name not in unconstructible and getattr(pa.lib, name) not in covered_type_ids |
There was a problem hiding this comment.
I really love the idea of this test. I worry that this is going to be too brittle and relies on the Pyarrow directory / file structure remaining as-is.
There was a problem hiding this comment.
what if we just remove this test? i dont really like it no matter how its implemented hahahaha
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
This branch has not been deployed
2 participants
Closes #3714