Skip to content

Preserve identity timestamp partition values - #98

Merged
osipovartem merged 2 commits into
embucket-sync-df55.0.0from
fix/partition-identity-timestamptz
Oct 9, 2026
Merged

osipovartem merged 2 commits into
embucket-sync-df55.0.0from
fix/partition-identity-timestamptz

Conversation

@osipovartem

Copy link
Copy Markdown

Summary

  • Accept Arrow microsecond timestamp identity partition keys and emit Iceberg Timestamp or TimestampTZ values according to timezone metadata.
  • Reject nullable partition batches before splitting so NULL rows are not silently dropped. Full NULL partition writing remains a follow-up.
  • Add focused unit tests for both timestamp kinds and nullable Int64/timestamp fail-closed behavior.
  • Add an ignored reproducible key-extraction microbenchmark.

Validation

  • Focused iceberg-rust partition tests: 2 passed.
  • cargo +1.95.0 clippy -p iceberg-rust --lib --tests and fmt passed.
  • Rustice end-to-end LTZ YEAR/identity catalog-reload test passed with this fork revision; eight existing PARTITION BY regressions and partition-filter test passed.
  • Release key-extraction microbenchmark (100k rows, 1024 keys, 100 iterations): Int64 95.2M rows/s; timestamptz 94.6M rows/s. This is not full write-path or before/after throughput.
  • Independent read-only review approved correctness, common-path cost, API compatibility, and tests after resolving its nullable-row finding.

@osipovartem
osipovartem merged commit 988ea41 into embucket-sync-df55.0.0 Oct 9, 2026
2 checks passed
@osipovartem
osipovartem deleted the fix/partition-identity-timestamptz branch October 9, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant