Skip to content

Restore Variant object field offsets after failed append (v59) - #7

Open
osipovartem wants to merge 1 commit into
variant-get-nested-extensionfrom
variant-object-rollback-v59
Open

osipovartem wants to merge 1 commit into
variant-get-nested-extensionfrom
variant-object-rollback-v59

Conversation

@osipovartem

Copy link
Copy Markdown

Backport of the standalone upstream fix in apache#11403 to the Arrow revision used by Rustice. Validated duplicate keys are rejected before mutating the field map, and a failed permitted overwrite restores the previous field offset. The default non-validating path keeps its original fields.insert operation.

Validation: 159 parquet-variant and 333 parquet-variant-compute unit tests pass; format and library Clippy pass (one unrelated Arrow warning). Quick builder benchmarks varied substantially between runs under WSL; the representative default path keeps the original single map insertion. Independent read-only review approved correctness, regressions, performance approach, API compatibility and test coverage.

Follow-up: update Rustice's pinned Arrow revision after merge.

@osipovartem

Copy link
Copy Markdown
Author

CI note: this PR changes only parquet-variant/src/builder/object.rs. The failing jobs are in unchanged files: cargo fmt reports parquet/src/encodings/decoding.rs, rustdoc reports arrow-avro redundant links, and the Linux/macOS/Windows test jobs fail parquet/tests/arrow_reader/bad_data.rs::non_standard_delta_blocks (expected an error, got a batch). Focused local parquet-variant tests and clippy pass. These failures appear unrelated to this one-file change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant