Skip to content

HDDS-14942. Implement manifest selection logic for rewrite based on snapshot delta#10145

Open
sreejasahithi wants to merge 1 commit intoapache:masterfrom
sreejasahithi:HDDS-14942
Open

HDDS-14942. Implement manifest selection logic for rewrite based on snapshot delta#10145
sreejasahithi wants to merge 1 commit intoapache:masterfrom
sreejasahithi:HDDS-14942

Conversation

@sreejasahithi
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR provides logic to determine the specific subset of Iceberg manifest files that require path rewriting, avoiding redundant processing of manifests.

  • Compute delta snapshots as the difference between the start and end table metadata versions. If no start version is provided, all snapshots are treated as delta.
  • Iterate over all snapshots in the end-version table and read each snapshot's manifest list in parallel.
  • If no start metadata is provided, include all manifests unconditionally.
  • If start metadata is provided, filter at the manifest level — only include manifests whose snapshotId belongs to the delta snapshot ID set.
  • Deduplicate manifest paths so that manifests shared across multiple snapshots are only collected once.

What is the link to the Apache JIRA

HDDS-14942

How was this patch tested?

https://github.com/sreejasahithi/ozone/actions/runs/24986534925

@sreejasahithi sreejasahithi marked this pull request as ready for review April 28, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant