Core: Fix ManifestFilterManager canContainDroppedFiles - #18197
Open
grantatspothero wants to merge 2 commits into
Open
grantatspothero wants to merge 2 commits into
grantatspothero wants to merge 2 commits into
Conversation
Orphaned DVs were sometimes never removed in ManifestFilterManager. canContainDroppedFiles only returned true for a delete manifest whose minSequenceNumber was below the dropDeleteFilesOlderThan threshold when data files were also being removed in the same commit. A commit that only calls dropDeleteFilesOlderThan (e.g. a plain append) never admitted delete manifests for that check.
The deletePaths/deleteFiles/removedDataFilePaths checks were chained as else-if branches, so once one of the earlier conditions applied, later ones were never evaluated. This meant a manifest could be skipped (and a dangling DV missed) whenever both deleteFiles and removedDataFilePaths were non-empty
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#13222 introduced detecting dangling DV during ManifestFilterManager. But there are edge cases that cause dangling DVs to be skipped by accident.
See each commit for a description of the bug and a test that reproduces the behavior.