Search before asking
Paimon version
master @ bbae07d / 2.2-SNAPSHOT
Compute Engine
Engine-agnostic (core); surfaces via Flink/Spark rollback procedures
Minimal reproduce step
- Create a primary-key table, commit once on main.
table.createBranch("branch1"), branchTable = table.switchToBranch("branch1"), commit once.
branchTable.createTag("branch_tag", 1), then commit twice more.
- Expire with
snapshot.num-retained.min/max = 1 via branchTable.newCommit("").expireSnapshots().
- Read
branchTable.copy(scan.tag-name = branch_tag): fails with FileNotFoundException: manifest-list-....
Cause: AbstractFileStore.newTagManager() (line 369) passes DEFAULT_MAIN_BRANCH to TagManager, while snapshotManager() (line 190) and AbstractFileStoreTable.tagManager() (line 817) use the table branch. newExpireSnapshots() (line 485) hands that main-branch manager to ExpireSnapshotsImpl.collectTaggedSnapshots() (line 454), which never sees branch tags.
What doesn't meet your expectations?
Branch tags should protect their snapshots, as on main. Instead the tagged snapshot's manifest list and data files are deleted.
Anything else?
Same root cause: automatic tags and _SUCCESS files of branch tables land in the main tag/ directory, and Flink RollbackToProcedure (line 72), RollbackToAsLatestProcedure (line 87) and Spark RollbackProcedure (line 116) cannot find branch tags. #9044 fixed the same pattern in StaticFromTagStartingScanner.
Are you willing to submit a PR?
Search before asking
Paimon version
master @ bbae07d / 2.2-SNAPSHOT
Compute Engine
Engine-agnostic (core); surfaces via Flink/Spark rollback procedures
Minimal reproduce step
table.createBranch("branch1"),branchTable = table.switchToBranch("branch1"), commit once.branchTable.createTag("branch_tag", 1), then commit twice more.snapshot.num-retained.min/max = 1viabranchTable.newCommit("").expireSnapshots().branchTable.copy(scan.tag-name = branch_tag): fails withFileNotFoundException: manifest-list-....Cause:
AbstractFileStore.newTagManager()(line 369) passesDEFAULT_MAIN_BRANCHtoTagManager, whilesnapshotManager()(line 190) andAbstractFileStoreTable.tagManager()(line 817) use the table branch.newExpireSnapshots()(line 485) hands that main-branch manager toExpireSnapshotsImpl.collectTaggedSnapshots()(line 454), which never sees branch tags.What doesn't meet your expectations?
Branch tags should protect their snapshots, as on main. Instead the tagged snapshot's manifest list and data files are deleted.
Anything else?
Same root cause: automatic tags and
_SUCCESSfiles of branch tables land in the maintag/directory, and FlinkRollbackToProcedure(line 72),RollbackToAsLatestProcedure(line 87) and SparkRollbackProcedure(line 116) cannot find branch tags. #9044 fixed the same pattern inStaticFromTagStartingScanner.Are you willing to submit a PR?