Repository navigation
Conversation
Totodore
changed the title
Totodore
changed the title
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc46f586aa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| continue; | ||
| } | ||
| // The index_uid might be different so we are overwriting it with the new one. | ||
| let mut split_metadata = recovered_split.split_metadata.clone(); |
There was a problem hiding this comment.
When this is a true metastore rebuild (for example, a new file-backed index or a fresh PostgreSQL database), delete-task opstamps restart at 1, but this clone retains each split's old delete_opstamp and only changes its index UID. DeleteTaskPlanner only considers splits whose delete opstamp is below the current last task opstamp, so a recovered split at opstamp 100 silently ignores newly submitted or resubmitted delete tasks 1 through 100. Rebase these values, or restore/advance the task-opstamp namespace together with the splits.
AGENTS.md reference: AGENTS.md:L21-L22
Useful? React with 👍 / 👎.
| .iter() | ||
| .map(|split_metadata| split_metadata.split_id().to_string()) | ||
| .collect(), | ||
| replaced_split_ids: Vec::new(), |
There was a problem hiding this comment.
When recovery runs against a stale or nonempty metastore and a recovered merge output names a parent that is still Published, the planner classifies that parent as merged and does not stage it, but this empty replacement list leaves the existing parent published. Publishing the recovered child then makes both parent and child searchable, duplicating all of the parent's documents; preserve the recovered parent links and include currently published known parents in the replacement set.
AGENTS.md reference: AGENTS.md:L21-L22
Useful? React with 👍 / 👎.
| } | ||
| // The index_uid might be different so we are overwriting it with the new one. | ||
| let mut split_metadata = recovered_split.split_metadata.clone(); | ||
| split_metadata.index_uid = index_uid.clone(); |
There was a problem hiding this comment.
When two indexes share an index_uri (or the configured URI otherwise contains splits from multiple indexes), object listing returns every .split file, and this assignment overwrites each recovered split's original index UID without checking its embedded index ID. Recovering index A therefore stages index B's splits into A, which can mix incompatible schemas and corrupt search results; validate the original index ID before allowing the expected incarnation-ID restamp.
AGENTS.md reference: AGENTS.md:L21-L22
Useful? React with 👍 / 👎.
This branch has not been deployed
1 participant
Description
Add a
tool recoversub-command toquickwit-clithat will recover the metastore state from split metadata.How was this PR tested?
Tested locally with rustfs. I deleted an index, recreated it and runned:
quickwit tool recover --config config.yaml --index my-index --dry-runandquickwit tool recover --config config.yaml --index my-index.I recovered every split that had the split metadata footer (feature introduced in #6689 by @guilload).
Tested on a qual environment. I deleted 2 split rows from the split table, run the script with
--dry-run:And then without
--dry-run: