feat(storage): open Foyer split range cache - #6702
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a50bb32fbb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| pub(crate) fn foyer_flush_on_close(policy: SplitRangeCacheWritePolicy) -> bool { | ||
| matches!(policy, SplitRangeCacheWritePolicy::WriteOnInsertion) |
There was a problem hiding this comment.
With the default WriteOnEviction policy, entries are not written to disk until they leave the memory cache, so returning false here causes a graceful close() to discard every still-resident entry instead of making it recoverable on the next open(). Conversely, write-on-insertion entries are already submitted to storage. This pairing should be inverted so normal restarts preserve the hot portion of the default cache.
Useful? React with 👍 / 👎.
Add FoyerSplitRangeCache::open/close so a process-wide hybrid cache can be built from SplitRangeDiskCacheConfig, including write policy, recovery, and compression.
Write-on-eviction keeps hot entries in memory until close, so flush them then. Write-on-insertion already persisted those entries.
94c1743 to
fa55d6f
Compare
1 participant
Summary
FoyerSplitRangeCache::open/closeto build a process-wide Foyer hybrid cache fromSplitRangeDiskCacheConfig.Stack
Test plan
cargo test -p quickwit-storage --features testsuite split_range_cachecargo clippy -p quickwit-storage --features testsuite --tests -- -D warnings