Add more (high-level) details about Pebble compression features#23245
Open
rmloveland wants to merge 1 commit intomainfrom
Open
Add more (high-level) details about Pebble compression features#23245rmloveland wants to merge 1 commit intomainfrom
rmloveland wants to merge 1 commit intomainfrom
Conversation
✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.
|
✅ Deploy Preview for cockroachdb-api-docs canceled.
|
✅ Netlify Preview
To edit notification comments on pull requests, go to your Netlify project configuration. |
RaduBerinde
reviewed
Apr 30, 2026
|
|
||
| ##### SST compression | ||
|
|
||
| Pebble compresses SSTable and blob value data to reduce physical storage use. The default profile, `fastest`, is optimized for low CPU overhead and is appropriate for most workloads. |
Member
There was a problem hiding this comment.
I would say that fastest uses MinLZ1 on amd64 and arm64 platforms.
|
|
||
| Pebble compresses SSTable and blob value data to reduce physical storage use. The default profile, `fastest`, is optimized for low CPU overhead and is appropriate for most workloads. | ||
|
|
||
| For advanced storage tuning, CockroachDB exposes the `storage.sstable.compression_algorithm` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}). The profile values are ordered by increasing compression effort: `fastest`, `fast`, `balanced`, and `good`. Higher-effort profiles can improve compression for some workloads, but can also increase CPU usage for writes, compactions, and reads that decompress data. Most users do not need to tune this setting. Work with [Cockroach Labs Support](https://support.cockroachlabs.com/) before changing this setting in production. |
Member
There was a problem hiding this comment.
These profiles enable selective use of Zstd1 depending on the block type, LSM level, and compression benefit. Higher-effort profiles use Zstd1 more frequently and can improve ..
|
|
||
| The output CSV file is periodically rewritten while the command is running. Even if the command is interrupted, you can still use the most recently written output. | ||
|
|
||
| ### Interpret results |
Member
There was a problem hiding this comment.
I don't think we should encourage customers to interpret these results in their current form. We should say that they should just consult us with the data.
|
|
||
| Changing `storage.sstable.compression_algorithm` does not immediately recompress existing SST files. SSTs are immutable, so a new setting applies as Pebble writes new SSTs or rewrites existing SSTs during compaction, ingestion, restore, or other SST-writing work. During a transition, a store can contain SSTs compressed with multiple algorithms. | ||
|
|
||
| To evaluate the compression behavior of an existing store or backup, use [`cockroach debug pebble db analyze-data`]({% link {{ page.version.version }}/cockroach-debug-pebble-db-analyze-data.md %}). Compare the compression ratio with the compression and decompression throughput for representative data before changing the cluster setting. |
Member
There was a problem hiding this comment.
I'd say "To evaluate the CPU usage vs size tradeoff on your particular data, use .."
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes DOC-16758