Batch no-run container reads during decoding - #571
Open
perfloop-agent wants to merge 2 commits into
Open
perfloop-agent wants to merge 2 commits into
perfloop-agent wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Batch no-run container payload reads during decoding from ordinary
io.Readervalues. The decoder now coalesces exact whole-container payloads without consuming bytes after the bitmap. It also keeps bitmap payloads aligned when an array payload precedes it in a shared read.Type of Change
Changes Made
What was changed?
roaringArray.readFromusing cardinalities from the descriptive header, with a bounded group size and exact container slices.ByteInput, run-container, and copy-on-write paths on their existing behavior.Why was it changed?
[]uint64.How was it changed?
Testing
go test ./...go test -tags appenginego tool unconvert ./...GOARCH=386 go buildGOARCH=arm go buildGOARCH=arm64 go buildChecks: 8 passed.
Formatting
go fmt ./...Fuzzing
go test -tags=gofuzz -run=TestGenerateSmatCorpusgo test -run='^$' -fuzz=FuzzSmat -fuzztime=300s -timeout=60sPerformance Impact
The sparse-reader benchmark covers unbuffered file and short-read reader cases. The existing
BenchmarkUnserializeReadFromrow remains a guard for the ordinarybytes.Readerdecode path.Workload:
Bitmap.ReadFrom of a no-run sparse bitmap with 1024 singleton containers from an unbuffered fileread-calls/opWorkload:
Bitmap.ReadFrom of a no-run sparse bitmap with 4096 singleton containers from an unbuffered fileread-calls/opWorkload:
Bitmap.ReadFrom of a no-run sparse bitmap with 16384 singleton containers from an unbuffered fileread-calls/opWorkload:
Bitmap.ReadFrom of a no-run sparse bitmap with 1024 singleton containers through a 257-byte short-read file readerread-calls/opWorkload:
Bitmap.ReadFrom of a no-run sparse bitmap with 4096 singleton containers through a 257-byte short-read file readerread-calls/opWorkload:
Bitmap.ReadFrom of a no-run sparse bitmap with 16384 singleton containers through a 257-byte short-read file readerread-calls/opBreaking Changes
None.
Related Issues
None.
Additional Notes
No public API changes.
Generated by Perfloop. Human sponsor: Tomás Senart. Measurements and checks.