Repository navigation
bench: measure render cost across setups and inputs - #617
Closed
lessthanjacob wants to merge 1 commit into
Closed
lessthanjacob wants to merge 1 commit into
lessthanjacob wants to merge 1 commit into
Conversation
rake benchmarks only enforces a few throughput floors, so a render-path change can get slower and still pass CI. rake engines varies one factor at a time (input kind, size, nesting, fields, extensions, and so on), checks that engines agree before timing, and prints the change against a saved run via BASELINE=. On main it measures V1. When Blueprinter::V2 is defined it also measures V2 and V2+Legacy. Results go to benchmarks/results/, which is gitignored. Signed-off-by: Jacob Sheehy <jacobjsheehy@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| def table(axis, rows) | ||
| head = [axis, 'engine', 'median ms', 'time', 'alloc/obj', 'alloc'] | ||
| ["| #{head.join(' | ')} |", "|#{head.map { ' --- |' }.join}", | ||
| *rows.map { |r| "| #{r.map { |c| c.to_s.gsub('|', '\\|') }.join(' | ')} |" }] |
| end | ||
|
|
||
| def md_table(head, rows) | ||
| cells = ->(row) { row.map { |c| c.to_s.gsub('|', '\\|') }.join(' | ') } |
Contributor
Author
|
Closing for now. This is not ready for review yet. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist:
Why
rake benchmarksonly enforces a few throughput floors. A render-path change can get slower and still pass CI.rake enginesis the comparison to put in a render-path PR: one factor at a time, then a few combined profiles, with the change against a savedmainrun.What
bundle exec rake engines(spec/benchmarks/engines/). V1 on this branch. WhenBlueprinter::V2is defined, the same run also measures V2 and V2+Legacy.BASELINE=<results json>prints the change in median time and allocations per rendered object.benchmarks/results/(gitignored).LIST=1prints the sweep names.AGENTS.mdanddocs/testing.mdtell render-path changes to report this comparison.rake benchmarksstill has to pass.Not run in CI.
Check
Smoke-tested on this machine with
LIST=1andSWEEPS=input_kind SAMPLES=1(V1 only, YJIT off). Fullbundle exec rakewas not run locally.Made with Cursor