feat(#1284): benchmark unlint-non-existing-defect with +unlint metas - #1285
Open
VasilevNStas wants to merge 3 commits into
Open
feat(#1284): benchmark unlint-non-existing-defect with +unlint metas#1285VasilevNStas wants to merge 3 commits into
VasilevNStas wants to merge 3 commits into
Conversation
Contributor
Author
|
@volodya-lombrozo plz review this |
volodya-lombrozo
suggested changes
Aug 20, 2026
volodya-lombrozo
left a comment
volodya-lombrozo
left a comment
Member
There was a problem hiding this comment.
@VasilevNStas Currently we create S ... XXL benchmarks using jeo-maven-plugin which works pretty well, but it generates XMIR that is a bit of artifical. Maybe we can reconsider our strategy to generating these files? Maybe it's better to generate them by parsing real EO programs? At least, we can substitute S, M examples. What do u think?
VasilevNStas
commented
Aug 20, 2026
Contributor
Author
There was a problem hiding this comment.
Good suggestion about using real EO programs instead of bytecode-derived XMIR for the S/M benchmarks. That would make the benchmark data closer to what the lints actually process in production. I see two ways to do it and would like your call on the scope:
- A focused follow-up that only adds this
scansXmirWithUnlintsbenchmark for the+unlintpath (what this PR does), and - A bigger refactor that replaces
jeo-maven-pluginfixtures for S/M (or all sizes) with parsed EO programs.
I kept this PR to option 1 because switching the existing S..XXL fixtures touches SourceSize, BenchmarkState and the README table, and it is a separate concern from #1284 (which is specifically about the +unlint path having no coverage at all).
If you prefer, I can prepare option 2 as another PR after this one lands. Meanwhile, note this PR also surfaced a pre-existing infra issue: the proguard profile (active on JDK < 22) fails on the rultor merge (JDK 21) and locally on JDK 21 with Obfuscation failed, unrelated to the diff — I described it in #1238 with the MAVEN_OPTS/-P!proguard suggestion.
Happy to proceed either way.
VasilevNStas
force-pushed
the
fix/issue-1284-unlint-benchmark
branch
from
August 20, 2026 20:19
79e3121 to
de21d00
Compare
Contributor
Author
|
@volodya-lombrozo |
Member
|
/benchmark |
Contributor
Benchmark comparison for #1285Comparing this branch against the Biggest changes
Full results (430 entries)
|
volodya-lombrozo
suggested changes
Aug 26, 2026
volodya-lombrozo
left a comment
volodya-lombrozo
left a comment
Member
There was a problem hiding this comment.
@VasilevNStas I don't have the new benchmark results in the list above. That said, I should.
VasilevNStas
force-pushed
the
fix/issue-1284-unlint-benchmark
branch
from
August 30, 2026 19:44
de21d00 to
2459187
Compare
github-actions
Bot
added
the
core
Contributor
Author
|
@volodya-lombrozo |
Contributor
Author
|
@volodya-lombrozo Good point — the new What changed on the branch:
Verified locally: |
VasilevNStas
force-pushed
the
fix/issue-1284-unlint-benchmark
branch
from
September 5, 2026 10:49
2459187 to
ad56c14
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix #1284
What
Adds a JMH benchmark that measures
LtUnlintNonExistingDefecton an XMIR that actually contains+unlintmetas.Why
The existing
scansXmirbenchmark is driven by real-world bytecode-derived XMIR (com.sun.jna.*,org.apache.hadoop...FSNamesystem), none of which contains a single+unlintmeta. So it only measures the "no+unlintat all" fast path ofLtUnlintNonExistingDefect.#1233 changed the other path too: when
+unlintmetas are present, the lint now filtersthis.lintsdown to the referenced names before computingexisting(). Until now there was no fixture exercising that path, so a regression there would go unnoticed by the benchmark.How
New benchmark method
scansXmirWithUnlintsinSourceBench, which parses the existing fixtureunlint-ascii-only.eo(it has+unlint ascii-onlyand a non-ASCII comment that produces a real defect) and runs the full lint set on it.Measured result (JMH, average time):
This gives a timing signal for the
+unlintcode path, so future changes toLtUnlintNonExistingDefect(faster, same or slower) become visible.Both
mvn test(606 tests) andmvn clean install -Pqulicepass.