Skip to content

Remove flaky performance comments from other pull requests - #71

Open
SteveBronder wants to merge 3 commits into
masterfrom
remove-flaky-perf-comment
Open

Remove flaky performance comments from other pull requests#71
SteveBronder wants to merge 3 commits into
masterfrom
remove-flaky-perf-comment

Conversation

@SteveBronder

Copy link
Copy Markdown
Collaborator

This removes postComment usage. This removes comments such as this from being generated in stan pull requests. The comment is rarely ever useful and moreso confusing to new contributors. Because of random timing differences or cpu usage on the machine the performance test you can see swings of 15-20% from run to run with the same code. So I think it is just better we totally remove that code path and do not generate the comment.

@WardBrian

Copy link
Copy Markdown
Member

cc @dylex -- we could probably simplify the rest of this file if this is removed

@dylex

dylex commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Yeah a lot of the work in this file was producing those comments. If they aren't wanted anymore, we can clean more of it up.

(btw, I suspect most of the current variance is just which machine it's assigned to -- we do now have a dedicated exclusive node for short benchmarking like this if it would be better to move it there instead. It is linux -- not sure if the part that runs on the old mac is relevant for this but it's probably not so stable either.)

@WardBrian

Copy link
Copy Markdown
Member

btw, I suspect most of the current variance is just which machine it's assigned to -- we do now have a dedicated exclusive node for short benchmarking like this if it would be better to move it there instead.

I'd be curious to give that a try. They still might not be terribly meaningful comments, though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants