Remove flaky performance comments from other pull requests - #71
Remove flaky performance comments from other pull requests#71SteveBronder wants to merge 3 commits into
Conversation
|
cc @dylex -- we could probably simplify the rest of this file if this is removed |
|
Yeah a lot of the work in this file was producing those comments. If they aren't wanted anymore, we can clean more of it up. (btw, I suspect most of the current variance is just which machine it's assigned to -- we do now have a dedicated exclusive node for short benchmarking like this if it would be better to move it there instead. It is linux -- not sure if the part that runs on the old mac is relevant for this but it's probably not so stable either.) |
I'd be curious to give that a try. They still might not be terribly meaningful comments, though |
This removes
postCommentusage. This removes comments such as this from being generated in stan pull requests. The comment is rarely ever useful and moreso confusing to new contributors. Because of random timing differences or cpu usage on the machine the performance test you can see swings of 15-20% from run to run with the same code. So I think it is just better we totally remove that code path and do not generate the comment.