-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Relax a test to address poorer than expected performance on Windows? #60
Comments
I can reproduce the test regression locally on Mac. versioninfo
Runs fine on Curiously, I cannot reproduce on Windows 10. |
Thanks @skleinbo for the further investigation. This is pretty weird. I'm wondering if we can reproduce this error on any Julia version prior to 1.9 . |
@ablaom Both versions, pre- and post #59, pass for me on Julia 1.8.5. The poor performance isn't real by the way, but for some reason recompilation seems to be triggered. This is going to be impossible to debug I fear. I can make the test pass by putting literally any kind of additional print statement in that test set. For example, annotating CategoricalDistributions.jl/test/arithmetic.jl Lines 60 to 61 in 66f23a3
with Is it a good ideal to rely on a single measurement of wall time anyway? This seems prone to spurious failure. |
Okay, great! Please see #63. |
closed by #63 |
A Windows fail was detected for a PR making a change apparently unrelated to the failing test. Here is the log:
https://github.com/JuliaAI/CategoricalDistributions.jl/actions/runs/6160604583/job/16728423043?pr=59#step:6:232
Instead of a 10x speedup in a test for array-optimised arithmetic, we're only getting 2x speed up. I guess this is because of some change in the virtual machine used by GitHub for windows testing, or similar?
It may be the fail in intermittent.
In any case, a little investigation is probably warranted before relaxing the test.
The text was updated successfully, but these errors were encountered: