Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R2: model decrease overflow with Float32 on brownden problem #214

Open
d-monnet opened this issue May 25, 2023 · 1 comment
Open

R2: model decrease overflow with Float32 on brownden problem #214

d-monnet opened this issue May 25, 2023 · 1 comment

Comments

@d-monnet
Copy link
Contributor

d-monnet commented May 25, 2023

Hi there,
I noticed that R2 overflows with Float32 on the first iteration of Brownden problem from OptimizationProblems.ADNLPProblems.
The overflow comes from the way the model decrease is computed: ΔTk = norm_∇fk^2 / σk. Since norm_∇fk is really big, norm_∇fk^2 overflows.
I notice that by comparing R2 with my own implementation of R2 that computes the model decrease as ΔTk = dot(∇fk,∇fk / σk) which does not overflow and converges.
However I acknowledge that computing ΔTk as in R2 is faster than what I implemented since norm_∇fk is already computed, while my implementation has to compute an extra dot product.

Not sure what you want to do to address that, if it is any problem at all, just thought I'd mention it.

@d-monnet
Copy link
Contributor Author

Maybe something like ΔTk = (norm_∇fk/σk)^2 * σk would help, but underflow might occur if σk is big. I suppose there's no easy fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant