Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.98%. Comparing base (dd6448b) to head (0087151).
Report is 2 commits behind head on main.

@@           Coverage Diff           @@
##             main    #2474   +/-   ##
=======================================
  Coverage   99.98%   99.98%           
=======================================
  Files         191      191           
  Lines       16856    16864    +8     
=======================================
+ Hits        16854    16862    +8     
  Misses          2        2           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This is an awesome tutorial, thanks a lot for the contribution!

Some additional linear algebraic comments:

        # Inverse of the gram matrix.
        self.V = torch.linalg.inv(x.T @ x)

It's generally inadvisable to compute the inverse of a matrix. Rather than doing that, you'd typically compute a matrix decomposition of x.T @ x and use that for solves down the line using forward-backward substitutions. Could you modify the code to that end? Happy to provide more specifics / code changes if that would be useful.

Also, some of this can get hairy in 32bit precision if the gram matrix is ill-conditioned. In general we recommend folks use botorch with 64bit precision (except when running on larger data sets on a GPU where perf really counts). Could you modify the tutorial to use torch.float64? Either by choosing the dtype explicitly or by setting the default torch dtype at the beginning of the tutorial.

torch.Double seems to improve the results, I see why this is recommended now. Switched the torch.linalg.inv out for torch.linalg.cholesky followed by torch.cholesky_inverse. I am not sure if torch.cholesky_inverse does forwards-backwards under the hood, but it gives the same result as torch.linalg.inv.

torch.Double seems to improve the results, I see why this is recommended now.

Great

Switched the torch.linalg.inv out for torch.linalg.cholesky followed by torch.cholesky_inverse. I am not sure if torch.cholesky_inverse does forwards-backwards under the hood, but it gives the same result as torch.linalg.inv.

torch.cholesky_inverse still computes the inverse explicitly. It's better than torch.linalg.inv since you're using the knowledge that the matrix is pd, but instead of computing the inverse explicitly and then doing matmuls you should use torch.cholesky_solve (which does forward-backward substitution). So if you have A x = b you want to do x = torch.cholesky_solve(b, torch.linalg.cholesky(A)) instead of x = torch.cholesky_inverse(torch.linalg.cholesky(A)) @ x. You can assign self.L = torch.linalg.cholesky(A) instead of self.V = torch.cholesky_inverse(L) and use that both to compute the Least squares estimate as well as in the posterior call (swapping out self.V @ X.squeeze().T for the respective cholesky_solve() call).

okay, now I got it. Replaced the V @ x with torch.cholesky_solve(b, L) in the appropriate places.

@jakee417 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@jakee417 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@jakee417 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@jakee417 merged this pull request in 81f2a88.