You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have a keyword, double_precision to do a majority of calculations in double precision. However, we always read in floating point data from the uvfits file. This renders some of the double precision calculations moot . For example, when trying to do double precision linear least squares solving in the calibration, the data in the equation is always float and thus the answer is always float (related to the underflow error of most convergences being zero that Mike saw).
We can read in double precision from the uvfits file. Should this be the same keyword? Also, I'm unsure if the data truly is double in the uvfits file or if it's being casted to a double on read-in. @bhazelton Is the data natively double?
The text was updated successfully, but these errors were encountered:
We have a keyword,
double_precision
to do a majority of calculations in double precision. However, we always read in floating point data from the uvfits file. This renders some of the double precision calculations moot . For example, when trying to do double precision linear least squares solving in the calibration, the data in the equation is always float and thus the answer is always float (related to the underflow error of most convergences being zero that Mike saw).We can read in double precision from the uvfits file. Should this be the same keyword? Also, I'm unsure if the data truly is double in the uvfits file or if it's being casted to a double on read-in. @bhazelton Is the data natively double?
The text was updated successfully, but these errors were encountered: