Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: std::bad_alloc when using writeCDF #1580

Open
natiguvi opened this issue Aug 1, 2024 · 1 comment
Open

Error: std::bad_alloc when using writeCDF #1580

natiguvi opened this issue Aug 1, 2024 · 1 comment

Comments

@natiguvi
Copy link

natiguvi commented Aug 1, 2024

I am reading a large number of ASCCI files from four years of daily data, and packing it on an SpatRaster, then I'm assing it dates and saving it as an .nc file using writeCDF, but I get:

"Error: std::bad_alloc".

I already did this about two months ago for rasters of the same size but with more years and it worked perfectly. Now when I run 4 years, I get the mentioned error.

If I run 3 years I get this one:

"Error en (function (cond) : error in evaluating the argument 'x' in selecting a method for function 'crs': std::bad_alloc",

And if I run 2 years it seems to work but I don't think is a memory problem because this is the output from mem_info(writeCDF(temp_rasters3, name, overwrite=TRUE,compression=9)):

"------------------------
Memory (GB)
"------------------------
check threshold : 1 (memmin)
available : 139.71
allowed (95%) : 132.72
needed (n=1) : 16.36
"------------------------
proc in memory : TRUE
nr chunks : 1
------------------------"

I was trying to attach the temp raster made when reading ASCCI files but is too big (10gb), I'm not attaching code because ASCCI files are bigger and I'm only reading it, changing crs, and packaging. The errors mentioned are when trying to save it as .nc, as raster there's no problem:

writeRaster(temp_rasters3, "temp_rasters3.tif", overwrite=TRUE)
filename : D:/temp/temp_rasters3.tif
compute stats : 1, GDAL: 0, minmax: 0, approx: 1
driver : GTiff
disk available: 417.2 GB
disk needed : 16.4 GB
memory avail. : 176.8 GB
memory allow. : 167.96 GB
memory needed : 65.466 GB (2 copies)
in memory : true
block size : 1238 rows
n blocks : 100
pb : 3

I'm using Windows 11 Pro 64, this is the output of packageVersion("terra")(I'm using dev version) and of terra::gdal(lib="all"):

packageVersion("terra")
[1] ‘1.7.81’

terra::gdal(lib="all")
gdal proj geos
"3.8.4" "9.3.1" "3.12.1"

These are the options for Terra, but I also tried the default:

terraOptions(memfrac=0.95, tempdir = "D:/Temp", verbose=T, print=TRUE,steps = 100)

Thank you in advance, I look forward to your reply.

@oshuwilson
Copy link

I've been having a similar issue despite running the code on an HPC cluster with more than enough RAM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants