Should I write new BPCells matrices for each step in Seurat v5?

[rjmusich]

I have been working with Seurat v5 recently using an 80 sample, ~1 million total cells and thought it was a 'smart' thing to write BPCells matrices at each step so that I could rollback to a previous version if something went wrong. After extensive downstream analysis, I started to encounter a "too many open files" error that would not let me proceed any further with the analysis. In theory, there should only be 80 files open, one for each sample, but I think what is happening is that the Seurat v5 object has memory of the files that were saved at each step and therefore I artificially ballooned the object to an absurd size unknowingly. Thus if I read in the Seurat object at step 10, it is technically reading in the files from step 1, 2, 3, ..., 10 and then throwing the error.

My solution so far is to essentially copy the metadata with cell typing information from my furthest downstream object and copying that into the initial Seurat v5 object that is using the raw BPCells matrices. For future reference, I wanted to know what the correct way to utilize BPCells to avoid this 'too many open files' error. If I wanted to save the Seurat object at each step, should I just write the object to the same folder where the initial BPCells matrices are saved? My thought was that this would overwrite the actual matrices themselves and would therefore result in a loss of information over time, but I think I have learned that saving the Seurat v5 object does not alter the actual BPCells matrices themselves but changes what cells that object pulls in.

If someone is more knowledgeable on how BPCells works (or thinks this is more of a question for the BPCells devs), I would appreciate the help!