Task definition matters in terms of memory usage #141
Labels
Computational Performance
Relates to the computational efficiency of the cohort extraction
Documentation
Improvements or additions to documentation
priority:high
Things that are high priority, but do not warrant an immediate hotfix
As suggested by @Oufattole, this kind of config seems to use excessive amounts of memory (more than 400 GB on MIMIC-IV in my case)
using regex mitigates this:
https://github.com/Oufattole/meds-torch/blob/main/MIMICIV_INDUCTIVE_EXPERIMENTS/configs/tasks/mortality/in_icu/first_24h.yaml
I'm not sure what the reason could be, but perhaps it's something to attend users to. Tested with es-aces 0.5.1 and command:
aces-cli --multirun hydra/launcher=joblib data=sharded data.standard=meds data.root="$MIMICIV_MEDS_DIR/data" "data.shard=$(expand_shards $MIMICIV_MEDS_DIR/data)" cohort_dir="$cohort_dir" cohort_name="$TASK_NAME"
The text was updated successfully, but these errors were encountered: