Skip to content

Home

Jump to bottom
João F Silva edited this page Mar 11, 2022 · 8 revisions

PatientTM - Patient Trajectory Modelling

PatientTM is a solution for modelling patient trajectories according to different clinical outcomes (e.g. readmission prediction, diagnoses prediction) using multimodal clinical information, which includes textual information from clinical notes (we only use discharge notes), timing data, and coding information from standard vocabularies for the representation of patient admissions and trajectories.

PatientTM can operate at two different levels, performing predictions at the individual admission level or at a sequence-based level where multiple admissions are considered through a sliding window mechanism. As of now, PatientTM supports the prediction of two different clinical outcomes: early patient readmission (30 days), and next visit diagnoses prediction.

Running the system

Example commands for running PatientTM in the different possible configurations are provided below.

Important notes to consider when configuring the desired python command:

  • To use GPU resources during model development, provide the CUDA_VISIBLE_DEVICES environment variable before the Python command.

  • PatientTM supports different input features, which can be used either separately or combined in feature sets. Input features to be used must be provided in the Python command using the -feat argument. Please provide one or more features from the following list: clinical_text, daystoprevadmit, duration, diag_ccs, proc_ccs, small_diag_icd9, small_proc_icd9, cui.

  • Precomputed representations for clinical text can be used instead of having the model generate runtime representations every iteration. For that, provide the precomputed_text flag in the running command. This is useful when the user does not intend to finetune the clinicalBERT model, greatly speeding up model execution.

  • PatientTM supports the prediction of diagnoses codes using two different standard vocabularies: a simplified version of ICD9, and CCS. To select the type of vocabulary to be used, provide the --codes_to_predict argument with one the following parameters: small_diag_icd9, diag_ccs.

  • Subsampling is only provided as an option in readmission prediction (through the --subsampling flag), where positive and negative samples are randomly drawn from folds in a balanced strategy.

  • Admission/Sequence level prediction is configured through the --task_name input argument. For readmission prediction on the admission level use readmission, for diagnosis prediction on the admission level use diagnosis_prediction, and for sequence level prediction use trajectory_modelling.

    • When using the trajectory_modelling option, further distinction is provided through the trajectory_subtask_name argument where on the following must be selected: readmission, diagnosis.

    • Finally, if diagnosis is selected in trajectory_subtask_name, the user must provide the desired type of code to predict through --codes_to_predict: small_diag_icd9 , diag_ccs.

Admission-level prediction

Early readmission (30 days)

CUDA_VISIBLE_DEVICES=0 python3 ./main.py   --task_name readmission   --readmission_mode discharge   --do_train   --data_dir /PatientTM/data/extended_folds/discharge_subjectsplit/   --bert_model /PatientTM/model/pretraining   --max_seq_length 512   --output_dir /PatientTM/results/result_discharge --early_stop --train_batch_size 64 --num_train_epochs 30 --learning_rate 0.001 --subsampling -feat daystoprevadmit duration clinical_text

Next visit diagnoses

CUDA_VISIBLE_DEVICES=0 python3 ./main.py   --task_name diagnosis_prediction   --readmission_mode discharge   --do_train   --data_dir /PatientTM/data/extended_folds/discharge_subjectsplit/  --bert_model /PatientTM/model/pretraining   --max_seq_length 512   --output_dir /PatientTM/results/result_discharge --early_stop  --train_batch_size 64 --num_train_epochs 30 --learning_rate 0.001 --codes_to_predict small_diag_icd9 -feat clinical_text small_diag_icd9

Extra: Precomputed vs Runtime text representations

When using the admission-level architecture in the readmission or diagnosis_prediction tasks with clinical text as an input feature, it is possible to use precomputed representations or to generate embedding representations for clinical text in runtime.

  • Precomputed text representations greatly speed up model training and can be used by providing the --precomputed_text flag. However, they require an additional preprocessing procedure where clinical text representations are firstly extracted with clinicalBERT and saved to disk.

  • Runtime generated text representations are much more time-expensive but provide the benefit of enabling fine-tuning clinicalBERT. PatientTM provides the possibility of defining the number of BERT layers to fine-tune (ordered from the last to first BERT layer) through flag --bert_num_trainable_layers. It is also possible to define the number of epochs for BERT fine-tuning through flag --bert_finetune_epochs.

Example code for finetuning clinicalBERT and using runtime generated text representations:

CUDA_VISIBLE_DEVICES=0 python3 ./main.py   --task_name readmission   --readmission_mode discharge   --do_train   --data_dir /PatientTM/data/extended_folds/discharge_subjectsplit/   --bert_model /PatientTM/model/pretraining   --max_seq_length 512   --output_dir /PatientTM/results/result_discharge --early_stop --train_batch_size 64 --num_train_epochs 3 --bert_finetune_epochs 3 --bert_num_trainable_layers 2  --learning_rate 0.0001 --subsampling -feat clinical_text

Sequence-level prediction

Currently, when using multiple admissions to provide context in patient trajectory modelling, PatientTM uses pretrained admission-level models to precompute embeddings for each patient admission.

Important notes to consider in the sequence-level architecture:

  • PatientTM processes patient timelines using a sliding window mechanism that iterates over the patient trajectory, creating windows of 3 and 6 (these values were predefined) admissions. To select the size of the sliding window to be used, provided the --visit_sliding_window argument with 3 or 6.

  • Two different recurrent architectures are supported: GRU and LSTM. The desired architecture must be provided using the --recurrent_network flag. Other parameters can also be defined such as: --recurrent_hidden_size, --recurrent_num_layers,

  • To use bidirectionality in the recurrent architecture, provide the --bidirectional boolean flag.

  • Preliminary feature: --multi_hot_diag can be used to run the system using a multi-hot encoding of the input diagnoses codes, however this is still experimental and is used to compare with the LIG-Doctor solution. In the feature it will be expanded to support multi-hot encoding with embedding representations.

Early readmission (30 days)

CUDA_VISIBLE_DEVICES=0 python3 ./main.py   --task_name trajectory_modelling --trajectory_subtask_name readmission  --readmission_mode discharge   --do_train   --data_dir /PatientTM/data/extended_folds/discharge_subjectsplit_trajectory/ --output_dir /PatientTM/results/result_discharge  --train_batch_size 64 --num_train_epochs 30 --learning_rate 0.001 --recurrent_num_layers 2 --recurrent_hidden_size 100 --recurrent_network LSTM --bidirectional --visit_sliding_window 3

Next visit diagnoses

CUDA_VISIBLE_DEVICES=0 python3 ./main.py   --task_name trajectory_modelling --trajectory_subtask_name diagnosis --codes_to_predict small_diag_icd9  --readmission_mode discharge   --do_train   --data_dir /PatientTM/data/extended_folds/discharge_subjectsplit_trajectory/ --output_dir /PatientTM/results/result_discharge  --train_batch_size 64 --num_train_epochs 30 --learning_rate 0.001 --recurrent_num_layers 2 --recurrent_hidden_size 100 --recurrent_network LSTM --bidirectional --visit_sliding_window 3
Clone this wiki locally