How healthylaife/mimic-iv-data-pipeline works
This repository is a configurable MIMIC-IV preparation and downstream-analysis pipeline. Its user-facing workflow, documented around mainPipeline.ipynb, selects a cohort and features, processes structured and clinical-text data into time-aligned datasets, and supports modeling and evaluation. The README also describes multimodal extensions for notes, imaging, ECG, echo, and waveforms; their embedding notebooks are represented as a distinct branch. The graph combines README-described capabilities with sampled source behavior; unsampled subsystem wiring is shown only where documented.
Main entry points: mainPipeline.ipynb Workflow notebook · data Processed features · feature_selection_hosp.py Hospital features · feature_selection_icu.py ICU features
Components 19
Pipeline entry
- Workflow notebook
mainPipeline.ipynb
Cohort preparation
- Cohort extraction
preprocessing/day_intervals_preproc/day_intervals_cohort_v3.py - Cohort files
data
Feature processing
- Hospital features
preprocessing/hosp_module_preproc/feature_selection_hosp.py - ICU features
preprocessing/hosp_module_preproc/feature_selection_icu.py - Hospital preprocessing
utils/hosp_preprocess_util.py - ICU preprocessing
utils/icu_preprocess_util.py - Cleaning and conversion
utils - Clinical notes
preprocessing/notes_preproc.py - Time-series assembly
model/data_generation.py - ICU time-series
model/data_generation_icu.py - Processed features
data
Multimodal extensions
- Modality embeddings
multimodal-pipeline/Multimodal-Embeddings
Modeling and evaluation
- Prediction models
model - Model evaluation
model/evaluation.py - Fairness measures
model/fairness.py - Analysis outputs
data
Other components
- Pipeline user
- MIMIC-IV sources
Connections 27
- Pipeline user → Workflow notebook: configures workflow
- MIMIC-IV sources → Workflow notebook: provides input data
- Workflow notebook → Cohort extraction: extracts cohort
- Cohort extraction → Cohort files: writes cohort
- Workflow notebook → Hospital features: selects features
- Workflow notebook → ICU features: selects features
- Hospital features → Hospital preprocessing: calls preprocessing
- ICU features → ICU preprocessing: calls preprocessing
- Hospital features → Cleaning and conversion: cleans and groups
- ICU features → Cleaning and conversion: cleans and groups
- Workflow notebook → Clinical notes: preprocesses notes
- Clinical notes → Processed features: produces note data
- Hospital features → Processed features: writes features
- ICU features → Processed features: writes features
- Time-series assembly → Cohort files: reads cohort
- Time-series assembly → Processed features: reads features
- ICU time-series → Cohort files: reads cohort
- ICU time-series → Processed features: reads features
- Workflow notebook → Time-series assembly: generates sequences
- Workflow notebook → ICU time-series: generates sequences
- MIMIC-IV sources → Modality embeddings: supplies modalities
- Modality embeddings → Processed features: produces embeddings
- Workflow notebook → Prediction models: supports modeling
- Prediction models → Model evaluation: evaluated by
- Model evaluation → Fairness measures: reports fairness
- Prediction models → Analysis outputs: produces results
- Model evaluation → Analysis outputs: writes metrics