How starmpcc/remed works
Main Components
• Data Sources & Preprocessing:
– External EHR Data (e.g., MIMIC-IV, eICU, MEDS dataset) and cohort inputs (from ACES).
– Integration with the external Integrated-EHR-Pipeline for initial data extraction.
– Scripts under “scripts/meds/” (e.g., process_meds.py) show the preprocessing phase.
• Event Encoding:
– A separately pre-trained event encoder (using modules from src/models, for example, the “eventencoder” and kernel modules in src/models/kernels).
– The encoding phase is separately handled by scripts such as encode_events.sh and corresponding Python commands.
• Model Training:
– The core REMed model (implemented across src/models, with different variants like mega, rmt, s4, etc.) is trained by drivers in main.py and various trainers under src/trainer (including models like remed.py, bioclinicalbert_encode.py, long.py, short.py).
– Training configurations and distributed training settings (using accelerate launch, DeepSpeed, FSDP configs from config/) need to be highlighted.
• Inference / Prediction:
– A dedicated prediction stage (using predict.sh scripts and testing routines also reflected in main.py) and a “standalone_remed.py” module for lightweight single-instance prediction.
• Supporting Components:
– Utility functions (src/utils) for miscellaneous trainer operations.
– Build and configuration: configuration files in config directory and build scripts for kernel compilation (using setup.py in src/models/kernels).
– Logging and experiment tracking integration via wandb.
Relationships & Data Flow
– Start with External Data Sources (EHR databases, MEDS dataset) feeding into the Preprocessing component.
– Preprocessed data (in .h5 and .tsv formats) feeds into the Event Encoding stage where the pretrained encoder processes the raw events.
– The encoded data then becomes input to the REMed training module, which uses various trainer scripts to update model weights.
– Once the model is trained, the prediction/inference block (either within the full pipeline or standalone) accepts new event sequences and returns predictions.
– Highlight the control flow managed through main.py and configuration files (that link to training, encoding, and prediction processes).
Architectural Patterns & Design Principles
– Modularity: clear separation between preprocessing (scripts/meds), model code (src/models), training logic (src/trainer) and utility functions.
– Extensibility: multiple training types (short, remed, long, etc.) and support for both a full pipeline and a standalone prediction mode.
– Accelerated execution: integration with GPU-based computation (CUDA kernels, deepspeed and FSDP configs) shown by the specialized kernel implementations.
– Pipeline orchestration: use of “accelerate launch” for distributed training and inference, as indicated by command-line examples in the README.
Relevant Technologies & Frameworks
– PyTorch & related deep learning libraries (e.g., transformers, accelerate).
– CUDA (with dedicated C++ and .cu kernel files in src/models/kernels).
– Dependency management and configurations (e.g., conda and pip install commands in the README).
– Potential logging/experiment tracking using wandb.
Elements
– Include configuration nodes (point to files in the config directory) to indicate how runtime behavior is controlled.
– Optionally, add annotations to show external integrations (e.g., integration with wandb for logging or CUDA for acceleration).