How lrsoenksen/haim works
Main Components
• Data Ingestion & Preprocessing:
• Initial stage where raw clinical data (from sources like MIMIC-IV and MIMIC-CXR) are gathered.
• Scripts and notebooks (files beginning with "1_") for creating pickle files and generating embeddings.
• Usage of external data sources (e.g., Physionet) and how data is prepared for later steps.
• Modeling/Training:
• Module that handles modeling (scripts labeled "2_"). There are separate scripts for mortality prediction, length-of-stay modeling, and pathology diagnosis.
• That this stage applies machine learning models on preprocessed data.
• That different clinical tasks are modeled with specific scripts and that multimodal data (tabular, time-series, text, images) is utilized.
• Evaluation & Result Generation:
• Component where analysis and visualization occur. This is represented by the "3_" series of scripts and notebooks, which produce result tables, ROC/PR curves, and calculate Shapley values.
• Diagrammatic representation of reporting (plots, metrics like AUROC, AUPRC, F1 score).
• Utility and API Components:
• Supplementary modules like MIMIC_IV_HAIM_API.py and prediction_util.py, which support data access or shared functionality.
• Environment Setup:
• A note on configuration provided by the requirements file (0-requirements.txt) and the environment specifications (env folder).
Relationships and Interactions
• Finally, illustrate that model outputs are passed on to the evaluation stage for performance measurement and visualization.
• Optionally, include conditional paths or dependency links indicating optional steps (e.g., handling missing "notes" data if not available).
Architectural Patterns and Design Principles
• Modular design of the system. Each directory or file group (0, 1, 2, 3) corresponds to a step in the pipeline.
• Separation of concerns: preprocessing, modeling, and result evaluation are clearly divided.
Relevant Technologies and Frameworks
• That the project is built in Python (version 3.6.9, as per README). Although not explicitly described, mention common ML libraries or data processing frameworks implicitly likely used.
• Jupyter notebooks are employed for interactive tasks; ensure to differentiate between script-based modules and notebook-based demonstration/reporting.
• The use of YAML in the env directory suggests a reliance on environment configuration and possibly Docker or Conda environments for reproducibility.
• Be very detailed: even if the implementation is script-based, show sub-components like “Data Cleaning”, “Embedding Generation”, “Model Training”, “Visualization”, and “Shapley Value Calculation” as separate modules if needed.
• Requirements/environment configuration with the actual modules, indicating that the initial setup (requirements file and env.yaml) supports all other components.