How bowang-lab/ecg-fm works
1. Repository role
- This repo is primarily a research landing/experiment repo for ECG-FM, not the full model implementation.
- The actual deep learning training and inference stack lives in the external fairseq_signals framework; this repo provides project-specific data, label handling, notebooks, and analysis scripts around it.
2. Main system boundary
- The architectural boundary is between:
- data preparation / annotation / analysis in this repo, and
- model execution in fairseq_signals.
- This repo mostly orchestrates how ECG data is turned into manifests, labels, and checkpoints that fairseq_signals can consume.
3. Data layer
- data/ contains example datasets and intermediate artifacts.
- For code_15, the flow is visible end-to-end:
- org/ = original .mat ECG records
- preprocessed/ = cleaned/normalized .mat files
- segmented/ = shorter segmented ECG windows as .mat
- labels/ = label definition and label arrays (label_def.csv, labels.csv, y.npy)
- meta.csv, meta_split.csv, segmented.csv, segmented_split_incomplete.csv = metadata and split bookkeeping
- data/mimic_iv_ecg/ contains the labeler assets used for free-text ECG report processing.
4. Preprocessing and dataset shaping
- The repo represents a multi-stage ECG pipeline: raw signals are standardized, segmented, and then referenced via metadata tables.
- The resulting artifacts are designed to become dataset manifests for training/validation splits.
- splits/ stores split definitions for supported corpora, including MIMIC-IV-ECG and PhysioNet.
5. Labeling subsystem
- labeler/ is a self-contained text-to-label utility layer.
- It implements pattern-based medical text labeling and label normalization using:
- JSON pattern/template files
- string utilities and preprocessing helpers
- conversion helpers to/from JSON
- matcher and rule logic
- This is a separate subsystem from ECG signal modeling: it operates on text/report content to produce structured labels.
6. Labeler assets and semantics
- data/mimic_iv_ecg/labeler/ holds the rule and ontology files used by the labeler.
- The JSON files encode entities, descriptors, temporal patterns, connectives, uncertainty mapping, and compound expressions.
- Architecturally, this is a rule-engine / ontology-driven labeling layer feeding downstream classification tasks.
7. Model checkpoints and experiment artifacts
- ckpts/ and outputs/ are empty placeholders in the repo, indicating where training outputs and experiment results are expected to land.
- The actual published model weights are externalized to HuggingFace, not committed here.
- This separation keeps the repo focused on reproducible tooling rather than storing large binary model assets.
8. Training and inference entry points
- The repo does not contain the core trainer; instead, it documents how to invoke fairseq-hydra-train with ECG-FM configs.
- The README shows two main model workflows:
- pretraining on MIMIC-IV-ECG + PhysioNet 2021
- finetuning on MIMIC-IV-ECG diagnosis labels
- These jobs depend on manifests, label files, pretrained checkpoints, and Hydra config overrides.
9. Analysis / downstream utility scripts
- scripts/embs.py likely extracts embeddings for downstream use.
- scripts/linear.py supports linear probing experiments, using frozen or pretrained representations.
- scripts/saliency.py supports attribution/saliency visualization, matching the saliency figure shown in the README.
- These scripts sit downstream of the pretrained model and help evaluate representation quality and interpretability.
10. Notebook-based workflows
- notebooks/ contains the user-facing workflows:
- infer_quickstart.ipynb for inference and visualization
- infer_cli.ipynb for command-line result extraction/post-processing
- labeler.ipynb for the pattern labeler demo
- These notebooks are the practical interface for engineers and researchers interacting with the repo.
11. External dependencies and infrastructure
- The important external dependencies affecting architecture are:
- fairseq_signals for training/inference
- Hydra for config-driven execution
- PyTorch checkpoints for saved model weights
- external datasets: MIMIC-IV-ECG and PhysioNet Challenge 2021
- The repo is therefore best understood as a data + configuration + analysis layer around an external ECG model framework.
12. End-to-end data flow
- The architecture flow is:
- raw ECG records and report text
- preprocessing / segmentation / labeling
- manifest and split generation
- model pretraining or finetuning in fairseq_signals
- checkpoint output to ckpts/ or outputs/
- inference, embedding extraction, probing, and saliency analysis
- The labeler can optionally sit upstream of finetuning, converting report text into structured targets.
Main entry points: pattern_labeler.py Pattern labeler · README.md README · meta.csv Metadata · org Raw ECG
Components 21
Data layer
ECG datasets, intermediate artifacts, manifests, and split tables.
- Raw ECG · Original .mat ECG records for the code_15 example dataset.
data/code_15/org - Preprocessed ECG · Cleaned and normalized ECG .mat files.
data/code_15/preprocessed - Segments · Short segmented ECG windows generated from preprocessed records.
data/code_15/segmented - Metadata · Metadata and bookkeeping tables for code_15.
data/code_15/meta.csv - Labels · Label definitions and target arrays for code_15.
data/code_15/labels/labels.csv - Label defs · Label vocabulary and definitions used for the code_15 task.
data/code_15/labels/label_def.csv - Target array · Binary or multi-label target matrix for code_15.
data/code_15/labels/y.npy - MIMIC labeler · Pattern and ontology files used to label MIMIC-IV-ECG report text.
data/mimic_iv_ecg/labeler - Split defs · Dataset split definitions for MIMIC-IV-ECG and PhysioNet corpora.
splits
Labeler
Rule-based text-to-label normalization for ECG report content.
- Pattern labeler · Main text-to-label matching and normalization engine.
labeler/pattern_labeler.py - Matcher rules · Entity, descriptor, uncertainty, connective, and compound matching logic.
labeler/pattern_matcher.py - Preprocess utils · String cleanup, preprocessing, and labeler helper utilities.
labeler/preprocess.py - JSON I/O · Conversion helpers to and from JSON pattern files.
labeler/from_json.py
Analysis & workflows
Notebooks and scripts for inference, probing, embeddings, and saliency.
- Notebooks · Interactive workflows for inference, CLI post-processing, and labeler demos.
notebooks - Embeddings · Extracts embeddings for downstream use.
scripts/embs.py - Linear probe · Runs linear probing experiments on frozen or pretrained representations.
scripts/linear.py - Saliency · Generates attribution and saliency visualizations.
scripts/saliency.py
External stack
Model execution and pretrained assets live outside this repo.
- fairseq_signals · External framework that performs pretraining and finetuning.
- Checkpoints · Expected location for model checkpoints and experiment outputs in this repo.
ckpts - Outputs · Expected location for generated results and experiment artifacts.
outputs
Other components
- README · Explains workflows, datasets, and how this repo connects to the external model stack.
README.md
Connections 21
- README → Raw ECG: documents · Describes the end-to-end ECG dataset flow.
- README → Pattern labeler: documents · Explains the text labeling subsystem.
- README → fairseq_signals: depends on · Training and inference are delegated to the external framework.
- Raw ECG → Preprocessed ECG: preprocess · Raw ECG records are cleaned and normalized.
- Preprocessed ECG → Segments: segment · Preprocessed signals are split into shorter windows.
- Segments → Metadata: index · Segmented signals are referenced by metadata tables.
- Metadata → Labels: join · Metadata is paired with label tables for training.
- Labels → Target array: materialize · Labels are converted to target arrays.
- MIMIC labeler → Pattern labeler: feeds · Pattern and ontology files drive the labeler.
- Preprocess utils → Pattern labeler: supports · Preprocessing utilities normalize report text before matching.
- Matcher rules → Pattern labeler: implements · Matcher logic underpins the labeler engine.
- JSON I/O → Pattern labeler: serializes · JSON conversion supports pattern/template loading and export.
- Split defs → fairseq_signals: configures · Split definitions shape training and validation runs.
- fairseq_signals → Checkpoints: writes · Training produces checkpoints and saved model weights.
- fairseq_signals → Outputs: writes · Inference and experiments generate outputs here.
- Notebooks → fairseq_signals: orchestrates · Notebooks invoke inference and post-processing workflows.
- Embeddings → fairseq_signals: consumes · Embedding extraction uses trained representations.
- Linear probe → fairseq_signals: consumes · Linear probing runs on pretrained or frozen models.
- Saliency → fairseq_signals: consumes · Saliency analysis requires model predictions and gradients.
- Pattern labeler → Linear probe: can supply labels · Structured labels can feed downstream probing tasks.
- Metadata → fairseq_signals: manifests · Metadata tables are shaped into training manifests.