# willxxy/ecg-byte architecture

> Architecture diagram of the GitHub repository willxxy/ecg-byte, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/willxxy/ecg-byte
- Repository: https://github.com/willxxy/ecg-byte

## Overview

##### Data Sources and Ingestion
- Raw ECG Data: Represent external datasets (PTB-XL, MIMIC-IV and ECG-QA). Note that the README instructs users to download and prepare data in a “data” folder.
- External Mapping: There is a mapping process that aligns the PTB-XL/MIMIC data with ECG-QA (using scripts provided in the README and instructions from other repositories).
- Pretrain Data: Also include data curated by ECG-Chat for the pretrain stage.

##### Preprocessing Pipeline
- Preprocess folder: Contains scripts (e.g. preprocess.sh) and Python modules in “preprocess” that load raw ECG files, handle segments (e.g., segment lengths such as 2500 samples) and apply transformations (including PCA).
- Sampling: The “sample_ecg.sh” script is used to sample the ECGs and produce a text file used for training the tokenizer.
- Data Loader: “data_loader.py” ties into ingestion by loading and batching the processed ECG segments.

##### ECG-Byte Tokenizer
- Training and Implementation: The “train_tokenizer.py” script (triggered via “train_tok.sh”) and the “rust_bpe” folder (compiled via maturin develop) are responsible for training the ECG-specific byte-pair encoding.
- Analysis of Tokenization: The analysis folder contains modules to plot token distribution (“token_dist.sh”) and track token merging (“track_encoding.sh”).

##### Model Training
There are two major training “pathways”:
A. End-to-End LLM Training:
- Main Training: “main.py” is the central script that ties together the loaded data, the preprocessed tokenized input, a chosen foundation model (e.g., Llama 3.2 1B) and the end-to-end architecture.
- Custom Scheduler: “scheduler.py” encapsulates a custom learning rate schedule used for both end-to-end and two-stage training.
- Runners: Scripts in the “runners” folder (e.g., inference.py, train.py, interpret.py) orchestrate the training and inferencing routines.

B. Two-Stage Training Methods:
- Stage 1 Pretraining: “pretrain.py” (invoked via “pretrain.sh”) processes and trains using models in the “models” folder (like a ResNet variant for feature extraction).
- Stage 2 Finetuning: “finetune.py” (run via “finetune.sh”) takes a checkpoint from stage 1 (passed as “first_check”) and fine-tunes further on the ECG-QA (question–answer) tasks.

##### Inference and Analysis
- Inference: Use “scripts/inference.sh” (which calls main.py with inference flag or “finetune.py” for two-stage) to generate answers from the trained LLM.
- Attention and Visualization: “interp_analysis.py” (and related scripts) provides visual analysis of attention maps. Also, the analysis folder has components to measure token distributions and merge tracking.

##### Auxiliary and Utility Modules
- Models: The “models” folder includes model definitions for the various network architectures. This abstraction is used in both training and finetuning.
- Utils: Contains helper functions across the codebase for file I/O, plotting (viz_utils.py), model checkpoint handling, and other repeated logic.

##### Execution Scripts & Environment
- “scripts” folder: Contains bash scripts for token training, end-to-end training, two-stage training, inference, and analysis.

Relationships and Data Flow:
- Dependencies like the custom scheduler feeding into training scripts.

Design Tips:
– Use clear labels for each component/block (e.g., “Raw Data”, “Preprocessing”, “ECG-Byte Tokenizer”, “End-to-End Training”, “Stage 1 Pretraining”, “Stage 2 Finetuning”, “Inference & Analysis”).
– Optionally, include annotations for scripts (e.g., “train_tok.sh”, “pretrain.sh”, “finetune.sh”) to show how the automation triggers various parts of the pipeline.
– Since many utilities and tests also exist (e.g. “test/” folder, “utils/”), you may omit extensive test details but note that the design supports thorough validation and modularity.

- Data Input (raw datasets, external mapping)
↓
- Preprocessing (data_loader, preprocess, sampling)
↓
- ECG-Byte Tokenizer (Rust BPE, train_tokenizer.py)
↓
- Training Pipelines:
- End-to-End LLM Training (main.py, model training, scheduler, runners)
- Two-Stage Training (pretrain.py → finetune.py; with checkpoint handoff)
↓
- Inference & Analysis (inference.sh, attention interp_analysis.py, token distribution visualizations)
- While running continuously on an environment set up with appropriate GPU/PyTorch and HuggingFace integration.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Layer"
        raw1["PTB-XL (Raw ECG Data)"]
        raw2["MIMIC-IV (Raw ECG Data)"]
        raw3["ECG-QA Mapping"]
        raw4["ECG-Chat Data (Pretrain)"]
    end

    subgraph "Preprocessing Pipeline"
        preproc["Preprocess Scripts (preprocess)"]
        sampler["ECG Sampler (sample_ecg.sh)"]
        loader["Data Loader (data_loader.py)"]
    end

    subgraph "ECG-Byte Tokenizer"
        tokenizerA["Rust BPE Core (rust_bpe)"]
        tokenizerB["Training Script (train_tokenizer.py)"]
        tokenAnalysis["Token Analysis (token_dist.sh, track_encoding.sh)"]
    end

    subgraph "Model Training"
        subgraph "End-to-End LLM Training"
            mainScript["Main Training (main.py)"]
            scheduler["Custom Scheduler (scheduler.py)"]
            runners["Runners (runners folder)"]
        end
        subgraph "Two-Stage Training"
            stage1["1st Stage Pretraining (pretrain.py)"]
            stage2["2nd Stage Finetuning (finetune.py)"]
        end
    end

    subgraph "Inference & Analysis"
        inferenceScript["Inference (scripts/inference.sh)"]
        visualAnalysis["Visualization Analysis (interp_analysis.py)"]
    end

    subgraph "Auxiliary / Utilities"
        models["Models (models folder)"]
        utils["Utils (viz_utils.py, etc.)"]
    end

    %% Data flow connections
    raw1 --> preproc
    raw2 --> preproc
    raw3 --> preproc
    raw4 --> preproc

    preproc --> sampler
    sampler --> loader

    loader --> tokenizerB
    tokenizerA --> tokenizerB
    tokenizerB --> tokenAnalysis

    tokenizerB --> mainScript
    mainScript --> scheduler
    mainScript --> runners

    loader --> stage1
    stage1 --> stage2

    mainScript --> inferenceScript
    stage2 --> inferenceScript

    mainScript --- visualAnalysis
    stage2 --- visualAnalysis
    tokenAnalysis --- visualAnalysis

    models --- mainScript
    models --- stage1
    models --- stage2

    utils --- preproc
    utils --- mainScript
    utils --- inferenceScript

    note1[Environment: Ubuntu 20.04, GPUs, PyTorch, HuggingFace Integration]
    inferenceScript --- note1

    %% Class assignments and styles
    class raw1,raw2,raw3,raw4 dataLayer;
    class preproc,sampler,loader,tokenizerA,tokenizerB,tokenAnalysis preprocess;
    class mainScript,scheduler,runners training;
    class stage1,stage2 twoStage;
    class inferenceScript,visualAnalysis analysis;
    class models,utils auxiliary;

    classDef dataLayer fill:#cce5ff,stroke:#007bff,stroke-width:2px;
    classDef preprocess fill:#d4edda,stroke:#28a745,stroke-width:2px;
    classDef training fill:#f8d7da,stroke:#dc3545,stroke-width:2px;
    classDef twoStage fill:#ffe5b4,stroke:#fd7e14,stroke-width:2px;
    classDef analysis fill:#e2d6f3,stroke:#6f42c1,stroke-width:2px;
    classDef auxiliary fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
```
