# hanktseng131415go/ramem architecture

> Architecture diagram of the GitHub repository hanktseng131415go/ramem, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hanktseng131415go/ramem
- Repository: https://github.com/hanktseng131415go/ramem

## Overview

##### Project Type and Purpose
- This is a domain‐specific deep‐learning pipeline for real‐time automatic measurement of M-mode echocardiography using instance segmentation with panel attention. It is built on top of a modified Yolact_minimal framework, incorporating new modules (e.g., UPANet V2, panel attention) and an AMEM measurement algorithm.
- Key use cases:
- Training and fine-tuning segmentation models on both standard datasets (PASCAL, COCO) and the custom MEIS echocardiography dataset
- Real-time detection and automatic measurement on images or video streams
- Evaluation workflows, both with and without measurement, comparing against human labels

##### File Structure Analysis
- Top‐level orchestrator:
- main.py – handles train / evaluate / detect modes, parses command‐line args, loads config
- Configuration:
- config.py – global settings, dataset paths, model hyperparameters
- Core pipelines:
- detect.py – implements the AMEM measurement functions (measure_av, measure_lv) and handles detection logic
- eval.py – evaluation routines (without measurement)
- modules/ – model definitions for different backbones and attention mechanisms:
– resnet.py, resnet_panel.py, rtmdet.py, swin_transformer.py, upanets_v2.py, yolact.py
- External materials (readmematerials/) and raw data placeholders (raw_image, data/)
- No classical frontend/backend separation; rather a modular codebase following a pipeline pattern (data loader → model → post-processing → measurement)

##### Key Architectural Patterns
- Pipeline Pattern: clear separation between data preprocessing, model inference, post-processing (AMEM), and I/O
- Modular Design: interchangeable backbone modules and attention mechanisms
- Strategy Pattern: selection of measurement functions (measure_av vs. measure_lv) based on input type
- Configuration-driven: behavior controlled by flags in main.py and settings in config.py

##### Main Components
a. Data Sources
- PASCAL / COCO datasets (external)
- MEIS dataset (raw_image, annotations, human results)
b. Data Preprocessing
- utils/augmentations.py
- utils/coco.py
c. Model Assembly
- modules/yolact.py (mask head, detection heads)
- modules/* (backbone and attention variants: ResNet, UPANet V2 panel attention, RTMDet, Swin Transformer)
d. Training & Evaluation Engine
- main.py (train loop, eval loop)
- eval.py (evaluation metrics, plotting)
e. Detection & Measurement
- detect.py (inference, mask post-processing, AMEM measurement algorithms)
f. Output
- utils/output_utils.py (saving images, videos, logs)
- Or saved video frames

##### Relationships and Data Flows
1. Data Source → Preprocessing → Model Input
2. Model Input → Backbone + Attention Module → Feature Maps
4. Raw Predictions → Post-Processing (NMS, decoding) → Instance Masks
5. Instance Masks → AMEM Measurement Algorithms → Numeric Outputs
6. Numeric Outputs → Output Utils → Saved Images / Videos / Logs
- Training/Evaluation: Model Input + Ground Truth → Loss Computation → Backprop → Updated Weights
- Config Flags (annotated) drive which modules are instantiated and which mode is active (train vs detect)

##### Highlighted Technologies and Frameworks
- PyTorch (model definitions, training loop)
- OpenCV (cv2 image/video I/O)
- COCO API (dataset parsing)
- Google Drive (external weight storage)
- NumPy / Pandas (data handling for MEIS, human labels)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources"
        DS1["PASCAL/COCO Datasets"]:::data
        DS2["MEIS Echocardiography Dataset"]:::data
    end

    %% Configuration / Control
    subgraph "Configuration & Control"
        CFG["Configuration Manager"]:::control
        MAIN["Orchestrator (main.py)"]:::control
    end

    %% Preprocessing Modules
    subgraph "Data Preprocessing"
        AUG["Augmentations"]:::processing
        COCO["COCO Parsing"]:::processing
    end

    %% Model Assembly
    subgraph "Model Assembly"
        YOLO["Base Yolact Framework"]:::processing
        subgraph "Backbone & Attention Variants"
            RES["ResNet Backbone"]:::processing
            RPANEL["ResNet + Panel Attention"]:::processing
            UPAV2["UPANet V2 Panel Attention"]:::processing
            RTMD["RTMDet Backbone"]:::processing
            SWIN["Swin Transformer Backbone"]:::processing
        end
    end

    %% Training & Evaluation Engine
    subgraph "Training & Evaluation Engine"
        TRAIN["Train Loop"]:::processing
        EVAL["Evaluation Engine"]:::processing
    end

    %% Detection & Measurement Pipeline
    subgraph "Detection & Measurement"
        DET["Detection & Measurement Pipeline"]:::processing
        AMEM["AMEM Measurement Algorithms"]:::processing
    end

    %% Utilities
    subgraph "Utilities"
        BOX["Box & Mask Operations"]:::util
        COMMON["Common Helper Functions"]:::util
        TIMER["Timing & Profiling"]:::util
        OUTPUT["Output Utilities"]:::io
    end

    %% Data Flows
    DS1 -->|"images"| AUG
    DS1 -->|"annotations"| COCO
    DS2 -->|"raw images"| AUG
    DS2 -->|"human labels"| COMMON

    CFG -.->|flags| MAIN
    MAIN -->|"select mode"| TRAIN
    MAIN -->|"select mode"| EVAL
    MAIN -->|"select mode"| DET

    AUG -->|"augmented images"| YOLO
    COCO -->|"parsed data"| YOLO
    COMMON --> YOLO

    YOLO -->|"feature maps"| RES
    YOLO -->|"feature maps"| RPANEL
    YOLO -->|"feature maps"| UPAV2
    YOLO -->|"feature maps"| RTMD
    YOLO -->|"feature maps"| SWIN

    RES -->|"enhanced features"| YOLO
    RPANEL -->|"enhanced features"| YOLO
    UPAV2 -->|"enhanced features"| YOLO
    RTMD -->|"enhanced features"| YOLO
    SWIN -->|"enhanced features"| YOLO

    TRAIN -->|"model input + gt"| YOLO
    TRAIN -->|"loss"| COMMON
    TRAIN -->|"updated weights"| YOLO

    EVAL -->|"model input + gt"| YOLO
    EVAL -->|"metrics"| COMMON

    DET -->|"images/video"| YOLO
    YOLO -->|"raw predictions"| BOX
    BOX -->|"filtered masks"| AMEM
    AMEM -->|"measurements"| OUTPUT

    DET -->|"timing info"| TIMER
    TRAIN -->|"timing info"| TIMER
    EVAL -->|"timing info"| TIMER

    OUTPUT -->|"saved images/videos/logs"| MAIN

    %% Click Events
    click CFG "https://github.com/hanktseng131415go/ramem/blob/main/config.py"
    click MAIN "https://github.com/hanktseng131415go/ramem/blob/main/main.py"
    click DET "https://github.com/hanktseng131415go/ramem/blob/main/detect.py"
    click EVAL "https://github.com/hanktseng131415go/ramem/blob/main/eval.py"
    click AUG "https://github.com/hanktseng131415go/ramem/blob/main/utils/augmentations.py"
    click COCO "https://github.com/hanktseng131415go/ramem/blob/main/utils/coco.py"
    click YOLO "https://github.com/hanktseng131415go/ramem/blob/main/modules/yolact.py"
    click RES "https://github.com/hanktseng131415go/ramem/blob/main/modules/resnet.py"
    click RPANEL "https://github.com/hanktseng131415go/ramem/blob/main/modules/resnet_panel.py"
    click UPAV2 "https://github.com/hanktseng131415go/ramem/blob/main/modules/upanets_v2.py"
    click RTMD "https://github.com/hanktseng131415go/ramem/blob/main/modules/rtmdet.py"
    click SWIN "https://github.com/hanktseng131415go/ramem/blob/main/modules/swin_transformer.py"
    click OUTPUT "https://github.com/hanktseng131415go/ramem/blob/main/utils/output_utils.py"
    click BOX "https://github.com/hanktseng131415go/ramem/blob/main/utils/box_utils.py"
    click COMMON "https://github.com/hanktseng131415go/ramem/blob/main/utils/common_utils.py"
    click TIMER "https://github.com/hanktseng131415go/ramem/blob/main/utils/timer_mini.py"

    %% Styles
    classDef data fill:#ccffcc,stroke:#006600;
    classDef processing fill:#cce5ff,stroke:#003366;
    classDef io fill:#ffe5cc,stroke:#663300;
    classDef control fill:#e0e0e0,stroke:#999999;
    classDef util fill:#f0e0ff,stroke:#990099;
```
