How hanktseng131415go/ramem works
Project Type and Purpose
• This is a domain‐specific deep‐learning pipeline for real‐time automatic measurement of M-mode echocardiography using instance segmentation with panel attention. It is built on top of a modified Yolact_minimal framework, incorporating new modules (e.g., UPANet V2, panel attention) and an AMEM measurement algorithm.
• Key use cases:
• Training and fine-tuning segmentation models on both standard datasets (PASCAL, COCO) and the custom MEIS echocardiography dataset
• Real-time detection and automatic measurement on images or video streams
• Evaluation workflows, both with and without measurement, comparing against human labels
File Structure Analysis
• Top‐level orchestrator:
• main.py – handles train / evaluate / detect modes, parses command‐line args, loads config
• Configuration:
• config.py – global settings, dataset paths, model hyperparameters
• Core pipelines:
• detect.py – implements the AMEM measurement functions (measure_av, measure_lv) and handles detection logic
• eval.py – evaluation routines (without measurement)
• modules/ – model definitions for different backbones and attention mechanisms:
– resnet.py, resnet_panel.py, rtmdet.py, swin_transformer.py, upanets_v2.py, yolact.py
• External materials (readmematerials/) and raw data placeholders (raw_image, data/)
• No classical frontend/backend separation; rather a modular codebase following a pipeline pattern (data loader → model → post-processing → measurement)
Key Architectural Patterns
• Pipeline Pattern: clear separation between data preprocessing, model inference, post-processing (AMEM), and I/O
• Modular Design: interchangeable backbone modules and attention mechanisms
• Strategy Pattern: selection of measurement functions (measure_av vs. measure_lv) based on input type
• Configuration-driven: behavior controlled by flags in main.py and settings in config.py
Main Components
a. Data Sources
• PASCAL / COCO datasets (external)
• MEIS dataset (raw_image, annotations, human results)
b. Data Preprocessing
• utils/augmentations.py
• utils/coco.py
c. Model Assembly
• modules/yolact.py (mask head, detection heads)
• modules/* (backbone and attention variants: ResNet, UPANet V2 panel attention, RTMDet, Swin Transformer)
d. Training & Evaluation Engine
• main.py (train loop, eval loop)
• eval.py (evaluation metrics, plotting)
e. Detection & Measurement
• detect.py (inference, mask post-processing, AMEM measurement algorithms)
f. Output
• utils/output_utils.py (saving images, videos, logs)
• Or saved video frames
Relationships and Data Flows
1. Data Source → Preprocessing → Model Input
2. Model Input → Backbone + Attention Module → Feature Maps
4. Raw Predictions → Post-Processing (NMS, decoding) → Instance Masks
5. Instance Masks → AMEM Measurement Algorithms → Numeric Outputs
6. Numeric Outputs → Output Utils → Saved Images / Videos / Logs
• Training/Evaluation: Model Input + Ground Truth → Loss Computation → Backprop → Updated Weights
• Config Flags (annotated) drive which modules are instantiated and which mode is active (train vs detect)
Highlighted Technologies and Frameworks
• PyTorch (model definitions, training loop)
• OpenCV (cv2 image/video I/O)
• COCO API (dataset parsing)
• Google Drive (external weight storage)
• NumPy / Pandas (data handling for MEIS, human labels)