# giabach1106/chest_x_ray_abnormal architecture

> Architecture diagram of the GitHub repository giabach1106/chest_x_ray_abnormal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/giabach1106/chest_x_ray_abnormal
- Repository: https://github.com/giabach1106/chest_x_ray_abnormal

## Overview

##### Project type and purpose
- This is a domain‐specific machine learning pipeline for detecting abnormalities in chest X-ray images (Kaggle “vinbigdata” competition).
- It comprises multiple ML sub-pipelines: data preparation, classification (EfficientNetB4), object detection (YOLOv4 via Darknet, YOLOv5 & YOLOv7 via PyTorch), evaluation, post-processing, and submission generation.

##### File structure
- Top‐level folders:
- DATA (raw CSVs and kept placeholders)
- src/prepare_data (data cleaning, augmentation, splits)
- src/classification/EfficientNetB4 (training/eval scripts & notebooks, WandB logs)
- src/detection/yolov4/darknet (C/C++ build & inference for YOLOv4)
- src/detection/yolov5 and src/detection/yolov7 (PyTorch‐based detection pipelines, configs, utils, Flask/Triton deploy examples)
- src/eval, src/post_processing (notebooks for metric calculation and result refinement)
- Notice build scripts (Makefile, CMakeLists.txt, Dockerfiles), shell wrappers (.sh/.ps1), Jupyter notebooks for experimentation, and WandB/Comet tracking folders.

##### Key insights from README
- Focus: lung anomaly detection via classification + localization/detection.
- No centralized web front end—pure ML research code.
- Evaluation and submission CSVs are generated as final deliverables.

##### Main components & their interactions
a. Data Storage
- Raw images + annotation CSVs in DATA/ and external storage (e.g., S3 or local disk).
b. Data Preparation
- src/prepare_data: processes raw CSVs and images → augmented datasets, train/val/test splits.
→ Output: ready-to-consume datasets for each model.
c. Classification Pipeline
- src/classification/EfficientNetB4
– Python scripts & notebooks train EfficientNetB4 on binary labels.
– Uses WandB for experiment tracking.
– Produces classification model and CSV predictions.
d. Detection Pipelines
- YOLOv4 (Darknet C/C++)
– Build via Make/CMake → darknet executable
– Scripts for training, inference, streaming
- YOLOv5 & YOLOv7 (PyTorch)
– python train.py / detect.py
– YAML config files define architectures/datasets
– Utils: custom dataloaders, augmentations, experiment logging (WandB, Comet, ClearML), Dockerfiles for deployment, example Flask/Triton server
– Output: PyTorch models (.pt), ONNX/TensorRT artifacts, CSV submissions
e. Evaluation & Post-Processing
- src/eval: Jupyter notebooks compute metrics (mAP, accuracy)
- src/post_processing: refine detection masks/predictions
f. Submission
- CSV writers in each pipeline produce final submission files for competition.

##### Architectural patterns & design principles
- Modular ML pipelines: separate directories per model and stage
- Experiment reproducibility: notebooks + shell wrappers + environment files (.yaml, requirements.txt)
- Extensibility: multiple detection back-ends can be swapped (Darknet vs PyTorch)
- Clear data flow: raw → processed → train/test → model → eval → submission
- Infrastructure as code: Dockerfiles and CI config in detection folders

##### Technologies & frameworks
- Python (ML logic, PyTorch, Flask/Triton)
- Jupyter Notebooks (experimental workflows)
- Bash/PowerShell scripts (automation)
- Darknet (C/C++, GPU kernels)
- WandB/Comet/ClearML (tracking)
- Docker (containerization)
- YAML (configs for models & datasets)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    DataStorage["Data Storage"]:::data
    %% Preparation
    DataPreparation["Data Preparation\n(src/prepare_data)"]:::processing
    %% Pipelines
    Classification["Classification Pipeline\n(EfficientNetB4)"]:::compute
    YOLOv4["YOLOv4 Pipeline\n(Darknet)"]:::compute
    YOLOv5["YOLOv5 Pipeline\n(PyTorch)"]:::compute
    YOLOv7["YOLOv7 Pipeline\n(PyTorch)"]:::compute
    %% Tracking
    Tracking["Experiment Tracking\n(WandB/Comet/ClearML)"]:::tracking
    %% Post stages
    Evaluation["Evaluation\n(src/eval)"]:::processing
    PostProcessing["Post-Processing\n(src/post_processing)"]:::processing
    Submission["Submission Generation"]:::artifact

    %% Flows
    DataStorage -->|"raw images, CSVs"| DataPreparation
    DataPreparation -->|"prepared datasets"| Classification
    DataPreparation -->|"prepared datasets"| YOLOv4
    DataPreparation -->|"prepared datasets"| YOLOv5
    DataPreparation -->|"prepared datasets"| YOLOv7

    Classification -->|".pt, CSV"| Evaluation
    YOLOv4 -->|"bbox predictions, logs"| Evaluation
    YOLOv5 -->|".pt, ONNX, CSV"| Evaluation
    YOLOv7 -->|".pt, CSV"| Evaluation

    Classification -->|logs| Tracking
    YOLOv4 -->|logs| Tracking
    YOLOv5 -->|logs| Tracking
    YOLOv7 -->|logs| Tracking

    Evaluation -->|"metrics"| PostProcessing
    PostProcessing -->|"refined results"| Submission

    %% Click Events
    click DataStorage "https://github.com/giabach1106/chest_x_ray_abnormal/tree/main/DATA/"
    click DataPreparation "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/src/prepare_data/Data_processing.py"
    click Classification "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/src/classification/EfficientNetB4/train_efficientNet_b4.py"
    click YOLOv4 "https://github.com/giabach1106/chest_x_ray_abnormal/tree/main/src/detection/yolov4/darknet/Makefile"
    click YOLOv5 "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/src/detection/yolov5/train.py"
    click YOLOv7 "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/src/detection/yolov7/train.py"
    click Evaluation "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/src/eval/eval_classify.ipynb"
    click PostProcessing "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/src/post_processing/post_processing_yolov5.ipynb"
    click Submission "https://github.com/giabach1106/chest_x_ray_abnormal/blob/main/DATA/csv/sample_submission.csv"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef processing fill:#e6ffe6,stroke:#333,stroke-width:1px
    classDef compute fill:#ccffcc,stroke:#333,stroke-width:1px
    classDef tracking fill:#ffe0b3,stroke:#333,stroke-width:1px
    classDef artifact fill:#f2f2f2,stroke:#333,stroke-width:1px
```
