# hii0608/classify_streaming architecture

> Architecture diagram of the GitHub repository hii0608/classify_streaming, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hii0608/classify_streaming
- Repository: https://github.com/hii0608/classify_streaming

## Overview

##### Main Components
a. Ingestion & Detection Pipeline
- Video/Camera Input (could be a live stream or video files)
- Object Detection Service (YOLOv9 container)
- Tracking Service (DeepSORT module, either in same or chained container)
- Result Sink (could be a database or file store, here PostgreSQL + MLflow)
b. Orchestration & Workflow
- Apache Airflow
- Scheduler & Webserver
- Worker/Executor
- DAGs:
– object_detection_dag.py
– model_training_dag.py
c. Model Management
- MLflow Tracking Server (runs in its own container)
- Artifact Store (points to a volume or S3, if configured)
d. Data Storage
- PostgreSQL (for metadata, detection logs, training records)
e. Infrastructure & Deployment
- Docker Compose (defines services for airflow, mlflow, postgres, detection/training image)
- init-db.sh (initializes the PostgreSQL schema)
- Dockerfile.ml (builds the ML/DL environment with YOLO & DeepSORT)

##### Relationships and Data Flows
- Detection & Tracking outputs → written to PostgreSQL.
- Airflow object_detection_dag → triggers detection container on a schedule, monitors results in Postgres.
- Airflow model_training_dag →
– reads historical detections/training data from Postgres
– invokes training code in YOLOv9 container
– logs metrics & model artifacts to MLflow
– optionally updates “production” model for detection DAG
- MLflow UI ↔ User (browses experiments, metrics, artifacts)
- Airflow UI ↔ User (monitors DAG runs & task statuses)

##### Architectural Patterns & Design Principles
– Containerized microservices (each major component runs in its own container)
– Orchestration via Docker Compose at infra-level, Airflow at workflow-level
– Event/Data-driven: outputs from detection feed into databases and trigger downstream tasks
– Separation of concerns:
- Inference pipeline vs. training pipeline
- Storage (Postgres) vs. model registry (MLflow) vs. orchestration (Airflow)

##### Technologies & Frameworks
– YOLOv9 + PyTorch for detection & training
– DeepSORT for tracking
– Apache Airflow for DAG-based orchestration
– MLflow for experiment tracking & model registry
– PostgreSQL for structured metadata/storage
– Docker & Docker Compose for service definition & orchestration
– init-db.sh for DB schema/bootstrap

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Inputs and UIs
    VideoInput["Video/Camera Input"]:::external
    AirflowUI["Airflow UI"]:::external
    MLflowUI["MLflow UI"]:::external

    %% Inference Lane
    subgraph "Inference Lane"
        direction TB
        YOLO["YOLOv9 Detection & DeepSORT"]:::compute
    end

    %% Storage Lane
    subgraph "Storage Lane"
        direction TB
        Postgres["PostgreSQL Database"]:::storage
        ArtifactStore["MLflow Artifact Store"]:::storage
    end

    %% Training Lane
    subgraph "Training Lane"
        direction TB
        YOLOTrain["YOLOv9 Training Container"]:::compute
        MLflowServer["MLflow Tracking Server"]:::storage
    end

    %% Orchestration Lane
    subgraph "Orchestration Lane"
        direction TB
        AirflowScheduler["Airflow Scheduler & Webserver"]:::orchestr
        AirflowWorker["Airflow Worker/Executor"]:::orchestr
        ObjectDetectionDAG["Object Detection DAG"]:::orchestr
        ModelTrainingDAG["Model Training DAG"]:::orchestr
    end

    %% Data Flows
    VideoInput -->|"video stream"| YOLO
    YOLO -->|"detection boxes & features"| YOLO
    YOLO -->|"tracking outputs"| Postgres
    Postgres -->|"historical data"| ModelTrainingDAG
    Postgres -->|"detection logs"| ObjectDetectionDAG

    %% Control Flows (dashed)
    ObjectDetectionDAG -.->|"trigger inference task"| YOLO
    ModelTrainingDAG -.->|"launch training task"| YOLOTrain
    YOLOTrain -->|"logs metrics & artifacts"| MLflowServer
    MLflowServer -->|"model artifacts"| ArtifactStore
    MLflowServer -.->|"update production model"| YOLO

    %% UIs
    AirflowUI -->|"view DAGs & tasks"| AirflowScheduler
    MLflowUI -->|"browse experiments"| MLflowServer

    %% Orchestration connections
    AirflowScheduler --> ObjectDetectionDAG
    AirflowScheduler --> ModelTrainingDAG
    ObjectDetectionDAG --> AirflowWorker
    ModelTrainingDAG --> AirflowWorker

    %% Component Links
    click YOLO "https://github.com/hii0608/classify_streaming/blob/origin/docker/Dockerfile.ml"
    click AirflowScheduler "https://github.com/hii0608/classify_streaming/tree/origin/airflow/"
    click ObjectDetectionDAG "https://github.com/hii0608/classify_streaming/blob/origin/airflow/dags/object_detection_dag.py"
    click ModelTrainingDAG "https://github.com/hii0608/classify_streaming/blob/origin/airflow/dags/model_training_dag.py"
    click Compose "https://github.com/hii0608/classify_streaming/blob/origin/docker-compose.yml"
    click InitDB "https://github.com/hii0608/classify_streaming/blob/origin/init-db.sh"
    click StartServices "https://github.com/hii0608/classify_streaming/blob/origin/docker/start-services.sh"
    click Requirements "https://github.com/hii0608/classify_streaming/blob/origin/requirements.txt"
    click README "https://github.com/hii0608/classify_streaming/blob/origin/README.md"

    %% Docker Compose Context (infrastructure)
    subgraph "Infrastructure (Docker Compose)" 
        direction TB
        Compose["docker-compose.yml"]:::external
        InitDB["init-db.sh"]:::external
        StartServices["start-services.sh"]:::external
        Requirements["requirements.txt"]:::external
        README["README.md"]:::external
    end

    %% Styles
    classDef compute fill:#cceeff,stroke:#006699,stroke-width:1px
    classDef orchestr fill:#ccffcc,stroke:#339933,stroke-width:1px
    classDef storage fill:#ffebcc,stroke:#ff9900,stroke-width:1px
    classDef external fill:#f2e6ff,stroke:#9966cc,stroke-width:1px
```
