How hii0608/classify_streaming works
Main Components
a. Ingestion & Detection Pipeline
• Video/Camera Input (could be a live stream or video files)
• Object Detection Service (YOLOv9 container)
• Tracking Service (DeepSORT module, either in same or chained container)
• Result Sink (could be a database or file store, here PostgreSQL + MLflow)
b. Orchestration & Workflow
• Apache Airflow
• Scheduler & Webserver
• Worker/Executor
• DAGs:
– object_detection_dag.py
– model_training_dag.py
c. Model Management
• MLflow Tracking Server (runs in its own container)
• Artifact Store (points to a volume or S3, if configured)
d. Data Storage
• PostgreSQL (for metadata, detection logs, training records)
e. Infrastructure & Deployment
• Docker Compose (defines services for airflow, mlflow, postgres, detection/training image)
• init-db.sh (initializes the PostgreSQL schema)
• Dockerfile.ml (builds the ML/DL environment with YOLO & DeepSORT)
Relationships and Data Flows
• Detection & Tracking outputs → written to PostgreSQL.
• Airflow object_detection_dag → triggers detection container on a schedule, monitors results in Postgres.
• Airflow model_training_dag →
– reads historical detections/training data from Postgres
– invokes training code in YOLOv9 container
– logs metrics & model artifacts to MLflow
– optionally updates “production” model for detection DAG
• MLflow UI ↔ User (browses experiments, metrics, artifacts)
• Airflow UI ↔ User (monitors DAG runs & task statuses)
Architectural Patterns & Design Principles
– Containerized microservices (each major component runs in its own container)
– Orchestration via Docker Compose at infra-level, Airflow at workflow-level
– Event/Data-driven: outputs from detection feed into databases and trigger downstream tasks
– Separation of concerns:
• Inference pipeline vs. training pipeline
• Storage (Postgres) vs. model registry (MLflow) vs. orchestration (Airflow)
Technologies & Frameworks
– YOLOv9 + PyTorch for detection & training
– DeepSORT for tracking
– Apache Airflow for DAG-based orchestration
– MLflow for experiment tracking & model registry
– PostgreSQL for structured metadata/storage
– Docker & Docker Compose for service definition & orchestration
– init-db.sh for DB schema/bootstrap