How ghelani-kirtan/rt-detr-polyp-detection works
Project Type & Purpose
• This is a modular Machine-Learning system for real-time polyp detection & classification.
• It consists of three primary subsystems:
a. Data Ingestion & Preparation (data_pipelines)
b. Model Training & Zoo (src + configs + solver + zoo)
c. Inference & Deployment (scripts + tracker + tools + s3_tools)
High-Level File-Structure Analysis
• Top-level directories:
• configs/ YAML definitions for datasets, dataloaders, optimizers, model variants (RT-DETR v1 & v2).
• data_pipelines/ CLI-driven dataset download → clean → organize → prepare into COCO format.
• src/ Core training library:
– src/core: config & workspace management
– src/data: dataloaders, dataset wrappers, transforms
– src/nn: network architectures, backbones, loss & postprocessors
– src/optim: AMP, EMA, schedulers
– src/solver: training/evaluation engines for detection & classification
– src/zoo: ready-to-use RT-DETR(-v2) implementations
• scripts/ Inference system: CLI entrypoint (run_inference.py), core inference engine, config templates.
• tracker/ ByteTrack implementation for multi-object tracking.
• tools/ Model export (ONNX, TensorRT, CoreML), profiling, training helper.
• s3_tools/ & s3_sync.py S3 + HTTP API integration for data & model storage.
• Dockerfile & docker-compose.yml Containerized environment.
Key External Interactions
• HTTP API / S3 for dataset version download (data_pipelines).
• GPU resource via CUDA & PyTorch for training & inference.
• Video or camera input for real-time inference (scripts).
Main Components
a. Data Layer
• API/S3 Downloader → Cleaner → Organizer → Preparer → COCO JSON + Images
b. Training Layer
• Config Loader (YAML) ↔ Training Engine (clas_engine / det_engine)
• Model Zoo definitions → Backbone + RT-DETR Modules
• Optimizer / Scheduler / EMA → Checkpoint (model.pth)
c. Artifact Layer
• Checkpoints → Export Tools → ONNX, TRT, CoreML artifacts
• Model Storage (Local / S3)
d. Inference Layer
• Inference CLI (run_inference.py)
• Inference Engine / High-res variant
• Model Loader (ONNX / PyTorch)
• Tracker (ByteTrack) + EMA smoothing + Visualizer → Video stream output
e. Deployment Layer
• Docker Container & Compose Orchestration
• s3_tools / s3_sync for automated model & data upload/download
Relationships & Data Flows
• “Download → Clean → Organize → Prepare” (Data Pipeline stages)
• “Prepared Dataset + YAML Config → Training Engine → Model Checkpoint”
• “Checkpoint → Export → Serialized Model”
• “Serialized Model + Config → Inference Engine → Tracked Output”
• “Inference Engine → Calls Tracker Module”
• “CLI ↔ Core Libraries”
• “Docker Compose → Spins up inference & data pipeline services”
Architectural Patterns & Principles
• Layered, modular design with clear separation of concerns.
• Config-first approach: all behavior controlled via YAML.
• CLI wrappers for each subsystem (data, train, infer).
• Pluggable model zoo and config includes for easy experiment swapping.