How gratheon/models-queen-bee-detector works
1. What this repo is
- This is a single-purpose ML repository for a queen bee object detector.
- The repo centers on a trained YOLO-based model plus the tooling to download datasets, merge labels, train, evaluate, export, and serve inference over HTTP.
2. Top-level architecture
- The repo is mostly file and script driven, not a multi-service application.
- Main boundaries are:
- data ingestion/normalization in scripts/
- model training/evaluation/export in scripts/
- runtime inference API in src/server.py
- artifact storage in datasets/, weights/, and exports/
3. Dataset subsystem
- datasets/catalog.yaml is the machine-readable source of truth for dataset candidates and metadata.
- scripts/download_dataset.py pulls Roboflow datasets into local folders.
- scripts/merge_queen_dataset.py combines multiple source datasets into one normalized single-class queen dataset, preserving negative/background images and converting polygon annotations to bounding boxes.
- This is the main data-preparation boundary: raw external datasets come in, a normalized training dataset comes out.
4. Model training pipeline
- scripts/train.py is the training entry point, using Ultralytics YOLOv8 with a baseline yolov8n.pt backbone.
- Training reads a dataset YAML generated by the merge step and writes run artifacts under runs/.
- The README indicates the current baseline was trained at imgsz=512, epochs=60, and is optimized for queen detection specifically.
5. Evaluation pipeline
- scripts/evaluate.py evaluates saved weights against a chosen dataset split, typically test.
- The expected outputs are standard detection metrics: precision, recall, mAP50, and mAP50-95.
- This is a separate validation boundary from training, which makes it easy to compare candidate weights without retraining.
6. Export pipeline
- scripts/export_model.py converts the trained PyTorch weights into deployable formats such as ONNX and optionally TensorFlow.js.
- The repo’s intended browser path is ONNX plus onnxruntime-web.
- Export artifacts are intentionally kept out of source control, which reinforces the separation between model training and deployment packaging.
7. HTTP inference service
- src/server.py exposes the model as a lightweight HTTP service.
- The service supports:
- GET /health for readiness/weight presence
- GET / for a simple upload form
- POST / and POST /detect for multipart image inference
- The request path is image upload → model inference → detection JSON response, with optional confidence and IoU thresholds.
8. Runtime configuration and observability
- The service behavior is controlled by environment variables such as:
- model weight path
- max upload size
- confidence threshold
- IoU threshold
- Logging is integrated through the shared Gratheon logging library and supports request-scoped request IDs.
- If configured, logs can be exported to an OTLP/OpenTelemetry collector, so observability is an external runtime boundary rather than built into the model logic.
9. Deployment and packaging
- Dockerfile, docker-compose.yml, and docker-compose.dev.yml package the inference service for containerized use.
- restart.sh suggests a simple operational restart workflow.
- .github/workflows/deploy.yml indicates automated deployment from CI/CD.
- The repo is therefore deployable both as a local Python service and as a containerized runtime.
10. Artifacts and persistence
- weights/best.pt is the checked-in reference model artifact.
- datasets/, weights/, exports/, and runs/ are treated as artifact directories, but only selected reference assets are committed.
- The repo intentionally excludes large or environment-specific outputs like training runs, browser exports, and secrets.
11. Testing boundary
- tests/test_server_contract.py is a contract test for the HTTP API.
- Its presence implies the API shape is important and stabilized, especially around request/response behavior rather than internal model details.
- This makes the inference service a clear externally consumed component.
12. External dependencies that matter architecturally
- Roboflow is the upstream dataset source.
- Ultralytics YOLOv8 is the model/training framework.
- OpenTelemetry/OTLP is used for optional tracing/log export.
- A browser/web-app consumer is expected to call the HTTP service or consume exported ONNX/TFJS artifacts, but that frontend is outside this repo.
13. Overall data flow
- External bee datasets are downloaded and normalized locally.
- The merged dataset feeds YOLOv8 training.
- Trained weights are evaluated, exported, and stored.
- The best.pt artifact is then loaded by the HTTP service for live image detection.
- The repo is therefore a compact ML lifecycle repository with a single production inference boundary and several offline preparation stages.
Main entry points: best.pt Weights · datasets Datasets · train.py Train · Dockerfile Dockerfile
Components 17
Data prep
Dataset cataloging, download, and normalization into a single-class queen dataset.
- Dataset catalog · Machine-readable source of dataset candidates and labels.
datasets/catalog.yaml - Download · Pulls external Roboflow datasets into local storage.
scripts/download_dataset.py - Merge queen · Combines source datasets into a single-class queen dataset and converts annotations to boxes.
scripts/merge_queen_dataset.py - Datasets · Local dataset workspace and prepared training data.
datasets
Model lifecycle
Training, evaluation, and export of the detector weights.
- Train config · Training presets for queen bee detector runs.
configs/queen_bee.yaml - Primary config · Alternate or baseline training configuration.
configs/queen_bee_primary.yaml - Train · Ultralytics YOLOv8 training entry point using prepared dataset YAML.
scripts/train.py - Evaluate · Scores saved weights on a chosen split and reports detection metrics.
scripts/evaluate.py - Export · Converts trained weights to deployable formats such as ONNX and optional TFJS.
scripts/export_model.py - Weights · Checked-in reference detector weights and model outputs.
weights/best.pt - Exports · Exported runtime artifacts kept separate from source control.
exports
Serving
HTTP inference service and its contract.
- HTTP server · Lightweight image upload and detection service.
src/server.py - Contract test · Verifies the external HTTP request and response behavior.
tests/test_server_contract.py
Packaging
Containerization, deployment automation, and operational helpers.
- Dockerfile · Builds the inference service container image.
Dockerfile - Compose · Container definitions for local and development deployment.
docker-compose.yml - Dev compose · Development-oriented container composition.
docker-compose.dev.yml - Deploy CI · Automated deployment pipeline from GitHub Actions.
.github/workflows/deploy.yml
Connections 19
- Dataset catalog → Download: drives · Dataset catalog determines which sources are fetched.
- Download → Datasets: writes · Downloaded raw datasets are stored locally.
- Datasets → Merge queen: inputs · Merged dataset builder consumes downloaded source data.
- Merge queen → Datasets: produces · Normalization outputs a consolidated queen dataset.
- Train config → Train: configures · Training preset guides YOLOv8 runs.
- Primary config → Train: configures · Alternate baseline configuration feeds training.
- Datasets → Train: feeds · Prepared dataset YAML and images are used for training.
- Train → Weights: emits · Training produces model checkpoints and best weights.
- Weights → Evaluate: evaluates · Saved weights are scored against a dataset split.
- Evaluate → Weights: validates · Evaluation confirms candidate model quality.
- Weights → Export: exports · Trained weights are converted into deployable formats.
- Export → Exports: writes · Export pipeline stores ONNX/TFJS artifacts.
- Weights → HTTP server: loads · HTTP service uses the reference detector weights at runtime.
- HTTP server → Contract test: verified by · Contract test locks down API behavior.
- Dockerfile → HTTP server: packages · Container image wraps the inference server.
- Compose → Dockerfile: orchestrates · Compose defines the runtime container setup.
- Dev compose → Dockerfile: orchestrates · Dev compose mirrors the service for local iteration.
- Deploy CI → Compose: deploys · CI/CD workflow supports automated deployment.
- Deploy CI → Dockerfile: builds · Deployment pipeline builds container artifacts.