How iloncka-ds/culicidaelab works
Project Type and Purpose
• CulicidaeLab is an open-source, configuration-driven Python library for mosquito image analysis (classification, detection, segmentation).
• It’s not a monolithic web app or full-stack system; it’s a core inference engine with pluggable providers, datasets, and model backends, consumed by CLI, notebooks, and external applications (e.g., culicidaelab-server, mobile clients).
High-Level Architectural Layers
a. Configuration & Settings Layer
b. Core Services & Managers Layer
c. Predictor & Backend Layer
d. External Systems & Data Providers Layer
Main Components
a. Configuration Layer
• config_manager.py / config_models.py: loads YAML configs (app_settings, datasets, predictors, providers, species)
• settings.py: central Settings object
b. Core Services & Managers
• ResourceManager: handles caching/downloading model weights and datasets
• ProviderService: orchestrates data/weight retrieval via BaseProvider interfaces
• DatasetsManager: loads datasets based on config
• BasePredictor / BaseInferenceBackend: shared predictor interface
c. Predictor & Backend Layer
• Predictor Factory (backend_factory.py): maps Predictor classes to specific backends
• Predictor modules (classifier.py, detector.py, segmenter.py): user-facing classes
• Backends:
• classifier/_fastai.py, _onnx.py
• detector/_yolo.py
• segmenter/_sam.py
• ModelWeightsManager: versioning & retrieval of pretrained weights
d. Serve Module
• serve.py: CLI or programmatic entrypoint for serving predictions
e. External Systems
• HuggingFace, Kaggle, Roboflow (providers)
• Local filesystem / cache (for downloaded artifacts)
• Upstream datasets repositories
Relationships & Data Flow
1. User code or serve.py → get_settings()
2. Settings → config_manager → parse YAML → Settings object
3. Settings passed to Predictor constructor → ResourceManager & ProviderService invoked
4. ProviderService uses BaseProvider implementations to fetch models/datasets from external sources
5. ResourceManager caches artifacts on local disk
6. Predictor calls backend_factory → instantiates appropriate InferenceBackend
7. InferenceBackend loads model weights (via ModelWeightsManager) and ML framework (FastAI/ONNX/YOLO/SAM)
8. Predictor.predict(image) → backend.infer(image) → returns predictions
Architectural Patterns & Principles
• Dependency Injection: Settings object passed through constructors
• Factory Pattern: backend_factory selects the correct backend
• Strategy Pattern: interchangeable backends implementing the same inference interface
• Configuration-Driven: behavior entirely controlled via YAML, no hard-coded parameters
• Single Responsibility: each manager/service focuses on one concern (config, resources, providers, datasets, inference)
Technologies & Frameworks to Annotate
• Core: Python 3.11, Pydantic for config models
• ML frameworks: PyTorch/FastAI, ONNX Runtime, Ultralytics YOLO, SAM
• Providers: HuggingFace Hub SDK, Kaggle API, Roboflow API
• CLI & packaging: uv (uvicorn?), pytest for tests, mkdocs for docs
Essential Elements to Capture
• Config layer feeding all downstream components
• Core services/managers that abstract resource handling and data loading
• Predictor classes vs. backend implementations
• ModelWeightsManager as the nexus for weight versioning/download
• External providers and local cache boundary
• Entry points (Python API, CLI, notebooks)