How sea-ai/seametrics works
Project Type and Purpose
• Type: Open-source Python library (no separate frontend or backend service).
• Purpose: Provide modular implementations of computer-vision and related metrics (detection, panoptic, tracking, horizon, payload processing, user-friendly wrappers).
File Structure and Key Modules
At top level you have:
• Configuration & CI:
– .github/workflows/*.yml (CI pipelines, tests, SonarCloud)
– pyproject.toml, requirements.txt, sonar-project.properties
• Notebooks: Jupyter notebooks for demos & experiments.
• Seametrics package:
– seametrics/annotations (softmin, label utils)
– seametrics/detection (cocoeval-based mAP, pr-rec-f1)
– seametrics/panoptic (PQ metric, TM implementations)
– seametrics/tracking (track metrics)
– seametrics/horizon (horizon metrics computations)
– seametrics/payload (payload processing utilities)
– seametrics/user_friendly (high-level wrappers)
– seametrics/constants.py, seametrics/customMAP (AP wrapper)
• Tests: Unit tests mirror each submodule under tests/*.
README for Additional Insights
• Core functionality outlined: PrecisionRecallF1Support for detection, hugging-face integration hints.
• Optional dependencies (e.g., fiftyone) for extended workflows.
Main Components & Their Interactions
A. Library Core Modules (group into logical layers):
1. Core Utilities
– utils modules (shared utilities, constant definitions)
– compute_softmin, label2id mappings
2. Metric Implementations
– Detection metrics (cocoeval‐wrapped, pure numpy, torchmetrics adapter)
– Panoptic Quality (PQ) metrics
– Tracking metrics
– Horizon metrics
– Payload processor
3. User-Friendly Interface
– seametrics/user_friendly (one-line calls, simplified APIs)
B. External Dependencies
– numpy (numerical arrays)
– torchmetrics (optional)
– fiftyone (optional)
– pytest (testing)
C. CI/CD & Quality
– GitHub Actions pipelines (unit tests, linting, SonarCloud upload)
• User code → imports seametrics/user_friendly → internally calls specific metric modules → utils & constants → returns results.
• Notebooks → import seametrics modules directly for experimentation.
• CI pipelines → run tests against seametrics modules.
Architectural Patterns & Design Principles
• Modular subpackages per metric domain (separation of concerns).
• Shared utility layer for code reuse.
• Adapter pattern for wrapping external evaluation code (e.g., COCOeval → torchmetrics → numpy).
• Optional-dependency gates (fiftyone integration only if installed).
• Test-first / test-driven structure: one test folder per submodule.
Essential Elements to Capture
– Subpackage boundaries and roles.
– Utility/core library relationships.
– Entry points for end users (user_friendly API, notebooks).
– External dependencies and optional integrations.
– CI/CD pipelines and their relation to code/test artifacts.