How oml-team/open-metric-learning works
Main Components
• Core Library Module (“oml”): This directory contains the basis for almost every function of the system. Break it into several key submodules:
– Configs: YAML configuration files for criterions, extractors, optimizers, miners, loggers, samplers, schedulers, etc.
– Datasets: Abstractions for image, text, and audio data along with handling both training and query/gallery datasets.
– Models: Implementation of various extractors (e.g., ViTExtractor, ECAPATDNNExtractor) and support for third-party models (e.g., HuggingFace wrapper).
– Losses, Miners, and Metrics: Modules dedicated to implementing metric-learning specific losses (e.g., TripletLossWithMiner, ArcFaceLoss) and mining strategies.
– Inference: Code for running retrieval (including postprocessing and evaluation) and generating embeddings.
– Lightning & Pipelines: Adaptors for PyTorch Lightning, modular training pipelines, and config-based experiment orchestration.
– Registry and Interfaces: Components to register available options (loggers, models, transforms, etc.) and define consistent interfaces to support plugging in alternate components.
• Pipelines: Separate training and postprocessing “recipes” that group together different configuration files and scripts for experiments. These also reinforce the modular design by decoupling data converters, extractors, and validation routines.
• CI/CD & Documentation: The presence of GitHub workflows, Docker configuration (files in “ci” and Dockerfiles), and detailed documentation files underline that development and deployment (including testing and packaging via setup.py/Makefile) are key parts of the system.
Relationships and Data Flow
• The configuration subsystem (oml/configs) acts as a central hub where experiment parameters are defined. Downstream components (losses, miners, samplers, and pipelines) all consume these configurations.
• Data flows from data loading (datasets module) → Data Transformations (via registry of transforms) → Model(s) (extractors in models/) → Loss & mining modules during training.
• Once a training epoch is done, the resulting embeddings are passed to inference modules which use retrieval and postprocessing (found under “oml/retrieval” and “oml/postprocessor”) to compute evaluation metrics.
• Logging and distributed training aspects (oml/lightning and registered loggers) are integrated with the training loop and evaluation pipelines.
• External integrations (e.g., HuggingFace Transformers in HFWrapper, PyTorch Lightning Trainer, external logging frameworks) are clearly represented as auxiliary modules interacting with the core components.
Important Architectural Patterns
• Pipeline / Data Flow Architecture: Different pipelines (training vs. inference) can be visually separated. For example, show two parallel flows—one for training (including data loader, feature extraction, loss computation, backpropagation loop) and one for retrieval/inference (data loader → feature extraction → postprocessing → evaluation).
• Integration with External Tools: Include nodes for PyTorch Lightning (as the orchestration/training engine), external logging (TensorBoard, Neptune, WandB, etc.), and Docker/CI systems for deployment and testing.
Technologies, Frameworks, and Libraries
• Python, PyTorch, and PyTorch Lightning are the primary frameworks.
• HuggingFace Transformers for language models in the NLP pipeline.
• External logging libraries – TensorBoard, MLFlow, Neptune, ClearML, WandB.
• CI/CD using GitHub Actions and Docker for containerization.
Specific Elements
• Main Configuration Component (config files coming from “oml/configs” and “pipelines”)
• Data Management and Loading – Different dataset modules for training, retrieval, etc.
• Model (Feature Extractors) Components – e.g., “ViTExtractor”, “HFWrapper”, “ECAPATDNNExtractor”
• Loss and Mining Modules – e.g., “TripletLossWithMiner”, “AllTripletsMiner”
• Training Orchestration – PyTorch Lightning modules and callbacks (under “oml/lightning”)
• Inference and Retrieval Processing – “oml/inference”, “oml/retrieval” and linked postprocessors.
• Logging – components that communicate with external logging systems.
• CI/CD/Packaging – Docker, CI workflows (optional for a high-level architecture, but useful to note for deployment pipelines)
• External Services – Pre-trained model sources (HuggingFace, external weights in Zoo), and any integration with vector search systems (as suggested by retrieval pipelines).
• Grouping similar functions in clusters (e.g., have one cluster for “Training Modules” and one for “Inference Modules”).
Happy diagramming!