# oml-team/open-metric-learning architecture

> Architecture diagram of the GitHub repository oml-team/open-metric-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oml-team/open-metric-learning
- Repository: https://github.com/oml-team/open-metric-learning

## Overview

##### Main Components
- Core Library Module (“oml”): This directory contains the basis for almost every function of the system. Break it into several key submodules:
– Configs: YAML configuration files for criterions, extractors, optimizers, miners, loggers, samplers, schedulers, etc.
– Datasets: Abstractions for image, text, and audio data along with handling both training and query/gallery datasets.
– Models: Implementation of various extractors (e.g., ViTExtractor, ECAPATDNNExtractor) and support for third-party models (e.g., HuggingFace wrapper).
– Losses, Miners, and Metrics: Modules dedicated to implementing metric-learning specific losses (e.g., TripletLossWithMiner, ArcFaceLoss) and mining strategies.
– Inference: Code for running retrieval (including postprocessing and evaluation) and generating embeddings.
– Lightning & Pipelines: Adaptors for PyTorch Lightning, modular training pipelines, and config-based experiment orchestration.
– Registry and Interfaces: Components to register available options (loggers, models, transforms, etc.) and define consistent interfaces to support plugging in alternate components.

- Pipelines: Separate training and postprocessing “recipes” that group together different configuration files and scripts for experiments. These also reinforce the modular design by decoupling data converters, extractors, and validation routines.

- CI/CD & Documentation: The presence of GitHub workflows, Docker configuration (files in “ci” and Dockerfiles), and detailed documentation files underline that development and deployment (including testing and packaging via setup.py/Makefile) are key parts of the system.

##### Relationships and Data Flow
- The configuration subsystem (oml/configs) acts as a central hub where experiment parameters are defined. Downstream components (losses, miners, samplers, and pipelines) all consume these configurations.
- Data flows from data loading (datasets module) → Data Transformations (via registry of transforms) → Model(s) (extractors in models/) → Loss & mining modules during training.
- Once a training epoch is done, the resulting embeddings are passed to inference modules which use retrieval and postprocessing (found under “oml/retrieval” and “oml/postprocessor”) to compute evaluation metrics.
- Logging and distributed training aspects (oml/lightning and registered loggers) are integrated with the training loop and evaluation pipelines.
- External integrations (e.g., HuggingFace Transformers in HFWrapper, PyTorch Lightning Trainer, external logging frameworks) are clearly represented as auxiliary modules interacting with the core components.

##### Important Architectural Patterns
- Pipeline / Data Flow Architecture: Different pipelines (training vs. inference) can be visually separated. For example, show two parallel flows—one for training (including data loader, feature extraction, loss computation, backpropagation loop) and one for retrieval/inference (data loader → feature extraction → postprocessing → evaluation).
- Integration with External Tools: Include nodes for PyTorch Lightning (as the orchestration/training engine), external logging (TensorBoard, Neptune, WandB, etc.), and Docker/CI systems for deployment and testing.

##### Technologies, Frameworks, and Libraries
- Python, PyTorch, and PyTorch Lightning are the primary frameworks.
- HuggingFace Transformers for language models in the NLP pipeline.
- External logging libraries – TensorBoard, MLFlow, Neptune, ClearML, WandB.
- CI/CD using GitHub Actions and Docker for containerization.

##### Specific Elements
- Main Configuration Component (config files coming from “oml/configs” and “pipelines”)
- Data Management and Loading – Different dataset modules for training, retrieval, etc.
- Model (Feature Extractors) Components – e.g., “ViTExtractor”, “HFWrapper”, “ECAPATDNNExtractor”
- Loss and Mining Modules – e.g., “TripletLossWithMiner”, “AllTripletsMiner”
- Training Orchestration – PyTorch Lightning modules and callbacks (under “oml/lightning”)
- Inference and Retrieval Processing – “oml/inference”, “oml/retrieval” and linked postprocessors.
- Logging – components that communicate with external logging systems.
- CI/CD/Packaging – Docker, CI workflows (optional for a high-level architecture, but useful to note for deployment pipelines)
- External Services – Pre-trained model sources (HuggingFace, external weights in Zoo), and any integration with vector search systems (as suggested by retrieval pipelines).

- Grouping similar functions in clusters (e.g., have one cluster for “Training Modules” and one for “Inference Modules”).

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Library Subgraph
    subgraph "Core Library (oml)"
        Configs["Configurations"]:::core
        Datasets["Datasets"]:::core
        Models["Models (Feature Extractors)"]:::core
        Losses["Losses"]:::core
        Miners["Miners"]:::core
        Metrics["Metrics"]:::core
        Inference["Inference"]:::core
        Retrieval["Retrieval"]:::core
        Lightning["Lightning & Training"]:::core
        Pipelines["Pipelines"]:::core
        Registry["Registry"]:::core
        Interfaces["Interfaces"]:::core
    end

    %% Models Submodules Subgraph
    subgraph "Models Submodules"
        ResNet["ResNet Extractors"]:::core
        Audio["Audio Extractors"]:::core
        ViT["ViT/Clip Extractors"]:::core
        Text["Text Models"]:::core
    end

    %% CI/CD and Documentation Subgraph
    subgraph "CI/CD & Documentation"
        CICD["CI/CD"]:::external
        Docs["Documentation"]:::external
    end

    %% External Integrations Subgraph
    subgraph "External Integrations"
        PyTorch["PyTorch & Lightning"]:::external
        HuggingFace["HuggingFace Transformers"]:::external
        Logging["External Logging"]:::external
        Docker["Docker"]:::external
    end

    %% Relationships in Core Library
    Configs -->|"configFeeds"| Datasets
    Configs -->|"configFeeds"| Models
    Configs -->|"configFeeds"| Losses
    Configs -->|"configFeeds"| Miners
    Configs -->|"configFeeds"| Lightning
    Configs -->|"configFeeds"| Pipelines

    Datasets -->|"dataFlow"| Models

    Models -->|"featureExtraction"| Losses
    Models -->|"featureExtraction"| Miners

    Losses -->|"computeLoss"| Lightning
    Miners -->|"mineSamples"| Lightning

    Lightning -->|"orchestrates"| Pipelines

    Models -->|"generateEmbeds"| Inference
    Inference -->|"postProcess"| Retrieval

    Registry ---|"registers"| Configs
    Registry ---|"registers"| Datasets
    Registry ---|"registers"| Models
    Registry ---|"registers"| Losses
    Registry ---|"registers"| Miners
    Registry ---|"registers"| Lightning
    Registry ---|"registers"| Pipelines

    Interfaces ---|"definesInterfaces"| Models
    Interfaces ---|"definesInterfaces"| Datasets
    Interfaces ---|"definesInterfaces"| Losses
    Interfaces ---|"definesInterfaces"| Miners
    Interfaces ---|"definesInterfaces"| Inference
    Interfaces ---|"definesInterfaces"| Retrieval

    %% Relationships for Models Submodules
    Models --> ResNet
    Models --> Audio
    Models --> ViT
    Models --> Text

    %% External Integrations Relationships
    PyTorch ---|"trainsVia"| Lightning
    HuggingFace ---|"supports"| Text
    Logging ---|"monitors"| Lightning
    Docker ---|"containerizes"| CICD

    %% CI/CD and Documentation Relationships
    CICD ---|"ensuresCI/CD"| Pipelines
    Docs ---|"documents"| Interfaces

    %% Click Events
    click Configs "https://github.com/oml-team/open-metric-learning/tree/main/oml/configs/"
    click Datasets "https://github.com/oml-team/open-metric-learning/tree/main/oml/datasets/"
    click ResNet "https://github.com/oml-team/open-metric-learning/tree/main/oml/models/resnet/"
    click Audio "https://github.com/oml-team/open-metric-learning/tree/main/oml/models/audio/"
    click ViT "https://github.com/oml-team/open-metric-learning/tree/main/oml/models/vit_clip/"
    click Text "https://github.com/oml-team/open-metric-learning/tree/main/oml/models/texts/"
    click Losses "https://github.com/oml-team/open-metric-learning/tree/main/oml/losses/"
    click Miners "https://github.com/oml-team/open-metric-learning/tree/main/oml/miners/"
    click Metrics "https://github.com/oml-team/open-metric-learning/tree/main/oml/metrics/"
    click Inference "https://github.com/oml-team/open-metric-learning/tree/main/oml/inference/"
    click Retrieval "https://github.com/oml-team/open-metric-learning/tree/main/oml/retrieval/"
    click Lightning "https://github.com/oml-team/open-metric-learning/tree/main/oml/lightning/"
    click Pipelines "https://github.com/oml-team/open-metric-learning/tree/main/pipelines/"
    click Registry "https://github.com/oml-team/open-metric-learning/tree/main/oml/registry/"
    click Interfaces "https://github.com/oml-team/open-metric-learning/tree/main/oml/interfaces/"
    click CICD "https://github.com/oml-team/open-metric-learning/tree/main/.github/workflows/"
    click Docs "https://github.com/oml-team/open-metric-learning/tree/main/docs/"

    %% Styles
    classDef core fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
```
