# open-edge-platform/training_extensions architecture

> Architecture diagram of the GitHub repository open-edge-platform/training_extensions, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/open-edge-platform/training_extensions
- Repository: https://github.com/open-edge-platform/training_extensions

## Overview

##### Project type and purpose
- This is an open-source, low-code transfer learning framework for computer vision built on PyTorch and OpenVINO.
- It exposes both a CLI (`otx`) and an API (the `Engine` class) that let users train, infer, optimize, and export models across tasks (classification, detection, segmentation, anomaly).
- Key subdomains:
- Data ingestion & preprocessing (`src/otx/core/data`, `src/otx/data/torch`)
- Model architectures & algorithms (`src/otx/algo`)
- Training engine & scheduling (`src/otx/engine`, callbacks, strategies)
- Configuration & recipes (`src/otx/core/config`, `src/otx/recipe`, `src/otx/tools/templates`)
- Export & deployment (`src/otx/core/exporter`)

##### File structure
- Top-level “src/otx” breakup:
- cli: user entry point for CLI
- engine: orchestration of training, scheduling, adaptive batch size
- core: configuration resolution, data abstractions, metrics, model wrappers, exporter
- data: dataset factories, caching, tiling, transform libraries
- algo: task-specific model definitions, backbones, heads, losses, samplers, explainability
- recipe & tools: YAML-based recipes and code/templates for one-stop model setups
- utils: cross-cutting utilities (logging, device selection, import helpers)
- Supporting folders: docs, tests (unit, integration, perf, regression), Docker configs, CI pipelines

##### README for insights
- Reinforces the “recipe → engine → algo → export” workflow.
- Auto-configuration and templating are first-class features.
- External dependencies: PyTorch, OpenVINO toolkit, Datumaro data frontend.

##### Main components
a. User Interfaces
- CLI (`otx` entrypoint)
- API (Python import of `Engine`)
b. Engine & Orchestration
- Engine core (`src/otx/engine/engine.py`)
- Adaptive BS search, callbacks, schedulers
c. Configuration Layer
- Recipe loader (`src/otx/recipe`)
- Core config resolver (`src/otx/core/config`)
- Template generator (`src/otx/tools/templates`)
d. Data Pipeline
- Dataset factory & caching (`src/otx/core/data`)
- Torch data wrapper & transforms (`src/otx/data/torch`)
- Tiling & pre-filtering modules
e. Algorithm & Model Layer
- Task-specific folders under `algo` (classification, detection, segmentation, anomaly, explain)
- Common modules: backbones, heads, losses, layers, samplers
- Integration with PyTorch models & OpenVINO conversion
f. Export & Deployment
- Native exporter (`src/otx/core/exporter/native.py`)
- Exportable-code demo package
g. External Services & Dependencies
- PyTorch, OpenVINO, Datumaro
- GPU/XPU drivers
- Docker for environment encapsulation

##### Relationships & interactions
- CLI/API → Engine
- Engine ← Recipe & Config → Template
- Engine → Data Pipeline → Raw dataset sources
- Engine → Model Building (algo modules) → Training loop
- Training loop uses Callbacks & Strategies → progress, metrics
- Upon completion, Engine → Exporter → Deployment artifacts
- Tools (e.g., converter) act as side-utilities on models or recipes

##### Architectural patterns & design principles
- Modular layering: UI | orchestration | config | data | algorithms | export
- Plugin-style recipe/template system for extensibility
- Separation of concerns: `core` handles abstractions, `algo` implements CV logic
- Dependency injection via config/resolver for tasks, devices, hyper-params
- CI/CD pipelines ensure quality: tests, docs, code scanning

##### Essential elements
- Labels: name each component (e.g., “Recipe Loader,” “Dataset Factory,” “classification Model,” “Exporter”)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        CLI["CLI (otx)"]:::ui
        API["Python API (Engine)"]:::ui
    end

    %% Orchestration & Config
    subgraph "Orchestration & Config"
        EngineCore["Engine Core"]:::orche
        AdaptiveBS["Adaptive BS Search"]:::orche
        AutoConfig["Auto-Configurator"]:::orche
        APIUtils["API Helpers"]:::orche
        Recipe["Recipe Loader"]:::orche
        ConfigResolver["Config Resolver"]:::orche
        Template["Template Generator"]:::orche
    end

    %% Data Pipeline
    subgraph "Data Pipeline"
        DatasetFactory["Dataset Factory"]:::data
        MemCache["In-Memory Cache"]:::data
        PreFilter["Pre-filtering"]:::data
        Tiling["Tiling Module"]:::data
        Transforms["Transform Libraries"]:::data
        TorchWrapper["Torch Data Wrapper"]:::data
        Validations["Data Validations"]:::data
    end

    %% Algorithm & Models
    subgraph "Algorithm & Models"
        Classification["Classification Module"]:::algo
        Detection["Detection Module"]:::algo
        Segmentation["Segmentation Module"]:::algo
        InstSeg["Instance Segmentation"]:::algo
        Anomaly["Anomaly Module"]:::algo
        Keypoint["Keypoint Detection"]:::algo
        ExplainAlgo["Explainability"]:::algo
        CommonAlgo["Common Modules"]:::algo
    end

    %% Export & Deployment
    subgraph "Export & Deployment"
        Exporter["Native Exporter"]:::export
        DemoPkg["Exportable-Code Demo"]:::export
    end

    %% External Services
    PyTorch(("PyTorch")):::external
    OpenVINO(("OpenVINO")):::external
    Datumaro(("Datumaro")):::external
    Docker(("Docker Env")):::external

    %% Connections
    CLI -->|"invokes"| EngineCore
    API -->|"calls"| EngineCore

    EngineCore -->|"loads YAML recipe"| Recipe
    EngineCore -->|"resolves config"| ConfigResolver
    EngineCore -->|"generates templates"| Template
    EngineCore -->|"auto-configures"| AutoConfig
    EngineCore -->|"uses API helpers"| APIUtils
    EngineCore -->|"searches batch-size"| AdaptiveBS

    EngineCore -->|"executes data pipeline"| DatasetFactory
    DatasetFactory --> MemCache
    DatasetFactory --> PreFilter
    PreFilter --> Tiling
    PreFilter --> Transforms
    EngineCore -->|"builds DataLoader"| TorchWrapper
    TorchWrapper --> Validations

    EngineCore -->|"loads model"| Classification
    EngineCore --> Detection
    EngineCore --> Segmentation
    EngineCore --> InstSeg
    EngineCore --> Anomaly
    EngineCore --> Keypoint
    EngineCore --> ExplainAlgo
    EngineCore --> CommonAlgo

    Classification --> PyTorch
    Detection --> PyTorch
    Segmentation --> PyTorch
    InstSeg --> PyTorch
    Anomaly --> OpenVINO
    Keypoint --> PyTorch
    ExplainAlgo --> PyTorch

    EngineCore -->|"exports model"| Exporter
    Exporter --> DemoPkg
    Exporter --> OpenVINO
    DemoPkg --> Docker

    %% Click Events
    click CLI "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/cli/cli.py"
    click API "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/engine/engine.py"
    click EngineCore "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/engine/engine.py"
    click AdaptiveBS "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/engine/adaptive_bs/bs_search_algo.py"
    click AutoConfig "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/engine/utils/auto_configurator.py"
    click APIUtils "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/engine/utils/api.py"
    click Recipe "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/recipe"
    click ConfigResolver "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/config/data.py"
    click Template "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/tools/templates"
    click DatasetFactory "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/data/factory.py"
    click MemCache "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/data/mem_cache.py"
    click PreFilter "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/data/pre_filtering.py"
    click Tiling "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/data/dataset/tile.py"
    click Transforms "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/data/transform_libs/torchvision.py"
    click TorchWrapper "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/data/torch/torch.py"
    click Validations "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/data/torch/validations.py"
    click Classification "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/classification"
    click Detection "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/detection"
    click Segmentation "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/segmentation"
    click InstSeg "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/instance_segmentation"
    click Anomaly "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/anomaly"
    click Keypoint "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/keypoint_detection"
    click ExplainAlgo "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/algo/explain/explain_algo.py"
    click CommonAlgo "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/algo/common"
    click Exporter "https://github.com/open-edge-platform/training_extensions/blob/develop/src/otx/core/exporter/native.py"
    click DemoPkg "https://github.com/open-edge-platform/training_extensions/tree/develop/src/otx/core/exporter/exportable_code/demo"
    click Docker "https://github.com/open-edge-platform/training_extensions/blob/develop/docker/Dockerfile.cuda"

    %% Styles
    classDef ui fill:#b2fab4,stroke:#333,stroke-width:1px;
    classDef orche fill:#a0c4ff,stroke:#333,stroke-width:1px;
    classDef data fill:#ffd6a5,stroke:#333,stroke-width:1px;
    classDef algo fill:#cdb4db,stroke:#333,stroke-width:1px;
    classDef export fill:#e0e0e0,stroke:#333,stroke-width:1px;
    classDef external fill:#fff3bf,stroke:#333,stroke-width:1px;
```
