# nikonovone/face-embeddings architecture

> Architecture diagram of the GitHub repository nikonovone/face-embeddings, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nikonovone/face-embeddings
- Repository: https://github.com/nikonovone/face-embeddings

## Overview

##### Project type and purpose
- This is a containerized machine-learning training and inference pipeline built with PyTorch Lightning. Its goal is to train a face-embeddings model, convert the trained checkpoint to ONNX, and run inference on new images.

##### File structure
- Top-level orchestration:
- Dockerfile, .dockerignore, Makefile – build and run the container.
- pyproject.toml, .python-version – Python dependency and environment config.
- configs/train.yaml – training hyperparameters and paths.
- Source tree under src/:
- data/ – Datamodule, Dataset, transforms.
- models/ – face embedding model definition.
- training/pipeline.py, train.py – orchestrate training loop via PyTorch Lightning Trainer.
- callbacks/ – experiment tracking, debugging, freezing layers.
- utils/ – config loader, schedulers, metrics, logger, constants.
- convert.py, inference.py – checkpoint→ONNX conversion and inference routines.

##### Pull additional insights from README
- Uses Make targets (build, run, train, convert, inference).
- Data layout conventions under data/ for inference.
- Training outputs saved under ./checkpoints.

a. Main components
- Container/Image layer (Docker + Make).
- Configuration store (configs/train.yaml + .env).
- Data Ingestion (raw images → src/data transforms + Datamodule).
- Model Definition (src/models/face_module.py).
- Training Engine (PyTorch Lightning Trainer in train.py & pipeline.py).
- Callbacks (experiment tracking, debug, freeze).
- Checkpoint storage (./checkpoints).
- Conversion module (convert.py → ONNX).
- Inference module (inference.py).

b. Relationships and interactions
- Data flow: raw images → Dataset → DataModule → Training Engine.
- Training Engine invokes Model + Callbacks, writes checkpoints.
- Conversion module reads checkpoints, outputs ONNX.
- Inference reads ONNX + new images, produces embeddings.

c. Architectural patterns / design principles
- Modular layering: Data / Model / Training / Inference.
- Dependency injection of config into all modules.
- Callback extensibility for tracking, debugging, freezing.
- Single-responsibility scripts for train, convert, inference.

d. Key technologies/frameworks
- Docker + Make for reproducible environment.
- PyTorch Lightning for training orchestration.
- Hydra-style config loader (src/utils/config.py).
- ONNX for portable model exporting.
- VIPS or PIL for image transforms.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration Layer"
        Dockerfile["Dockerfile"]:::orchestration
        Dockerignore[".dockerignore"]:::orchestration
        Makefile["Makefile"]:::orchestration
    end

    %% Configuration
    subgraph "Configuration Store"
        TrainYAML["configs/train.yaml"]:::config
        EnvExample[".env.example"]:::config
        ConfigLoader["Config Loader\n(src/utils/config.py)"]:::config
    end

    %% Data Ingestion
    subgraph "Data Ingestion" 
        RawImages["Raw Images (Host Volume)"]:::data
        Transform["transform.py\n(src/data/transform.py)"]:::data
        Dataset["dataset.py\n(src/data/dataset.py)"]:::data
        DataModule["datamodule.py\n(src/data/datamodule.py)"]:::data
    end

    %% Core ML Components
    subgraph "Core ML Components"
        ModelDef["face_module.py\n(src/models/face_module.py)"]:::core
        subgraph "Training Engine"
            TrainScript["train.py\n(src/train.py)"]:::core
            Pipeline["pipeline.py\n(src/training/pipeline.py)"]:::core
            subgraph "Callbacks"
                DebugCB["debug.py\n(src/callbacks/debug.py)"]:::core
                TrackCB["experiment_tracking.py\n(src/callbacks/experiment_tracking.py)"]:::core
                FreezeCB["freeze.py\n(src/callbacks/freeze.py)"]:::core
            end
        end
    end

    %% Checkpoints
    Checkpoints["checkpoints/"]:::storage

    %% Post-Training
    subgraph "Post-Training"
        Converter["convert.py"]:::post
        Inference["inference.py"]:::post
    end

    %% Utility Support
    subgraph "Utility Support"
        Logger["logger.py\n(src/utils/logger.py)"]:::util
        Metrics["metrics.py\n(src/utils/metrics.py)"]:::util
        Schedulers["schedulers.py\n(src/utils/schedulers.py)"]:::util
        Constants["constants.py\n(src/utils/constants.py)"]:::util
        VIPSConfig["vips_config.py\n(src/utils/vips_config.py)"]:::util
    end

    %% Flow Connections
    Dockerfile -->|build/run| Dockerignore
    Dockerignore --> Makefile
    Makefile -->|"start container"| TrainScript
    Makefile -->|"start container"| Converter
    Makefile -->|"start container"| Inference

    TrainYAML -->|provides config| ConfigLoader
    EnvExample -->|env vars| ConfigLoader
    ConfigLoader -->|injects config| DataModule
    ConfigLoader -->|injects config| TrainScript

    RawImages -->|images| Transform
    Transform --> Dataset
    Dataset --> DataModule
    DataModule -->|batches| Pipeline

    TrainScript --> Pipeline
    Pipeline -->|loads model| ModelDef
    Pipeline -->|uses callbacks| DebugCB
    Pipeline -->|uses callbacks| TrackCB
    Pipeline -->|uses callbacks| FreezeCB
    Pipeline -->|trains| ModelDef
    Pipeline -->|saves checkpoint.ckpt| Checkpoints

    Checkpoints -->|read checkpoint| Converter
    Converter -->|exports ONNX| Inference
    Inference -->|loads ONNX + new images| DataModule
    Inference -->|outputs embeddings| RawImages

    Logger -->|supports| Pipeline
    Metrics -->|supports| TrackCB
    Schedulers -->|supports| Pipeline
    Constants -->|supports| ConfigLoader
    VIPSConfig -->|supports| Transform

    %% Click Events
    click Dockerfile "https://github.com/nikonovone/face-embeddings/tree/main/Dockerfile"
    click Dockerignore "https://github.com/nikonovone/face-embeddings/blob/main/.dockerignore"
    click Makefile "https://github.com/nikonovone/face-embeddings/tree/main/Makefile"
    click TrainYAML "https://github.com/nikonovone/face-embeddings/blob/main/configs/train.yaml"
    click EnvExample "https://github.com/nikonovone/face-embeddings/blob/main/.env.example"
    click ConfigLoader "https://github.com/nikonovone/face-embeddings/blob/main/src/utils/config.py"

    click Transform "https://github.com/nikonovone/face-embeddings/blob/main/src/data/transform.py"
    click Dataset "https://github.com/nikonovone/face-embeddings/blob/main/src/data/dataset.py"
    click DataModule "https://github.com/nikonovone/face-embeddings/blob/main/src/data/datamodule.py"

    click ModelDef "https://github.com/nikonovone/face-embeddings/blob/main/src/models/face_module.py"
    click TrainScript "https://github.com/nikonovone/face-embeddings/blob/main/src/train.py"
    click Pipeline "https://github.com/nikonovone/face-embeddings/blob/main/src/training/pipeline.py"

    click DebugCB "https://github.com/nikonovone/face-embeddings/blob/main/src/callbacks/debug.py"
    click TrackCB "https://github.com/nikonovone/face-embeddings/blob/main/src/callbacks/experiment_tracking.py"
    click FreezeCB "https://github.com/nikonovone/face-embeddings/blob/main/src/callbacks/freeze.py"

    click Checkpoints "https://github.com/nikonovone/face-embeddings/tree/main/checkpoints/"

    click Converter "https://github.com/nikonovone/face-embeddings/blob/main/src/convert.py"
    click Inference "https://github.com/nikonovone/face-embeddings/blob/main/src/inference.py"

    click Logger "https://github.com/nikonovone/face-embeddings/blob/main/src/utils/logger.py"
    click Metrics "https://github.com/nikonovone/face-embeddings/blob/main/src/utils/metrics.py"
    click Schedulers "https://github.com/nikonovone/face-embeddings/blob/main/src/utils/schedulers.py"
    click Constants "https://github.com/nikonovone/face-embeddings/blob/main/src/utils/constants.py"
    click VIPSConfig "https://github.com/nikonovone/face-embeddings/blob/main/src/utils/vips_config.py"

    %% Styles
    classDef orchestration fill:#FFA500,stroke:#333,stroke-width:1px
    classDef config fill:#F9D423,stroke:#333,stroke-width:1px
    classDef data fill:#9AE66E,stroke:#333,stroke-width:1px
    classDef core fill:#5DADE2,stroke:#333,stroke-width:1px
    classDef storage fill:#AED6F1,stroke:#333,stroke-width:1px
    classDef post fill:#B39CD0,stroke:#333,stroke-width:1px
    classDef util fill:#D3D3D3,stroke:#333,stroke-width:1px
```
