How nikonovone/face-embeddings works
Project type and purpose
• This is a containerized machine-learning training and inference pipeline built with PyTorch Lightning. Its goal is to train a face-embeddings model, convert the trained checkpoint to ONNX, and run inference on new images.
File structure
• Top-level orchestration:
• Dockerfile, .dockerignore, Makefile – build and run the container.
• pyproject.toml, .python-version – Python dependency and environment config.
• configs/train.yaml – training hyperparameters and paths.
• Source tree under src/:
• data/ – Datamodule, Dataset, transforms.
• models/ – face embedding model definition.
• training/pipeline.py, train.py – orchestrate training loop via PyTorch Lightning Trainer.
• callbacks/ – experiment tracking, debugging, freezing layers.
• utils/ – config loader, schedulers, metrics, logger, constants.
• convert.py, inference.py – checkpoint→ONNX conversion and inference routines.
Pull additional insights from README
• Uses Make targets (build, run, train, convert, inference).
• Data layout conventions under data/ for inference.
• Training outputs saved under ./checkpoints.
a. Main components
• Container/Image layer (Docker + Make).
• Configuration store (configs/train.yaml + .env).
• Data Ingestion (raw images → src/data transforms + Datamodule).
• Model Definition (src/models/face_module.py).
• Training Engine (PyTorch Lightning Trainer in train.py & pipeline.py).
• Callbacks (experiment tracking, debug, freeze).
• Checkpoint storage (./checkpoints).
• Conversion module (convert.py → ONNX).
• Inference module (inference.py).
b. Relationships and interactions
• Data flow: raw images → Dataset → DataModule → Training Engine.
• Training Engine invokes Model + Callbacks, writes checkpoints.
• Conversion module reads checkpoints, outputs ONNX.
• Inference reads ONNX + new images, produces embeddings.
c. Architectural patterns / design principles
• Modular layering: Data / Model / Training / Inference.
• Dependency injection of config into all modules.
• Callback extensibility for tracking, debugging, freezing.
• Single-responsibility scripts for train, convert, inference.
d. Key technologies/frameworks
• Docker + Make for reproducible environment.
• PyTorch Lightning for training orchestration.
• Hydra-style config loader (src/utils/config.py).
• ONNX for portable model exporting.
• VIPS or PIL for image transforms.