# vinaybr03/face-liveness-detection architecture

> Architecture diagram of the GitHub repository vinaybr03/face-liveness-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vinaybr03/face-liveness-detection
- Repository: https://github.com/vinaybr03/face-liveness-detection

## Overview

1. **Repository purpose**
   - This is a face-liveness detection system that classifies a short clip as **real** or **fake/spoofed**.
   - The architecture is organized around a single model family that is trained in PyTorch, exported to ONNX, then deployed as TFLite or ONNX for inference.

2. **Core ML boundary**
   - The central model lives in `model.py` and is trained/tested as a **multi-modal classifier**.
   - Inputs are described as:
     - a short **image clip** of face frames
     - an **8-dimensional sensor vector stream** per timestep
   - The important architectural boundary is that the model consumes a fixed-length clip, not a single frame, so inference and training both operate on sequence windows.

3. **Data layer**
   - `data/train/{real,fake}` and `data/test/{real,fake}` provide a binary-labeled dataset split.
   - `dataset.py` is the ingestion layer that turns the folder structure into model-ready samples, applies preprocessing, and likely assembles clip-level examples.
   - The repo’s data contract is: RGB images resized to 224×224 plus sensor features normalized/packed into the clip.

4. **Training subsystem**
   - `train.py` is the training entry point and produces the canonical PyTorch artifact `liveness_model.pth`.
   - Training output also includes `training_history.json`, which feeds the analysis pipeline.
   - `analyze_results.py` is a separate post-training subsystem for metrics, plots, and confusion matrix generation, keeping evaluation out of the training loop.

5. **Model export pipeline**
   - The repository has a clear conversion chain:
     - `convert_torch_to_onnx.py` converts PyTorch weights to `liveness_model.onnx`
     - `convert_onnx_to_tflite.py` produces a FP32 TFLite model
     - `convert_onnx_to_tflite_int8.py` produces an INT8 quantized model
   - This export boundary is important: training happens in PyTorch, but deployment targets are lighter runtimes optimized for mobile or edge use.

6. **Deployed inference formats**
   - Three model artifacts are kept in the repo root:
     - `liveness_model.pth` for training/experimentation
     - `liveness_model.onnx` for portable runtime support
     - `liveness_model_fp32.tflite` and `liveness_model_int8.tflite` for mobile/embedded inference
   - The INT8 model is the preferred low-latency deployment path.

7. **API/server layer**
   - `app.py`, `app_tflite32.py`, and `app_onnx.py` are three alternative serving frontends over the same detection logic.
   - They represent a deployment abstraction boundary: same app behavior, different inference backends.
   - The README indicates Flask is used to expose the service on `localhost:5000`.

8. **External interaction flow**
   - The server exposes:
     - `POST /detect` for face localization
     - `POST /predict` for liveness classification
     - `GET /` for the web UI
   - The `/predict` payload combines the two modalities: a 10-frame `image_clip` plus 10 sensor readings.
   - This makes the request contract clip-based rather than image-only.

9. **Browser/UI assets**
   - `templates/index.html` is the UI shell for the Flask app.
   - `static/models/tiny_face_detector_model-*` suggests the frontend also performs client-side or browser-side face detection using a prepackaged detector model.
   - That creates a split architecture: browser handles capture/face localization UI, while the Python backend handles liveness inference.

10. **Video inference path**
   - `predict_video.py` and `predict_video_onnx.py` are offline/CLI inference paths for processing video files.
   - These scripts are separate from the Flask app, so the repo supports both interactive service deployment and batch evaluation on recorded video.

11. **Artifacts and supporting outputs**
   - `images/` stores generated plots/figures, likely from analysis or documentation.
   - `training_history.json` acts as the bridge between training and evaluation reporting.
   - The repo keeps both code and trained artifacts together, so architecture diagrams should show the model lifecycle end-to-end.

12. **Technology and runtime boundaries**
   - The stack is primarily **PyTorch + TensorFlow/TFLite + ONNX + Flask**, with `opencv-python`, `numpy`, `Pillow`, `scikit-learn`, `matplotlib`, and `seaborn` supporting preprocessing, inference, and evaluation.
   - The material architectural choice is portability: train in PyTorch, export through ONNX, deploy with TFLite or ONNX depending on performance/runtime constraints.

13. **What to draw as the main systems**
   - A useful diagram would show four main zones:
     - **Dataset and preprocessing**
     - **Training and evaluation**
     - **Model conversion/export**
     - **Deployment/inference**
   - The key flows are:
     - data folders → dataset loader → PyTorch training
     - trained weights → ONNX/TFLite conversions
     - deployed model → Flask/video inference
     - training logs → analysis/metrics

14. **Overall architectural takeaway**
   - This is not a monolithic app; it is a **model development and deployment pipeline** with multiple inference entry points.
   - The most important design decisions are the clip-based multimodal input, the binary real/fake output, and the runtime split between high-level training and edge-optimized serving.

Main entry points: [`liveness_model.onnx`](https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model.onnx) (ONNX model), [`templates/index.html`](https://github.com/vinaybr03/face-liveness-detection/blob/main/templates/index.html) (Web UI), [`model.py`](https://github.com/vinaybr03/face-liveness-detection/blob/main/model.py) (Liveness model), [`app.py`](https://github.com/vinaybr03/face-liveness-detection/blob/main/app.py) (Flask app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data layer"]
  node_data_split[("Train/Test data<br/>binary image data")]
  node_dataset_py["Dataset<br/>clip loader<br/>[dataset.py]"]
end

subgraph group_train_eval["Training & eval"]
  node_train_py["Train<br/>[train.py]"]
  node_history_json["History<br/>metrics log"]
  node_analyze_py["Analyze<br/>results reporting<br/>[analyze_results.py]"]
end

subgraph group_export["Model export"]
  node_torch_onnx_py["To ONNX<br/>export script"]
  node_onnx_tflite_py["To TFLite<br/>export script"]
  node_onnx_int8_py["To INT8<br/>quant export"]
  node_pth_artifact[("PTH model<br/>trained weights<br/>[liveness_model.pth]")]
  node_onnx_artifact[("ONNX model<br/>portable model")]
  node_tflite_fp32[("FP32 TFLite<br/>mobile model")]
  node_tflite_int8[("INT8 TFLite<br/>quantized model")]
end

subgraph group_serve["Serving & inference"]
  node_flask_app["Flask app<br/>web service<br/>[app.py]"]
  node_app_onnx_py["ONNX app<br/>serve backend<br/>[app_onnx.py]"]
  node_app_tflite_py["TFLite app<br/>serve backend<br/>[app_tflite32.py]"]
  node_video_cli["Video infer<br/>batch inference<br/>[predict_video.py]"]
  node_video_onnx_cli["Video ONNX<br/>batch inference"]
end

subgraph group_ui["Browser UI"]
  node_index_html["Web UI<br/>template<br/>[index.html]"]
  node_tiny_face_model[("Face detector<br/>browser asset")]
end

node_model_py["Liveness model<br/>multimodal classifier<br/>[model.py]"]

node_data_split -->|"samples"| node_dataset_py
node_dataset_py -->|"clips"| node_model_py
node_model_py -->|"train"| node_train_py
node_train_py -->|"save"| node_pth_artifact
node_train_py -->|"log"| node_history_json
node_history_json -->|"report"| node_analyze_py
node_pth_artifact -->|"export"| node_torch_onnx_py
node_torch_onnx_py -->|"write"| node_onnx_artifact
node_onnx_artifact -->|"convert"| node_onnx_tflite_py
node_onnx_artifact -->|"quantize"| node_onnx_int8_py
node_onnx_tflite_py -->|"write"| node_tflite_fp32
node_onnx_int8_py -->|"write"| node_tflite_int8
node_model_py -->|"predict"| node_flask_app
node_onnx_artifact -->|"serve"| node_app_onnx_py
node_tflite_fp32 -->|"serve"| node_app_tflite_py
node_tflite_int8 -->|"serve"| node_app_tflite_py
node_model_py -->|"batch"| node_video_cli
node_onnx_artifact -->|"batch"| node_video_onnx_cli
node_flask_app -->|"ui"| node_index_html
node_index_html -->|"detect"| node_tiny_face_model
node_index_html -->|"POST /predict"| node_flask_app
node_index_html -->|"POST /detect"| node_flask_app

click node_data_split "https://github.com/vinaybr03/face-liveness-detection/tree/main/data"
click node_dataset_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/dataset.py"
click node_model_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/model.py"
click node_train_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/train.py"
click node_history_json "https://github.com/vinaybr03/face-liveness-detection/blob/main/training_history.json"
click node_analyze_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/analyze_results.py"
click node_torch_onnx_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/convert_torch_to_onnx.py"
click node_onnx_tflite_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/convert_onnx_to_tflite.py"
click node_onnx_int8_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/convert_onnx_to_tflite_int8.py"
click node_pth_artifact "https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model.pth"
click node_onnx_artifact "https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model.onnx"
click node_tflite_fp32 "https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model_fp32.tflite"
click node_tflite_int8 "https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model_int8.tflite"
click node_flask_app "https://github.com/vinaybr03/face-liveness-detection/blob/main/app.py"
click node_app_onnx_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/app_onnx.py"
click node_app_tflite_py "https://github.com/vinaybr03/face-liveness-detection/blob/main/app_tflite32.py"
click node_video_cli "https://github.com/vinaybr03/face-liveness-detection/blob/main/predict_video.py"
click node_video_onnx_cli "https://github.com/vinaybr03/face-liveness-detection/blob/main/predict_video_onnx.py"
click node_index_html "https://github.com/vinaybr03/face-liveness-detection/blob/main/templates/index.html"
click node_tiny_face_model "https://github.com/vinaybr03/face-liveness-detection/blob/main/static/models/tiny_face_detector_model-weights_manifest.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_split,node_dataset_py toneBlue
class node_train_py,node_history_json,node_analyze_py toneAmber
class node_torch_onnx_py,node_onnx_tflite_py,node_onnx_int8_py,node_pth_artifact,node_onnx_artifact,node_tflite_fp32,node_tflite_int8 toneMint
class node_flask_app,node_app_onnx_py,node_app_tflite_py,node_video_cli,node_video_onnx_cli toneRose
class node_index_html,node_tiny_face_model toneIndigo
class node_model_py toneNeutral
```

## Components

### Data layer

Dataset folders and ingestion into clip-level samples.

- [Train/Test data](https://github.com/vinaybr03/face-liveness-detection/tree/main/data): binary image data · Real/fake folders for train and test splits. · `data`
- [Dataset](https://github.com/vinaybr03/face-liveness-detection/blob/main/dataset.py): clip loader · Builds fixed-length multimodal samples from folder structure and preprocessing. · `dataset.py`

### Training & eval

PyTorch training and post-training analysis.

- [Train](https://github.com/vinaybr03/face-liveness-detection/blob/main/train.py): PyTorch training · Training entry point that produces the canonical .pth artifact. · `train.py`
- [History](https://github.com/vinaybr03/face-liveness-detection/blob/main/training_history.json): metrics log · Training history consumed by analysis and reporting. · `training_history.json`
- [Analyze](https://github.com/vinaybr03/face-liveness-detection/blob/main/analyze_results.py): results reporting · Generates metrics, plots, and confusion matrix after training. · `analyze_results.py`

### Model export

Conversion path from PyTorch to ONNX and TFLite.

- [To ONNX](https://github.com/vinaybr03/face-liveness-detection/blob/main/convert_torch_to_onnx.py): export script · Converts PyTorch weights into portable ONNX format. · `convert_torch_to_onnx.py`
- [To TFLite](https://github.com/vinaybr03/face-liveness-detection/blob/main/convert_onnx_to_tflite.py): export script · Produces FP32 TFLite model from ONNX. · `convert_onnx_to_tflite.py`
- [To INT8](https://github.com/vinaybr03/face-liveness-detection/blob/main/convert_onnx_to_tflite_int8.py): quant export · Produces INT8 quantized TFLite model for low-latency deployment. · `convert_onnx_to_tflite_int8.py`
- [PTH model](https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model.pth): trained weights · Canonical PyTorch artifact for experimentation and training reuse. · `liveness_model.pth`
- [ONNX model](https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model.onnx): portable model · Runtime-neutral model used by ONNX inference and conversion paths. · `liveness_model.onnx`
- [FP32 TFLite](https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model_fp32.tflite): mobile model · Floating-point TFLite deployment artifact. · `liveness_model_fp32.tflite`
- [INT8 TFLite](https://github.com/vinaybr03/face-liveness-detection/blob/main/liveness_model_int8.tflite): quantized model · Preferred edge deployment artifact. · `liveness_model_int8.tflite`

### Serving & inference

Flask app, CLI video inference, and deployment backends.

- [Flask app](https://github.com/vinaybr03/face-liveness-detection/blob/main/app.py): web service · Main HTTP service exposing detection and prediction endpoints. · `app.py`
- [ONNX app](https://github.com/vinaybr03/face-liveness-detection/blob/main/app_onnx.py): serve backend · Serving frontend using ONNX inference. · `app_onnx.py`
- [TFLite app](https://github.com/vinaybr03/face-liveness-detection/blob/main/app_tflite32.py): serve backend · Serving frontend using TFLite inference. · `app_tflite32.py`
- [Video infer](https://github.com/vinaybr03/face-liveness-detection/blob/main/predict_video.py): batch inference · Offline video processing path using the Python model runtime. · `predict_video.py`
- [Video ONNX](https://github.com/vinaybr03/face-liveness-detection/blob/main/predict_video_onnx.py): batch inference · Offline video inference path backed by ONNX. · `predict_video_onnx.py`

### Browser UI

Web template and bundled face detector assets.

- [Web UI](https://github.com/vinaybr03/face-liveness-detection/blob/main/templates/index.html): template · Browser interface shell for capture and prediction. · `templates/index.html`
- [Face detector](https://github.com/vinaybr03/face-liveness-detection/blob/main/static/models/tiny_face_detector_model-weights_manifest.json): browser asset · Bundled frontend detector assets for client-side face localization. · `static/models/tiny_face_detector_model-weights_manifest.json`

### Other components

- [Liveness model](https://github.com/vinaybr03/face-liveness-detection/blob/main/model.py): multimodal classifier · Consumes image clips plus 8D sensor streams and predicts real vs fake. · `model.py`

## Connections

- Train/Test data → Dataset: samples · Dataset folders are ingested into clip-level training examples.
- Dataset → Liveness model: clips · Preprocessed image clips and sensor vectors feed the model.
- Liveness model → Train: train · Training optimizes the multimodal classifier.
- Train → PTH model: save · Produces the canonical PyTorch checkpoint.
- Train → History: log · Stores training metrics and history.
- History → Analyze: report · Post-training analysis reads the history file.
- PTH model → To ONNX: export · PyTorch weights are converted to ONNX.
- To ONNX → ONNX model: write · Creates the portable ONNX model.
- ONNX model → To TFLite: convert · ONNX is converted to FP32 TFLite.
- ONNX model → To INT8: quantize · ONNX is converted to INT8 TFLite.
- To TFLite → FP32 TFLite: write · Writes the floating-point TFLite artifact.
- To INT8 → INT8 TFLite: write · Writes the quantized TFLite artifact.
- Liveness model → Flask app: predict · Core liveness logic is served through Flask.
- ONNX model → ONNX app: serve · ONNX backend powers one deployment frontend.
- FP32 TFLite → TFLite app: serve · TFLite backend powers another deployment frontend.
- INT8 TFLite → TFLite app: serve · Quantized TFLite is the preferred low-latency backend.
- Liveness model → Video infer: batch · Offline video inference uses the model runtime.
- ONNX model → Video ONNX: batch · Offline video inference can also run through ONNX.
- Flask app → Web UI: ui · The web service renders the browser interface.
- Web UI → Face detector: detect · Frontend uses bundled face detector assets for localization.
- Web UI → Flask app: POST /predict · Browser submits clip-based multimodal inference requests.
- Web UI → Flask app: POST /detect · Browser requests face localization.
