# alihamza-coder/deceptron-fyp-final-year-project architecture

> Architecture diagram of the GitHub repository alihamza-coder/deceptron-fyp-final-year-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alihamza-coder/deceptron-fyp-final-year-project
- Repository: https://github.com/alihamza-coder/deceptron-fyp-final-year-project

## Overview

#### Purpose and Entry Points
Deceptron is a local desktop research application for multi-modal deception cue analysis of live or uploaded interview media. The UI launches through Eel at `Frontend/main.py`; browser-oriented launch is `Frontend/web_app.py`. A separate FastAPI analysis service starts at `backend/server.py`, with a CLI-oriented pipeline entry at `backend/modules/main.py`.

#### Desktop UI and Local Application Layer
The Eel-hosted Vanilla JS UI provides authentication, session capture, uploads/evidence vault, analysis views, dashboards, and reports. Page scripts call the backend API and manage browser media capture/preview.
- `Frontend/web/index.html`
- `Frontend/web/js/common/api.js`
- `Frontend/web/js/pages/start-session.js`

Local account, session, report, and upload metadata persistence is handled through TinyDB, while verification/reset emails are sent through SMTP.
- `Frontend/modules/database.py`
- `Frontend/modules/email_sender.py`
- `Frontend/config.py`

#### API Boundary and Orchestration
FastAPI is the compute-service boundary, exposing isolated voice, emotion, face, and full-pipeline endpoints. The full pipeline route coordinates media processing and combines lower-level model outputs; this cleanly separates desktop/UI state from heavy ML execution.
- `backend/server.py`
- `backend/api/routes/pipeline.py`
- `backend/api/routes/face.py`

#### Media Intake, Speaker, and Segment Stage
Video/audio input is processed into analyzable speech turns: FFmpeg extracts audio, PyAnnote diarizes speakers and identifies the primary subject, then segmentation combines Whisper transcripts with silence/VAD splitting. These answer-level segments are the primary units for downstream voice, language, and time-aligned visual analysis.
- `backend/modules/speaker_diarizer.py`
- `backend/modules/segment_manager.py`
- `backend/modules/forensic_voice_analyzer.py`

#### Vision and Emotion Analysis
Concurrent video modules derive behavioral features such as gaze/blinking, head pose, lip/jaw tension, facial asymmetry, and hand-to-face gestures. A separate emotion model produces time-based facial-expression classifications. The system establishes an early-session behavioral baseline and evaluates later changes relative to it.
- `backend/modules/eye_gaze_module.py`
- `backend/modules/head_pose_module.py`
- `backend/modules/emotion_detection_module.py`

#### Voice and Language Analysis
For each speech segment, the forensic voice analyzer extracts acoustic stress features and Whisper transcription. The NLP component sends transcript/context to Groq-hosted Llama 3.3 for bilingual linguistic deception indicators and cross-segment contradiction analysis; this is the principal runtime network dependency.
- `backend/modules/forensic_voice_analyzer.py`
- `backend/modules/nlp_deception_module.py`
- `backend/api/routes/voice.py`

#### Fusion, Reasoning, and Outputs
The fusion engine is the decision layer: it weights face behavior, emotion control, voice stress, NLP, and cross-modal mismatch; applies clustering/confirmation rules; and assigns risk/verdict. The reasoning engine generates English/Roman Urdu explanation text. Results are persisted locally and may include annotated or composite videos under the user’s `~/.deceptron/` storage.
- `backend/modules/fusion_engine.py`
- `backend/modules/reasoning_engine.py`
- `backend/modules/deception_pipeline.py`

#### Runtime and Deployment Constraints
Core runtimes are Python 3.9+, Eel, FastAPI, and local TinyDB. ML/media dependencies include PyTorch, MediaPipe, HSEmotion, PyAnnote, Whisper, Praat/Parselmouth, OpenCV, and FFmpeg. Whisper/PyAnnote weights can be preloaded for offline operation via `backend/download_models.py`; Groq NLP remains online. Both application tiers support PyInstaller packaging.
- `backend/requirements.txt`
- `Frontend/requirements.txt`
- `backend/download_models.py`

Main entry points: [`backend/modules/deception_pipeline.py`](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/deception_pipeline.py) (Deception pipeline), [`Frontend/web/index.html`](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/index.html) (Vanilla JS UI), [`backend/modules/forensic_voice_analyzer.py`](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/forensic_voice_analyzer.py) (Forensic voice analysis), [`backend/modules/fusion_engine.py`](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/fusion_engine.py) (Fusion and verdict)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_desktop["Desktop application"]
  node_eel_launcher["Eel desktop launcher<br/>Python entry point<br/>[main.py]"]
  node_web_launcher["Browser launcher<br/>Python entry point<br/>[web_app.py]"]
  node_web_ui["Vanilla JS UI<br/>desktop web UI<br/>[index.html]"]
  node_api_client["API client<br/>frontend transport<br/>[api.js]"]
  node_session_capture["Session capture<br/>page controller<br/>[start-session.js]"]
  node_local_data[("Local account and case data<br/>TinyDB storage<br/>[database.py]")]
  node_email["Verification email<br/>SMTP sender<br/>[email_sender.py]"]
end

subgraph group_api["Analysis service"]
  node_fastapi_server["FastAPI service<br/>API entry point<br/>[server.py]"]
  node_pipeline_route["Full pipeline route<br/>orchestration endpoint<br/>[pipeline.py]"]
  node_voice_route["Voice endpoint<br/>analysis endpoint<br/>[voice.py]"]
  node_face_route["Face endpoint<br/>analysis endpoint<br/>[face.py]"]
end

subgraph group_pipeline["Media analysis pipeline"]
  node_diarization["Speaker and segment stage<br/>media preparation"]
  node_voice_analysis["Forensic voice analysis<br/>acoustic and transcript analysis"]
  node_vision_analysis["Behavioral vision analysis<br/>visual feature extraction<br/>[eye_gaze_module.py]"]
  node_emotion_analysis["Emotion analysis<br/>expression classifier"]
  node_nlp_analysis["Linguistic deception analysis<br/>LLM analysis"]
  node_fusion["Fusion and verdict<br/>decision engine<br/>[fusion_engine.py]"]
  node_reasoning["Bilingual reasoning<br/>explanation generator"]
  node_deception_pipeline["Deception pipeline<br/>pipeline coordinator"]
  node_model_preload["Offline model preload<br/>model bootstrap<br/>[download_models.py]"]
end

node_groq{{"Groq Llama 3.3<br/>external LLM service"}}
node_analysis_storage[("Analysis artifacts<br/>local filesystem")]

node_eel_launcher -->|"hosts"| node_web_ui
node_web_launcher -->|"launches"| node_web_ui
node_web_ui -->|"uses"| node_api_client
node_session_capture -->|"submits media"| node_api_client
node_web_ui -->|"manages local state"| node_local_data
node_web_ui -->|"triggers verification"| node_email
node_api_client -->|"calls"| node_fastapi_server
node_fastapi_server -->|"serves"| node_pipeline_route
node_fastapi_server -->|"serves"| node_voice_route
node_fastapi_server -->|"serves"| node_face_route
node_pipeline_route -->|"runs"| node_deception_pipeline
node_voice_route -->|"runs"| node_voice_analysis
node_face_route -->|"runs"| node_vision_analysis
node_deception_pipeline -->|"prepares speech turns"| node_diarization
node_diarization -->|"provides segments"| node_voice_analysis
node_deception_pipeline -->|"analyzes video"| node_vision_analysis
node_deception_pipeline -->|"analyzes expressions"| node_emotion_analysis
node_voice_analysis -->|"provides transcripts"| node_nlp_analysis
node_nlp_analysis -->|"queries"| node_groq
node_voice_analysis -->|"voice stress"| node_fusion
node_vision_analysis -->|"behavior features"| node_fusion
node_emotion_analysis -->|"emotion changes"| node_fusion
node_nlp_analysis -->|"linguistic cues"| node_fusion
node_fusion -->|"verdict and evidence"| node_reasoning
node_reasoning -->|"explanations"| node_deception_pipeline
node_deception_pipeline -->|"writes artifacts"| node_analysis_storage
node_model_preload -.->|"preloads PyAnnote"| node_diarization
node_model_preload -.->|"preloads Whisper"| node_voice_analysis

click node_eel_launcher "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/main.py"
click node_web_launcher "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web_app.py"
click node_web_ui "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/index.html"
click node_api_client "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/js/common/api.js"
click node_session_capture "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/js/pages/start-session.js"
click node_local_data "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/modules/database.py"
click node_email "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/modules/email_sender.py"
click node_fastapi_server "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/server.py"
click node_pipeline_route "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/api/routes/pipeline.py"
click node_voice_route "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/api/routes/voice.py"
click node_face_route "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/api/routes/face.py"
click node_diarization "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/speaker_diarizer.py"
click node_voice_analysis "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/forensic_voice_analyzer.py"
click node_vision_analysis "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/eye_gaze_module.py"
click node_emotion_analysis "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/emotion_detection_module.py"
click node_nlp_analysis "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/nlp_deception_module.py"
click node_fusion "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/fusion_engine.py"
click node_reasoning "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/reasoning_engine.py"
click node_deception_pipeline "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/deception_pipeline.py"
click node_model_preload "https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/download_models.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_eel_launcher,node_web_launcher,node_web_ui,node_api_client,node_session_capture,node_local_data,node_email toneBlue
class node_fastapi_server,node_pipeline_route,node_voice_route,node_face_route toneAmber
class node_diarization,node_voice_analysis,node_vision_analysis,node_emotion_analysis,node_nlp_analysis,node_fusion,node_reasoning,node_deception_pipeline,node_model_preload toneMint
class node_groq,node_analysis_storage toneNeutral
```

## Components

### Desktop application

- [Eel desktop launcher](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/main.py): Python entry point · `Frontend/main.py`
- [Browser launcher](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web_app.py): Python entry point · `Frontend/web_app.py`
- [Vanilla JS UI](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/index.html): desktop web UI · Capture, evidence, analysis, dashboard, and report views. · `Frontend/web/index.html`
- [API client](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/js/common/api.js): frontend transport · `Frontend/web/js/common/api.js`
- [Session capture](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/web/js/pages/start-session.js): page controller · `Frontend/web/js/pages/start-session.js`
- [Local account and case data](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/modules/database.py): TinyDB storage · Accounts, sessions, reports, and upload metadata. · `Frontend/modules/database.py`
- [Verification email](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/Frontend/modules/email_sender.py): SMTP sender · `Frontend/modules/email_sender.py`

### Analysis service

- [FastAPI service](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/server.py): API entry point · `backend/server.py`
- [Full pipeline route](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/api/routes/pipeline.py): orchestration endpoint · `backend/api/routes/pipeline.py`
- [Voice endpoint](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/api/routes/voice.py): analysis endpoint · `backend/api/routes/voice.py`
- [Face endpoint](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/api/routes/face.py): analysis endpoint · `backend/api/routes/face.py`

### Media analysis pipeline

- [Speaker and segment stage](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/speaker_diarizer.py): media preparation · FFmpeg audio extraction, PyAnnote diarization, and answer-level segmentation. · `backend/modules/speaker_diarizer.py`
- [Forensic voice analysis](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/forensic_voice_analyzer.py): acoustic and transcript analysis · Extracts stress features and Whisper transcripts per segment. · `backend/modules/forensic_voice_analyzer.py`
- [Behavioral vision analysis](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/eye_gaze_module.py): visual feature extraction · Gaze, blink, pose, facial tension, asymmetry, and gestures. · `backend/modules/eye_gaze_module.py`
- [Emotion analysis](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/emotion_detection_module.py): expression classifier · Builds an early baseline and detects later expression changes. · `backend/modules/emotion_detection_module.py`
- [Linguistic deception analysis](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/nlp_deception_module.py): LLM analysis · Bilingual cues and cross-segment contradictions. · `backend/modules/nlp_deception_module.py`
- [Fusion and verdict](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/fusion_engine.py): decision engine · Weights modalities, applies confirmation rules, and assigns risk. · `backend/modules/fusion_engine.py`
- [Bilingual reasoning](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/reasoning_engine.py): explanation generator · `backend/modules/reasoning_engine.py`
- [Deception pipeline](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/modules/deception_pipeline.py): pipeline coordinator · Persists results and can generate annotated media. · `backend/modules/deception_pipeline.py`
- [Offline model preload](https://github.com/alihamza-coder/deceptron-fyp-final-year-project/blob/main/backend/download_models.py): model bootstrap · Preloads Whisper and PyAnnote weights. · `backend/download_models.py`

### Other components

- **Groq Llama 3.3**: external LLM service · Principal online runtime dependency.
- **Analysis artifacts**: local filesystem · Results and annotated/composite media under ~/.deceptron/.

## Connections

- Eel desktop launcher → Vanilla JS UI: hosts
- Browser launcher → Vanilla JS UI: launches
- Vanilla JS UI → API client: uses
- Session capture → API client: submits media
- Vanilla JS UI → Local account and case data: manages local state
- Vanilla JS UI → Verification email: triggers verification
- API client → FastAPI service: calls
- FastAPI service → Full pipeline route: serves
- FastAPI service → Voice endpoint: serves
- FastAPI service → Face endpoint: serves
- Full pipeline route → Deception pipeline: runs
- Voice endpoint → Forensic voice analysis: runs
- Face endpoint → Behavioral vision analysis: runs
- Deception pipeline → Speaker and segment stage: prepares speech turns
- Speaker and segment stage → Forensic voice analysis: provides segments
- Deception pipeline → Behavioral vision analysis: analyzes video
- Deception pipeline → Emotion analysis: analyzes expressions
- Forensic voice analysis → Linguistic deception analysis: provides transcripts
- Linguistic deception analysis → Groq Llama 3.3: queries
- Forensic voice analysis → Fusion and verdict: voice stress
- Behavioral vision analysis → Fusion and verdict: behavior features
- Emotion analysis → Fusion and verdict: emotion changes
- Linguistic deception analysis → Fusion and verdict: linguistic cues
- Fusion and verdict → Bilingual reasoning: verdict and evidence
- Bilingual reasoning → Deception pipeline: explanations
- Deception pipeline → Analysis artifacts: writes artifacts
- Offline model preload → Speaker and segment stage: preloads PyAnnote
- Offline model preload → Forensic voice analysis: preloads Whisper