alihamza-coder/deceptron-fyp-final-year-project

Purpose and Entry Points

Deceptron is a local desktop research application for multi-modal deception cue analysis of live or uploaded interview media. The UI launches through Eel at Frontend/main.py; browser-oriented launch is Frontend/web_app.py. A separate FastAPI analysis service starts at backend/server.py, with a CLI-oriented pipeline entry at backend/modules/main.py.

Desktop UI and Local Application Layer

The Eel-hosted Vanilla JS UI provides authentication, session capture, uploads/evidence vault, analysis views, dashboards, and reports. Page scripts call the backend API and manage browser media capture/preview.

• Frontend/web/index.html

• Frontend/web/js/common/api.js

• Frontend/web/js/pages/start-session.js

Local account, session, report, and upload metadata persistence is handled through TinyDB, while verification/reset emails are sent through SMTP.

• Frontend/modules/database.py

• Frontend/modules/email_sender.py

• Frontend/config.py

API Boundary and Orchestration

FastAPI is the compute-service boundary, exposing isolated voice, emotion, face, and full-pipeline endpoints. The full pipeline route coordinates media processing and combines lower-level model outputs; this cleanly separates desktop/UI state from heavy ML execution.

• backend/server.py

• backend/api/routes/pipeline.py

• backend/api/routes/face.py

Media Intake, Speaker, and Segment Stage

Video/audio input is processed into analyzable speech turns: FFmpeg extracts audio, PyAnnote diarizes speakers and identifies the primary subject, then segmentation combines Whisper transcripts with silence/VAD splitting. These answer-level segments are the primary units for downstream voice, language, and time-aligned visual analysis.

• backend/modules/speaker_diarizer.py

• backend/modules/segment_manager.py

• backend/modules/forensic_voice_analyzer.py

Vision and Emotion Analysis

Concurrent video modules derive behavioral features such as gaze/blinking, head pose, lip/jaw tension, facial asymmetry, and hand-to-face gestures. A separate emotion model produces time-based facial-expression classifications. The system establishes an early-session behavioral baseline and evaluates later changes relative to it.

• backend/modules/eye_gaze_module.py

• backend/modules/head_pose_module.py

• backend/modules/emotion_detection_module.py

Voice and Language Analysis

For each speech segment, the forensic voice analyzer extracts acoustic stress features and Whisper transcription. The NLP component sends transcript/context to Groq-hosted Llama 3.3 for bilingual linguistic deception indicators and cross-segment contradiction analysis; this is the principal runtime network dependency.

• backend/modules/forensic_voice_analyzer.py

• backend/modules/nlp_deception_module.py

• backend/api/routes/voice.py

Fusion, Reasoning, and Outputs

The fusion engine is the decision layer: it weights face behavior, emotion control, voice stress, NLP, and cross-modal mismatch; applies clustering/confirmation rules; and assigns risk/verdict. The reasoning engine generates English/Roman Urdu explanation text. Results are persisted locally and may include annotated or composite videos under the user’s ~/.deceptron/ storage.

• backend/modules/fusion_engine.py

• backend/modules/reasoning_engine.py

• backend/modules/deception_pipeline.py

Runtime and Deployment Constraints

Core runtimes are Python 3.9+, Eel, FastAPI, and local TinyDB. ML/media dependencies include PyTorch, MediaPipe, HSEmotion, PyAnnote, Whisper, Praat/Parselmouth, OpenCV, and FFmpeg. Whisper/PyAnnote weights can be preloaded for offline operation via backend/download_models.py; Groq NLP remains online. Both application tiers support PyInstaller packaging.

• backend/requirements.txt

• Frontend/requirements.txt

• backend/download_models.py