How srujanrana07/fakeifi_deepfake_voice_detection_pipeline works
1. This repository is centered on a Python backend for deepfake voice detection, with a research/ML training history and UI mockups around it. The main production path is a Flask API that accepts audio, processes it, runs inference, and returns a label plus confidence.
2. The core runtime boundary is:
audio client or frontend -> Flask app in app.py -> audio preprocessing / feature extraction in voice_analysis.py -> model inference in voice_model.py -> JSON response back to the client.
3. voice_analysis.py is the signal-processing layer. Based on the README, it performs the audio normalization pipeline: resampling to 16 kHz, mono conversion, optional silence trimming, amplitude normalization, and feature extraction such as MFCCs, spectrogram-related features, pitch, and silence statistics.
4. voice_model.py is the inference layer. It likely loads the trained CNN artifact from saved_models/SceneFake_CNN_SMOTE.h5 and uses models/label_encoder.joblib to map numeric outputs back to human-readable classes such as Real and Fake.
5. The primary model artifact is a 1D CNN trained on MFCC feature maps. The README indicates a binary classifier with ReLU/MaxPooling/Dropout blocks, Adam optimization, and sparse categorical cross-entropy. This suggests the backend is optimized for feature-based inference rather than raw waveform classification.
6. There are also classical ML artifacts under models/:
- random_forest.joblib
- knn.joblib
- label_encoder.joblib
These indicate a second branch of experimentation or fallback baselines, but the README makes clear the CNN is the main deployed model family.
7. Training and model exploration live in notebook/. CNN_Model_1.ipynb and RF_KNN.ipynb separate the deep learning pipeline from the baseline experiments, so the repo cleanly separates research notebooks from the runtime backend code.
8. The UI is not committed as source code here; only preview assets are present under assets/ui/web/. The README says the frontend is private, React-based, and communicates with the Flask backend. So for architecture diagrams, treat the frontend as an external client boundary rather than part of this repo’s source of truth.
9. app.py is the integration entry point and probably the only executable service in the repo. It likely owns request validation, file upload handling, orchestration of preprocessing and prediction, and response formatting. In architecture terms, this is the API gateway between user audio and the ML pipeline.
10. The data and model boundaries are important:
- raw audio is ephemeral request data
- extracted features are intermediate backend data
- .h5 and .joblib files are persisted model assets
- assets/ui/ contains static documentation/preview media, not application logic
11. The project is a research prototype rather than a production ML platform. There is no visible training service, feature store, database, message queue, or external inference service. That means the deployed architecture is intentionally simple: one Flask service plus local model files.
12. For a diagram, the most accurate high-level boxes are: client/frontend, Flask API, audio preprocessing/feature extraction, CNN inference model, and model artifact storage. A secondary, clearly separate box can represent notebook-based experimentation and baseline models, since those support research but are not part of the main request path.
Main entry points: voice_model.py Voice model · app.py Flask API · README.md Project docs · voice_analysis.py Audio analysis
Components 12
Production path
Request handling and inference flow for uploaded audio.
- Frontend client · Private React UI that uploads audio to the backend.
- Flask API · Main entry point; validates requests, handles uploads, and returns prediction responses.
app.py - Audio analysis · Normalizes audio and extracts MFCC, spectrogram, pitch, and silence features.
voice_analysis.py - Voice model · Loads the trained classifier and maps predictions to labels.
voice_model.py
Research assets
Model experimentation, notebooks, and saved artifacts outside the main request path.
- CNN weights · Primary trained CNN model file used for inference.
saved_models/SceneFake_CNN_SMOTE.h5 - Label encoder · Maps model outputs to human-readable classes.
models/label_encoder.joblib - Random forest · Classical ML baseline stored as a joblib artifact.
models/random_forest.joblib - KNN baseline · Nearest-neighbor baseline stored as a joblib artifact.
models/knn.joblib - CNN notebook · Training and exploration notebook for the deep learning pipeline.
notebook/CNN_Model_1.ipynb - Baseline notebook · Notebook for random forest and KNN experiments.
notebook/RF_KNN.ipynb - Project docs · Explains the pipeline, model choices, and deployment shape.
README.md
Static previews
Non-executable UI preview assets and documentation media.
- UI preview · Committed frontend preview/documentation asset, not application code.
assets/ui/web/web.txtt
Connections 14
- Frontend client → Flask API: upload audio · The frontend submits an audio file for detection.
- Flask API → Audio analysis: preprocess · API invokes the audio normalization and feature extraction pipeline.
- Audio analysis → Voice model: feature tensor · Extracted features are forwarded for inference.
- Voice model → CNN weights: load weights · Primary classifier parameters are read from the saved CNN artifact.
- Voice model → Label encoder: decode class · Numeric predictions are translated into Real/Fake labels.
- Voice model → Flask API: prediction · Inference result is returned to the API for response formatting.
- Flask API → Frontend client: json response · API returns predicted label and confidence to the caller.
- CNN notebook → CNN weights: train/export · Notebook represents the training path for the main model artifact.
- Baseline notebook → Random forest: train/export · Notebook represents the classical baseline experiments.
- Baseline notebook → KNN baseline: train/export · Notebook represents the classical baseline experiments.
- Project docs → Flask API: documents · Repository docs describe the Flask service and request flow.
- Project docs → Audio analysis: documents · Docs outline the audio normalization and feature extraction stages.
- Project docs → Voice model: documents · Docs describe the CNN inference layer and label decoding.
- UI preview → Frontend client: preview · Static UI asset reflects the external frontend boundary.