How nishitpatel01/fake_news_detection works
Project Type and Purpose
• This is a machine-learning pipeline for Fake News Detection written in Python. It contains an offline training pipeline (data preparation → feature engineering → model training) and an online inference layer (prediction via CLI or a lightweight web front end).
Main Components
a. Data Layer
• Raw data: liar_dataset/*.tsv (original Liar dataset)
• Flattened CSVs: train.csv, valid.csv, test.csv
b. Offline Training Pipeline
• DataPrep.py – reads raw CSV/TSV, tokenizes, stems, cleans, performs EDA
• FeatureSelection.py – vectorizes text (BOW, n-grams, TF-IDF), (optional Word2Vec/POS)
• classifier.py – trains multiple sklearn classifiers (Naive Bayes, LR, SVM, RF), performs grid search, evaluates, selects best models
• Model Artifacts – final_model.sav, model.pkl, my_model/
c. Online Inference Layer
• prediction.py – CLI script that loads the saved model and classifies a user-provided headline, outputs class & probability
• front.py + index + CSS/JS assets (bootstrap.css, animate.css, project.css, etc.) – lightweight web front end serving an HTML form that sends text to the backend for prediction
d. Environment & Dependencies
• Python 3.6 runtime
• Libraries: scikit-learn, numpy, scipy (installed via pip or conda)
Relationships & Data Flow
1. Raw TSV/CSV data → DataPrep.py → cleaned DataFrames
2. Cleaned data → FeatureSelection.py → feature matrices (X_train, X_test, X_valid)
3. Feature matrices → classifier.py → trained model(s) → serialized to disk
4. Online: User input (CLI or Web UI) → prediction.py (or front.py invokes prediction) → load model → return “True/False” + probability → display to user
Architectural Patterns & Design Principles
• Linear ETL pipeline for offline processing
• Separation of Concerns:
• Preprocessing, feature engineering, and model training are isolated modules
• Inference (prediction) is decoupled from training
• Web UI (front.py + static files) separated from the core ML code
• MVC-style for the web front end:
• View: index HTML + CSS
• Controller: front.py
• Model: prediction.py + serialized model