How vatshayan/fake-news-detection-project works
Main Components
a. Data Sources:
• Four real-world news datasets (CSV/JSON files).
b. Data Ingestion & Storage:
• File system or a lightweight local database (e.g., SQLite) where raw datasets reside.
c. Data Preprocessing Module:
• Text cleaning (tokenization, stop-word removal, stemming/lemmatization).
• Handling missing values, normalization.
d. Feature Engineering Module:
• Extraction of textual properties (n-grams, TF-IDF, readability scores, sentiment features).
e. Machine Learning Models:
• Individual classifiers (e.g., Logistic Regression, SVM, Decision Tree).
f. Ensemble Learning Module:
• Combiner that takes individual model outputs and applies voting or stacking.
g. Model Evaluation Module:
• Cross-validation, performance metrics (accuracy, precision, recall, F1-score).
h. User Interface / Execution Layer:
• Jupyter Notebook (fake_news_code_d.ipynb) for interactive exploration.
• Python script (fake_news_code_d.py) for batch runs.
i. Output & Persistence:
• Trained model artifacts (pickle files).
• Evaluation reports and visualizations (plots, logs).
Call Out Key Architectural Patterns
• Pipeline Architecture: clearly show linear stages.
• Ensemble Pattern: group individual model components feeding into a central “Ensemble Combiner.”
Technologies & Frameworks
• Data modules: pandas, NumPy
• Text processing: NLTK/spaCy, scikit-learn’s feature‐extraction
• Models & ensemble: scikit-learn
• Notebook: Jupyter
• File-based persistence: model.pkl, metrics.csv in an “Artifacts” storage bucket.