How subinoybera/toxic-tweettagger works
Project Type and Purpose
After reviewing the file tree and README, this is a production-grade, end-to-end machine learning system for hate speech detection (“Toxic-Tweet Tagger”). It combines:
• A modular ML pipeline (data ingestion → validation → feature engineering → training → evaluation) tracked via DVC.
• Experiment tracking and model registry using MLflow.
• A FastAPI inference service for real-time predictions.
• A minimal HTML/CSS frontend served by a Python script.
• MLOps automation with GitHub Actions, Docker, and deployment targets (HuggingFace Spaces, Vercel).
• Data storage in MongoDB Atlas and future monitoring hooks (Evidently drift checks, planned Prometheus/Grafana).
File Structure Analysis
Top-level directories and key files:
• app/ and run_app.py – houses the production FastAPI service and HTML templates.
• hf_serve_api/ – standalone containerized inference API for HuggingFace deployment.
• src/ – core ML pipeline code, organized by component (data_ingestion, preprocessing, validation, feature_engineering, training, evaluation), configuration, utilities, and pipeline orchestration.
• configs/ – YAML files for pipeline parameters and environment configurations.
• models/ & mlruns/ – artifacts and checkpoints managed by DVC and MLflow.
• notebooks/ – exploratory analysis and experiments.
• GitHub Actions workflows under .github/workflows for CI and release.
• Dockerfile, dvc.yaml, dvc.lock – define container build and data versioning.
• requirements-dev.txt, pytest.ini, tox.ini – development tooling and testing setup.
• static/ and templates/ – minimal user-facing UI.
Key Insights from README
The README emphasizes MLOps best practices: reproducible pipelines (DVC), experiment tracking and model registry (MLflow), CI/CD automation (GitHub Actions), containerized inference (FastAPI + Docker), and future monitoring (Evidently, Prometheus/Grafana). The UI is intentionally basic, and the core focus is on the ML system architecture.