How velocitatem/diet-coke works
Project type and purpose
• This is a command‑line knowledge‑distillation pipeline written in Python.
• Purpose: fine‑tune a BERT “teacher” on the IMDB sentiment dataset, extract soft predictions, train a lightweight Decision Tree “student” on TF‑IDF features to mimic the teacher, then evaluate both.
• Key use cases: efficient inference, low‑memory interpretable model.
File structure
• Top‑level directories and files
• configs/ : Hydra YAML configuration for data, model, distillation, training
• src/ : main Python package
– src/data : data loading (IMDBDatamodule), TF‑IDF transforms
– src/models : teacher wrapper, student wrapper, Distiller class
– src/train_teacher.py : BERT fine‑tuning entry point
– src/distill_to_tree.py : distillation entry point
– src/evaluate.py : evaluation script
– src/registry : model registry CLI and JSON store
– src/utils : logging, seeding utilities
• model_registry/ : JSON and CSV files tracking trained models and metrics
• scripts/ : environment setup and “run_*” pipeline orchestration
• setup.py, requirements*.txt : packaging and dependencies
• CI/CD
• .github/workflows/ci.yml runs tests, style checks.
README for additional insights
• Architecture is linear: data → teacher train → feature extract → distill → evaluate
• Uses Hydra for configuration overrides
• Logs metrics to TensorBoard under outputs/
Detailed capturing of essential architecture
• Hydra config central node feeding into all stages