How velocitatem/diet-coke works
<explanation>
1. Identify the project type and purpose
- This is a command‑line knowledge‑distillation pipeline written in Python.
- Purpose: fine‑tune a BERT “teacher” on the IMDB sentiment dataset, extract soft predictions, train a lightweight Decision Tree “student” on TF‑IDF features to mimic the teacher, then evaluate both.
- Key use cases: efficient inference, low‑memory interpretable model.
2. Analyze the file structure
- Top‑level directories and files
• configs/ : Hydra YAML configuration for data, model, distillation, training
• src/ : main Python package
– src/data : data loading (IMDBDatamodule), TF‑IDF transforms
– src/models : teacher wrapper, student wrapper, Distiller class
– src/train_teacher.py : BERT fine‑tuning entry point
– src/distill_to_tree.py : distillation entry point
– src/evaluate.py : evaluation script
– src/registry : model registry CLI and JSON store
– src/utils : logging, seeding utilities
• model_registry/ : JSON and CSV files tracking trained models and metrics
• scripts/ : environment setup and “run_*” pipeline orchestration
• setup.py, requirements*.txt : packaging and dependencies
- CI/CD
• .github/workflows/ci.yml runs tests, style checks.
3. Examine the README for additional insights
- Architecture is linear: data → teacher train → feature extract → distill → evaluate
- Uses Hydra for configuration overrides
- Logs metrics to TensorBoard under outputs/
4. How to create the system design diagram
a. Main components:
1. Data Ingestion & Preprocessing
- IMDBDatamodule (PyTorch Lightning style)
- TF‑IDF transformer
2. Teacher Training
- BERT model (HuggingFace)
- Trainer wrapper (train_teacher.py)
3. Distillation
- Soft‑label extraction from teacher
- Distiller class orchestrating student training
4. Student Training
- Decision Tree classifier (scikit‑learn)
5. Evaluation
- evaluate.py comparing metrics
6. Model Registry
- JSON/CSV files, CLI in src/registry/cli.py
7. Configuration
- Hydra configs (configs/*.yaml)
8. Orchestration Scripts
- shell scripts under scripts/ for env setup and pipeline runs
9. Logging & Monitoring
- TensorBoard logs via Lightning or custom logger
b. Relationships & data flows:
- Raw IMDB data → Data Ingestion → TF‑IDF features
- Features → Teacher Training → Fine‑tuned BERT checkpoint
- Teacher checkpoint + features → Distillation → Student model
- Both models + test set → Evaluation → metrics
- All models & metrics → Model Registry
- Config overrides flow into each stage via Hydra
c. Architectural patterns & principles:
- Pipeline pattern (linear stages)
- Modular design: separation of data, models, registry, utils
- Extensibility: Hydra config overrides, CLI hooks in registry
d. Key technologies & frameworks:
- Python, PyTorch Lightning (implicit from Datamodule pattern)
- HuggingFace Transformers (BERT teacher)
- scikit‑learn (Decision Tree student, TF‑IDF)
- Hydra (configuration management)
- TensorBoard (logging)
- JSON/CSV model registry
5. Tailoring guidelines for this pipeline tool
- Emphasize sequential stages rather than client‑server separation
- Show clearly how config parameters (via Hydra) feed into each script
- For extensibility, highlight registry CLI and configuration landing points
6. Diagram elements to include
- Box for each component with a clear, concise label
- Directional arrows showing flow of datasets, model artifacts, configs
- Color coding:
• Blue for data & preprocessing
• Green for teacher training
• Orange for distillation & student training
• Purple for evaluation & registry
- Distinct shapes:
• Cylinders for data stores (raw data, model_registry JSON/CSV)
• Rounded rectangles for scripts/entrypoints
• Rectangles for core classes/modules
7. Detailed capturing of essential architecture
- Annotate each arrow with the artifact name (e.g., “TF‑IDF features,” “.pt checkpoint,” “soft labels”)
- Show Hydra config central node feeding into all stages
- Represent scripts/CLI wrappers around core code as thin‐border boxes
- Include a legend for color and shape conventions
By following this structure, the resulting system design diagram will clearly convey the pipeline’s stages, data and model flows, configuration management, and extensibility points—ensuring an accurate, detailed architectural overview.
</explanation>