How imagination-research/lbt works
Project Type and Purpose
• This is a research codebase and experimental toolkit (open-source tool) for exploring “Learning by Teaching” (LbT) in large language models (LLMs). Rather than a traditional full-stack app, it’s a Python library plus CLI scripts that orchestrate data preparation, LbT-based prompting experiments (M1), DPO fine-tuning (M2), and prompt optimization (M3). It integrates with external LLM API endpoints (Azure OpenAI, HuggingFace).
File Structure Analysis
• Top-level directories:
• lbt/ : core Python package implementing LbT abstractions (models, exam maker/prompter/scorer, dataset adapters, utilities).
• scripts/: CLI entry points for dataset preparation and experiments (math and code variants).
• examples/config/: YAML config templates for M1 experiments (math, code).
• examples/datasets/: placeholder for processed data.
• tests/: unit tests for core modules (e.g., math extraction, exam-maker).
• lbt/patch/: patch for LMOps prompt_optimization (M3).
• pyproject.toml: packaging/build.
• pre-commit, pylint config: linting/CI.
README Insights
• Describes three methods:
• M1: search-based generation pipeline that uses “teaching problems” → generate rationales → student exams → LbT scoring and selection.
• M2: use LbT scores to fine-tune the teacher model via Direct Preference Optimization.
• M3: prompt optimization pipeline based on a patched Microsoft LMOps codebase.
• Emphasizes components: exam_maker, exam_prompter, model interfaces, exam_scorer, plus scripts for each stage.
a. Main Components
• CLI Layer (scripts/exam.py, code_exam.py, prepare_datasets.py, search_rationale.py)
• Core Library (lbt package):
– Dataset Adapters (fetching, formatting LeetCode & math).
– ExamMaker, ExamPrompter, ExamScorer classes.
– Model Interfaces (AzureOpenAI, HuggingFace).
– Utilities (logging, registry).
• External Services: Azure OpenAI API, HuggingFace Transformer Inference, LeetCode website/API.
• Third-party Codebases: DPO codebase (M2), patched LMOps/prompt_optimization (M3).
• Config Store: YAML files under examples/config.
• Data Store: Local file system for datasets and experiment outputs.
b. Relationships and Interactions
1. User invokes a CLI script with a YAML config →
2. CLI loads config → uses Dataset Adapters to load teaching/exam datasets.
3. CLI instantiates ExamMaker/Prompter/Scorer and Model Interface(s).
4. Prompter builds prompts (TP + TR or EP) → Model Interface sends to external API (Azure/HF) → receives responses.
5. Scorer parses model output → computes LbT scores → writes outputs.
6. Search_rationale script reads outputs → selects best teaching answer (M1) or aggregates for submission.
7. M2 pipeline: outputs + scores feed into DPO training codebase.
8. M3 pipeline: user applies patch, then calls LMOps main.py with config.py → internal prompt optimization loop with teacher/student API calls.
c. Architectural Patterns & Design Principles
• Plugin/Registry: dynamic instantiation of model, prompter, scorer by type string.
• Config-Driven orchestration: behavior controlled entirely by YAML.
• Separation of Concerns: core LbT logic (lbt/) decoupled from CLI adapters (scripts/) and external code (DPO, LMOps).
• Modular Dataset Adapters: code and math share a common adapter interface.
d. Key Technologies
• Python 3
• Azure OpenAI REST API
• HuggingFace Transformers
• YAML for configuration
• Third-party repos: direct-preference-optimization, Microsoft LMOps prompt_optimization
Detailed Capture of Architectural Elements
• Configuration points (e.g., registry lookup by “model_type”).
• Parallel splits and batch generation in M1 (can show loops or annotations).