How yc1999/llm2 works
Key Components to Include:
Main System Architecture
a. System 1 (LLM Component)
b. System 2 (Verifier Component)
Data Flow Components
• Input Processing Layer
• How the input data flows from datasets (./dataset/*.jsonl)
• Data preprocessing utilities (data_utils.py)
• Training Pipeline
• Verifier Training Module (train_verifier.py, verifier_trainer.py)
• Model Utils (model_utils.py)
• Configuration Management (configs.py)
• Inference Pipeline
• Inference Engine (inference.py)
• Verification Search (verify_search.py)
• Evaluation Module (evaluation.py)
Supporting Components
• Benchmarking System (benchmarks/)
• Configuration Management (config_files/)
• Utility Services (utils_stopping.py, modeling_attn_mask_utils.py)
Use a top-down approach with the following layers
• [Input Layer]
• ↓
• [Processing Layer]
• ↓
• [Core System Layer (System 1 & 2)]
• ↓
• [Output Layer]
Specific connection points
• How the LLM generates candidates
• Verifier's feedback loop
• Demonstrate the token quality exploration strategy
• Pairwise comparison mechanism
Technical Details
• Data formats (JSONL structure)
• Evaluation paths (GSM8k benchmark)
Specific Guidelines:
Bidirectional nature of the system
• LLM's candidate generation
• Verifier's feedback mechanism
• Token quality exploration
Clear separation between
• Training pipeline
• Inference pipeline
• Evaluation system
Configuration management
• How different config files affect system behavior
• Accelerator configurations (Zero3, etc.)
Data flow
• Input processing
• Token sampling
• Verification process
• Final output generation