How princeton-nlp/less works
Main Components
• Data Input & Preparation:
– Represent raw and processed datasets (e.g., training datasets like Flan v2, Dolly, COT, Open Assistant; evaluation datasets such as MMLU, TydiQA, BBH).
• Warmup Training Module:
– Use the scripts under less/scripts/train (specifically, warmup_lora_train.sh) in conjunction with the “run” directory examples. This stage initializes a LoRA-based training process on a small percentage (e.g., 5%) of the data.
• Gradient Datastore Builder:
– Highlight the gradient computation stage. Modules in the less/data_selection and less/scripts/get_info directories as well as scripts in “run/first_order_checking” form this component.
– This part builds the “gradient datastore” by collecting, storing, and processing gradients from the warmup model at selected checkpoints.
• Data Selection Module:
– Illustrated by components in less/data_selection and the matching scripts (e.g., matching.sh & write_selected_data). This module computes influence scores from gradients and selects the top-k influential training points.
• Final Training Module:
• Evaluation Module:
– Reflect the evaluation scripts found in the evaluation directory (which includes multiple subcomponents like alpaca_farm, bbh, codex_humaneval, gsm, toxicgen, truthfulqa, tydiqa).
– Emphasize that different evaluation scripts target different standardized evaluation datasets.
Relationships and Data Flow
• The output from the Warmup Training then becomes the source for the Gradient Datastore Builder, which calculates gradients across various checkpoints.
• The gradients are subsequently fed into the Data Selection Module that computes the influence scores (note the directional flow: gradients → influence score computation → selection outcomes).
• The selected top-k influential training samples then form the input for the Final Training Module that uses LoRA fine-tuning (or even full-parameter finetuning).
• Simultaneously, evaluation workflows tap outputs from both the warmup and fine-tuned models to assess performance across multiple tasks.
• Optionally, include a feedback loop from evaluation indicating further data selection adjustments or retraining loops for iterative refinement.
Architectural Patterns & Design Principles
• Modular Design: The project is organized into logically separated modules (data preparation, training, gradient extraction, selection, evaluation).
• Pipeline Architecture: Emphasize a sequential data flow where each stage produces outputs that are mandatory inputs for subsequent stages.
• Experimentation Focus: Many components (e.g., various evaluation scripts) reflect a research experiment design where modularity and reproducibility are key.
• Integration with External Tools: The dependency on PyTorch and shell scripts shows supported integration between Python modules and command-line utilities.
– Data Inputs: External datasets, training & validation files.
– Training Modules: “Warmup Training” (LoRA training), “Final Training”
– Gradient Collection: “Gradient Datastore Builder” utilizing scripts (e.g., get_train_lora_grads.sh, get_eval_lora_grads.sh)
– Data Matching & Selection: depicted with scripts like matching.sh and write_selected_data.py.
• Optionally, annotate any external tooling requirements (e.g., PyTorch version, installation steps from requirement.txt).
Final Recommendations for the Engineer
• Module is clearly annotated with its corresponding folder or script reference as indicated by the file tree.