How nivitra/seal works
Project type and purpose
• SEAL is a research framework for training large language models (LLMs) to generate “self-edits” via reinforcement learning and fine-tuning.
• It has two major domains, each in its own top-level folder:
– few-shot: adapting to new tasks via few-shot examples
– knowledge-incorporation: updating the model with new factual knowledge
File structure
• Top-level:
– few-shot/
· arclib/: core library for ARC self-editing (augmenters, representers, voting, update_model, etc.)
· inference/: multiple engine implementations (HF, vLLM, no-LoRA, OpenAI API) + preprocessing
· data/: raw ARC challenges & solutions + filtering scripts
· self-edit.py, eval-self-edits.py, predict_*.py, launch.sh
– knowledge-incorporation/
· data/: SQuAD train/val + synthetic data subfolders
· scripts/: bash orchestrations (train_SFT, query_server, CPT, TTT)
· src/: Python modules for dataset building, continual self-edit training, query servers, CPT routines
· results/: JSON outputs from each evaluation or run
• Common across both: requirements.txt, .env for API keys
README for additional insights
• Confirms that each domain includes its own data, code, experiments, and evaluation.
• Dependencies: OpenAI API, HuggingFace, Deepspeed, SLURM for job scheduling.
Main components & interactions
a. Data Layer
– Raw challenge/problem datasets (ARC, SQuAD)
– Filtering and synthetic data generation scripts
b. Core Library (arclib in few-shot; src/ in knowledge-incorporation)
– Functions to produce self-edits, update models, evaluate edits
– Utilities (prompts, caching, LM API wrappers)
c. Inference Engines
– Abstract interface + multiple backends (OpenAI, HF, vLLM, LoRA variants)
– Preprocessing module to convert inputs into engine calls
d. Training/Update Loop
– For knowledge-incorporation: SFT dataset builder, Deepspeed config, continual self-edits RL loop
– For few-shot: iterative self-editing pipeline with arc.py and update_model.py
e. Evaluation & Results
– Scripts to run evaluations (eval-self-edits-baseline, predict_custom, average_results)
– Results folder with JSON summaries per run
Architectural patterns & design principles
• Modular layering: Data → Core library → Inference → Training → Evaluation
• Strategy/Factory pattern for engine backends—swapable inference implementations
• Script-orchestration: Bash entrypoints call Python modules in a defined sequence
• Separation of concerns: each domain has its own data, code, and outputs
Technologies, frameworks, libraries
• Python modules: arclib, utils, src/EM, src/continual, src/query
• APIs: OpenAI, HuggingFace, vLLM
• Training: Deepspeed (stage3), SLURM scripts
• Data formats: JSON, JSONL
– Blue: Data & Preprocessing
– Green: Core library & training logic
– Orange: Inference engines
– Purple: Evaluation & results