# nivitra/seal architecture

> Architecture diagram of the GitHub repository nivitra/seal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nivitra/seal
- Repository: https://github.com/nivitra/seal

## Overview

##### Project type and purpose
- SEAL is a research framework for training large language models (LLMs) to generate “self-edits” via reinforcement learning and fine-tuning.
- It has two major domains, each in its own top-level folder:
– few-shot: adapting to new tasks via few-shot examples
– knowledge-incorporation: updating the model with new factual knowledge

##### File structure
- Top-level:
– few-shot/
· arclib/: core library for ARC self-editing (augmenters, representers, voting, update_model, etc.)
· inference/: multiple engine implementations (HF, vLLM, no-LoRA, OpenAI API) + preprocessing
· data/: raw ARC challenges & solutions + filtering scripts
· self-edit.py, eval-self-edits.py, predict_*.py, launch.sh
– knowledge-incorporation/
· data/: SQuAD train/val + synthetic data subfolders
· scripts/: bash orchestrations (train_SFT, query_server, CPT, TTT)
· src/: Python modules for dataset building, continual self-edit training, query servers, CPT routines
· results/: JSON outputs from each evaluation or run
- Common across both: requirements.txt, .env for API keys

##### README for additional insights
- Confirms that each domain includes its own data, code, experiments, and evaluation.
- Dependencies: OpenAI API, HuggingFace, Deepspeed, SLURM for job scheduling.

##### Main components & interactions
a. Data Layer
– Raw challenge/problem datasets (ARC, SQuAD)
– Filtering and synthetic data generation scripts
b. Core Library (arclib in few-shot; src/ in knowledge-incorporation)
– Functions to produce self-edits, update models, evaluate edits
– Utilities (prompts, caching, LM API wrappers)
c. Inference Engines
– Abstract interface + multiple backends (OpenAI, HF, vLLM, LoRA variants)
– Preprocessing module to convert inputs into engine calls
d. Training/Update Loop
– For knowledge-incorporation: SFT dataset builder, Deepspeed config, continual self-edits RL loop
– For few-shot: iterative self-editing pipeline with arc.py and update_model.py
e. Evaluation & Results
– Scripts to run evaluations (eval-self-edits-baseline, predict_custom, average_results)
– Results folder with JSON summaries per run

##### Architectural patterns & design principles
- Modular layering: Data → Core library → Inference → Training → Evaluation
- Strategy/Factory pattern for engine backends—swapable inference implementations
- Script-orchestration: Bash entrypoints call Python modules in a defined sequence
- Separation of concerns: each domain has its own data, code, and outputs

##### Technologies, frameworks, libraries
- Python modules: arclib, utils, src/EM, src/continual, src/query
- APIs: OpenAI, HuggingFace, vLLM
- Training: Deepspeed (stage3), SLURM scripts
- Data formats: JSON, JSONL

– Blue: Data & Preprocessing
– Green: Core library & training logic
– Orange: Inference engines
– Purple: Evaluation & results

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Few-Shot Pipeline
    subgraph "Few-Shot Pipeline"
        direction TB
        FSData["Few-Shot Data"]:::data
        click FSData "https://github.com/nivitra/seal/tree/main/few-shot/data/"
        FSPre["Preprocessing"]:::data
        click FSPre "https://github.com/nivitra/seal/blob/main/few-shot/inference/preprocess.py"
        FSUtils["Utilities"]:::utils
        click FSUtils "https://github.com/nivitra/seal/tree/main/few-shot/utils/"
        FSCore["Core Library (arclib)"]:::core
        click FSCore "https://github.com/nivitra/seal/tree/main/few-shot/arclib/"
        FSInfIface["Inference Interface"]:::inference
        click FSInfIface "https://github.com/nivitra/seal/blob/main/few-shot/inference/engine.py"
        FSInfHF["HF Backend"]:::inference
        click FSInfHF "https://github.com/nivitra/seal/blob/main/few-shot/inference/engine_hf.py"
        FSInfVLLM["vLLM Backend"]:::inference
        click FSInfVLLM "https://github.com/nivitra/seal/blob/main/few-shot/inference/engine_vllm.py"
        FSInfNoLORA["No-LoRA Backends"]:::inference
        click FSInfNoLORA "https://github.com/nivitra/seal/blob/main/few-shot/inference/engine_no_lora.py"
        click FSInfNoLORA "https://github.com/nivitra/seal/blob/main/few-shot/inference/engine_no_lora_vllm.py"
        FSSelfEdit["Self-Edit Pipeline"]:::scripts
        click FSSelfEdit "https://github.com/nivitra/seal/blob/main/few-shot/self-edit.py"
        FSEvalBase["Baseline Evaluation"]:::evaluation
        click FSEvalBase "https://github.com/nivitra/seal/blob/main/few-shot/eval-self-edits-baseline.py"
        FSEvalEdit["Edit Evaluation"]:::evaluation
        click FSEvalEdit "https://github.com/nivitra/seal/blob/main/few-shot/eval-self-edits.py"
        FSPredict["Prediction Scripts"]:::evaluation
        click FSPredict "https://github.com/nivitra/seal/blob/main/few-shot/predict_baseline.py"
        click FSPredict "https://github.com/nivitra/seal/blob/main/few-shot/predict_custom.py"
        FSLaunch["Launch Script"]:::scripts
        click FSLaunch "https://github.com/nivitra/seal/blob/main/few-shot/launch.sh"
        FSResult["Results Store"]:::evaluation
        click FSResult "https://github.com/nivitra/seal/blob/main/few-shot/final_results.json"
    end

    %% Knowledge-Incorporation Pipeline
    subgraph "Knowledge-Incorporation Pipeline"
        direction TB
        KIData["KI Data (SQuAD & Synthetic)"]:::data
        click KIData "https://github.com/nivitra/seal/tree/main/knowledge-incorporation/data/"
        KITxtGen["Synthetic Data Gen"]:::data
        click KITxtGen "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/data_generation/make_squad_data.py"
        click KITxtGen "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/data_generation/make_squad_data_openai.py"
        KIUtils["Shared Utilities"]:::utils
        click KIUtils "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/utils.py"
        %% SFT
        KISFTBuilder["SFT Dataset Builder"]:::core
        click KISFTBuilder "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/EM/build_SFT_dataset.py"
        KISFTTrain["SFT Training"]:::core
        click KISFTTrain "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/EM/train_SFT.py"
        KIDSConfig["Deepspeed Config"]:::core
        click KIDSConfig "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/EM/config/deepspeed_stage3.json"
        %% Continual Loop
        KIContLoop["Continual Self-Edits Loop"]:::core
        click KIContLoop "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/continual/continual_self_edits.py"
        KIAvgRes["Average Results"]:::evaluation
        click KIAvgRes "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/continual/average_results.py"
        %% Query & CPT
        KIQuerySrv["Query Server"]:::core
        click KIQuerySrv "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/query/query_server.py"
        KICPT["CPT Routines"]:::core
        click KICPT "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/query/CPT.py"
        %% TTT Server
        KITTTSrv["TTT Server"]:::core
        click KITTTSrv "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/src/inner/TTT_server.py"
        %% Scripts
        KIScriptSFT["Train SFT Script"]:::scripts
        click KIScriptSFT "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/train_SFT.sh"
        KIScriptData["Make SQuAD Data Scripts"]:::scripts
        click KIScriptData "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/make_squad_data.sh"
        click KIScriptData "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/make_squad_data_openai.sh"
        KIScriptCont["Continual Edits Script"]:::scripts
        click KIScriptCont "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/continual_self_edits.sh"
        KIScriptQuery["Query Server Script"]:::scripts
        click KIScriptQuery "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/query_server.sh"
        KIScriptCPT["CPT Script"]:::scripts
        click KIScriptCPT "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/CPT.sh"
        KIScriptTTT["TTT Server Script"]:::scripts
        click KIScriptTTT "https://github.com/nivitra/seal/blob/main/knowledge-incorporation/scripts/TTT_server.sh"
        %% Results
        KIResultsCont["Continual Results"]:::evaluation
        click KIResultsCont "https://github.com/nivitra/seal/tree/main/knowledge-incorporation/results/continual_self_edits/"
        KIResultsCPT["CPT Results"]:::evaluation
        click KIResultsCPT "https://github.com/nivitra/seal/tree/main/knowledge-incorporation/results/cpt/"
        KIResultsQuery["Query Server Results"]:::evaluation
        click KIResultsQuery "https://github.com/nivitra/seal/tree/main/knowledge-incorporation/results/query_server/"
    end

    %% External Dependencies
    OpenAI[(OpenAI API)]:::external
    HFHub[(HuggingFace Models)]:::external
    vLLMExt[(vLLM Service)]:::external
    Deepspeed[(Deepspeed)]:::external
    SLURM[(SLURM Scheduler)]:::external

    %% Few-Shot Flow
    FSData --> FSPre --> FSUtils --> FSCore --> FSInfIface --> FSInfHF
    FSInfIface --> FSInfVLLM
    FSInfIface --> FSInfNoLORA
    FSInfHF -->|calls| HFHub
    FSInfVLLM -->|calls| vLLMExt
    FSInfNoLORA -->|calls| HFHub
    FSCore --> FSSelfEdit --> FSEvalBase --> FSResult
    FSSelfEdit --> FSCore
    FSPredict --> FSResult
    FSLaunch --> FSSelfEdit
    FSLaunch --> FSPredict

    %% Knowledge-Incorporation Flow
    KIData --> KITxtGen --> KIUtils
    KIUtils --> KISFTBuilder --> KISFTTrain --> KIDSConfig
    KISFTTrain --> Deepspeed
    KISFTTrain --> SLURM
    KISFTTrain --> KIContLoop --> KIAvgRes --> KIResultsCont
    KIContLoop --> SLURM
    KIContLoop --> Deepspeed
    KIUtils --> KIQuerySrv --> KICPT --> KIResultsCPT
    KIQuerySrv --> OpenAI
    KICPT --> OpenAI
    KIUtils --> KITTTSrv --> OpenAI
    KITTTSrv --> SLURM
    KIScriptSFT --> KISFTTrain
    KIScriptData --> KITxtGen
    KIScriptCont --> KIContLoop
    KIScriptQuery --> KIQuerySrv
    KIScriptCPT --> KICPT
    KIScriptTTT --> KITTTSrv

    %% Styles
    classDef data fill:#BBDEFB,stroke:#2196F3
    classDef core fill:#C8E6C9,stroke:#388E3C
    classDef inference fill:#FFE0B2,stroke:#FB8C00
    classDef evaluation fill:#E1BEE7,stroke:#8E24AA
    classDef external stroke-dasharray:5 5,stroke:#999
    classDef utils fill:#CFD8DC,stroke:#607D8B
    classDef scripts fill:#FFF9C4,stroke:#FBC02D
```
