# ezelikman/quiet-star architecture

> Architecture diagram of the GitHub repository ezelikman/quiet-star, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ezelikman/quiet-star
- Repository: https://github.com/ezelikman/quiet-star

## Overview

##### Project type and purpose
- This is an open-source research codebase that patches the base Mistral implementation in Huggingface Transformers.
- Its goal is to enable “Quiet-STaR,” a self-reflective thought-token mechanism, by supplying new configuration and modeling files and then training/evaluating the patched model via standard Transformers’ Trainer.

##### File structure
- Top-level files:
– configuration_mistral.py – custom Config class extending Huggingface’s MistralConfig
– modeling_mistral.py – patched model implementation extending MistralForCausalLM
– quiet-star-train.py – training script that instantiates Trainer, Dataset, model, etc.
– zero-shotcot-eval.py – evaluation script for zero-shot chain-of-thought
– eval_helpers.py – utility functions for data processing and metrics
– README.md, LICENSE

- Key patterns:
– Plugin/monkey-patch style: two new modules override the HF core under transformers/models/mistral
– Standard Trainer pipeline: data → Trainer → patched model → outputs

##### Extract insights from README
- Emphasizes reproducibility on Transformers v4.37.0.dev0
- Warns about masking start/end thought tokens at inference
- Publishes a pre-trained model on the Huggingface Hub

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Subgraph: Core HF Components
    subgraph "Huggingface Transformers Core" 
        HFCore["Huggingface Transformers Core<br/>(v4.37.0.dev0)"]:::core
    end

    %% Subgraph: Custom Extensions
    subgraph "Custom Extensions" 
        Patched_Config["Patched Configuration"]:::custom
        Patched_Model["Patched Model Implementation"]:::custom
    end

    %% Subgraph: Data & Storage
    subgraph "Data & Storage"
        DataPipeline["Data Pipeline"]:::core
        Datasets[(Dataset Store)]:::db
        Checkpoints[(Checkpoints)]:::db
    end

    %% Subgraph: Training & Evaluation Scripts
    subgraph "Training & Evaluation"
        Trainer["Training Orchestrator"]:::custom
        EvalScript["Zero-Shot COT Eval"]:::util
        EvalHelpers["Utility Functions"]:::util
    end

    %% External Services & Docs
    Hub((Huggingface Hub)):::external
    Docs["README.md & LICENSE"]:::docs

    %% Connections: Training Flow
    Trainer -->|load_config| Patched_Config
    Trainer -->|load_model_impl| Patched_Model
    Trainer -->|load_data| DataPipeline
    DataPipeline -->|tokenize_and_batch| Trainer
    Trainer -->|forward_pass| Patched_Model
    Patched_Model -->|outputs| Checkpoints
    Trainer -->|push_to_hub| Hub

    %% Connections: Evaluation Flow
    EvalScript -->|load_checkpoint| Checkpoints
    EvalScript -->|preprocess| EvalHelpers
    EvalScript -->|infer_via_HFCore_and_PatchedModel| Patched_Model
    EvalScript -->|compute_metrics| EvalHelpers
    EvalScript -->|push_results| Hub

    %% Core Dependencies
    Patched_Config -->|inherits_from| HFCore
    Patched_Model -->|extends_MistralForCausalLM| HFCore

    %% Documentation
    Docs -.->|reproducibility_notes| Trainer
    Docs -.->|usage_reference| EvalScript

    %% Click Events
    click Patched_Config "https://github.com/ezelikman/quiet-star/blob/main/configuration_mistral.py"
    click Patched_Model "https://github.com/ezelikman/quiet-star/blob/main/modeling_mistral.py"
    click Trainer "https://github.com/ezelikman/quiet-star/blob/main/quiet-star-train.py"
    click EvalScript "https://github.com/ezelikman/quiet-star/blob/main/zero-shotcot-eval.py"
    click EvalHelpers "https://github.com/ezelikman/quiet-star/blob/main/eval_helpers.py"
    click Docs "https://github.com/ezelikman/quiet-star/blob/main/README.md"
    click Docs "https://github.com/ezelikman/quiet-star/tree/main/LICENSE"

    %% Styles
    classDef core fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef custom fill:#add8e6,stroke:#333,stroke-width:1px
    classDef util fill:#90ee90,stroke:#333,stroke-width:1px
    classDef external fill:#ffa500,stroke:#333,stroke-width:1px
    classDef docs fill:#f5f5dc,stroke:#333,stroke-width:1px
    classDef db fill:#ffffff,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
```
