# eleutherai/features-across-time architecture

> Architecture diagram of the GitHub repository eleutherai/features-across-time, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eleutherai/features-across-time
- Repository: https://github.com/eleutherai/features-across-time

## Overview

features-across-time is an experiment codebase for tracking learned features and model behavior over training. Its documented Pythia workflow builds n-gram data, evaluates models at training steps, and plots the resulting measurements. The tree also supports a distinct vision-experiment branch: preparing real and synthetic datasets, training vision models, and plotting results. Sampled source confirms key evaluation, data-generation, and plotting responsibilities; some links in the vision branch are based on module roles evident from names and README-level project context, not inspected call sites.

Main entry points: [`scripts/inference/ngrams.py`](https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/ngrams.py) (Pythia n-gram eval), [`scripts/inference/train_vision.py`](https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/train_vision.py) (Vision model training), [`scripts/preprocess/build_ngram_data.py`](https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/build_ngram_data.py) (N-gram data build), [`scripts/preprocess/build_vision_data.py`](https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/build_vision_data.py) (Vision data build)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_text["Text experiments"]
  node_text_data["N-gram data build"]
  node_text_eval["Pythia n-gram eval<br/>[ngrams.py]"]
  node_token_eval["Token-gram eval<br/>[tokengrams.py]"]
  node_text_artifacts["Text experiment artifacts"]
end

subgraph group_vision["Vision experiments"]
  node_vision_data["Vision data build"]
  node_third_order["Third-order synthesis"]
  node_vision_train["Vision model training<br/>[train_vision.py]"]
  node_vision_artifacts["Vision experiment artifacts"]
end

subgraph group_plots["Analysis plots"]
  node_single_plot["Pythia suite plots"]
  node_context_plot["In-context plots<br/>[plot_in_context.py]"]
  node_vision_plot["Vision plots<br/>[plot_vision.py]"]
  node_plot_artifacts["Figures"]
end

subgraph group_internal["Shared analysis"]
  node_divergences["Divergence metrics<br/>[divergences.py]"]
  node_distribution["Conrad sampler"]
end

node_researcher(("Researcher"))
node_model_source["Pythia checkpoints"]
node_dataset_source["Text datasets"]
node_vision_source["Vision datasets"]

node_researcher -->|"runs build"| node_text_data
node_researcher -->|"runs evaluation"| node_text_eval
node_text_data -.->|"loads tokens"| node_dataset_source
node_text_data -->|"writes data"| node_text_artifacts
node_text_eval -->|"reads data; writes results"| node_text_artifacts
node_text_eval -.->|"loads checkpoints"| node_model_source
node_text_eval -->|"computes metrics"| node_divergences
node_token_eval -.->|"reads n-gram artifacts"| node_text_artifacts
node_researcher -->|"runs preparation"| node_vision_data
node_vision_data -.->|"loads datasets"| node_vision_source
node_vision_data -->|"writes datasets"| node_vision_artifacts
node_third_order -.->|"loads dataset"| node_vision_source
node_third_order -->|"writes synthetic data"| node_vision_artifacts
node_researcher -->|"runs training"| node_vision_train
node_vision_train -.->|"loads datasets"| node_vision_source
node_vision_train -->|"reads data; saves results"| node_vision_artifacts
node_vision_train -.->|"samples distribution"| node_distribution
node_researcher -->|"runs plotting"| node_single_plot
node_single_plot -->|"reads results"| node_text_artifacts
node_context_plot -->|"reads results"| node_text_artifacts
node_single_plot -->|"writes figures"| node_plot_artifacts
node_context_plot -->|"writes figures"| node_plot_artifacts
node_researcher -->|"runs plotting"| node_vision_plot
node_vision_plot -->|"reads results"| node_vision_artifacts
node_vision_plot -->|"writes figures"| node_plot_artifacts

click node_text_data "https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/build_ngram_data.py"
click node_text_eval "https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/ngrams.py"
click node_token_eval "https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/tokengrams.py"
click node_vision_data "https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/build_vision_data.py"
click node_third_order "https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/third_order_approx.py"
click node_vision_train "https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/train_vision.py"
click node_single_plot "https://github.com/eleutherai/features-across-time/blob/main/scripts/plot/plot_single_rows.py"
click node_context_plot "https://github.com/eleutherai/features-across-time/blob/main/scripts/plot/plot_in_context.py"
click node_vision_plot "https://github.com/eleutherai/features-across-time/blob/main/scripts/plot/plot_vision.py"
click node_divergences "https://github.com/eleutherai/features-across-time/blob/main/scripts/script_utils/divergences.py"
click node_distribution "https://github.com/eleutherai/features-across-time/blob/main/scripts/script_utils/conrad_distribution.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_text_data,node_text_eval,node_token_eval,node_text_artifacts toneBlue
class node_vision_data,node_third_order,node_vision_train,node_vision_artifacts toneAmber
class node_single_plot,node_context_plot,node_vision_plot,node_plot_artifacts toneMint
class node_divergences,node_distribution toneRose
class node_researcher,node_model_source,node_dataset_source,node_vision_source toneIndigo
```

## Components

### Text experiments

- [N-gram data build](https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/build_ngram_data.py): `scripts/preprocess/build_ngram_data.py`
- [Pythia n-gram eval](https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/ngrams.py): `scripts/inference/ngrams.py`
- [Token-gram eval](https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/tokengrams.py): `scripts/inference/tokengrams.py`
- **Text experiment artifacts**

### Vision experiments

- [Vision data build](https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/build_vision_data.py): `scripts/preprocess/build_vision_data.py`
- [Third-order synthesis](https://github.com/eleutherai/features-across-time/blob/main/scripts/preprocess/third_order_approx.py): `scripts/preprocess/third_order_approx.py`
- [Vision model training](https://github.com/eleutherai/features-across-time/blob/main/scripts/inference/train_vision.py): `scripts/inference/train_vision.py`
- **Vision experiment artifacts**

### Analysis plots

- [Pythia suite plots](https://github.com/eleutherai/features-across-time/blob/main/scripts/plot/plot_single_rows.py): `scripts/plot/plot_single_rows.py`
- [In-context plots](https://github.com/eleutherai/features-across-time/blob/main/scripts/plot/plot_in_context.py): `scripts/plot/plot_in_context.py`
- [Vision plots](https://github.com/eleutherai/features-across-time/blob/main/scripts/plot/plot_vision.py): `scripts/plot/plot_vision.py`
- **Figures**

### Shared analysis

- [Divergence metrics](https://github.com/eleutherai/features-across-time/blob/main/scripts/script_utils/divergences.py): `scripts/script_utils/divergences.py`
- [Conrad sampler](https://github.com/eleutherai/features-across-time/blob/main/scripts/script_utils/conrad_distribution.py): `scripts/script_utils/conrad_distribution.py`

### Other components

- **Researcher**
- **Pythia checkpoints**
- **Text datasets**
- **Vision datasets**

## Connections

- Researcher → N-gram data build: runs build
- Researcher → Pythia n-gram eval: runs evaluation
- N-gram data build → Text datasets: loads tokens
- N-gram data build → Text experiment artifacts: writes data
- Pythia n-gram eval → Text experiment artifacts: reads data; writes results
- Pythia n-gram eval → Pythia checkpoints: loads checkpoints
- Pythia n-gram eval → Divergence metrics: computes metrics
- Token-gram eval → Text experiment artifacts: reads n-gram artifacts
- Researcher → Vision data build: runs preparation
- Vision data build → Vision datasets: loads datasets
- Vision data build → Vision experiment artifacts: writes datasets
- Third-order synthesis → Vision datasets: loads dataset
- Third-order synthesis → Vision experiment artifacts: writes synthetic data
- Researcher → Vision model training: runs training
- Vision model training → Vision datasets: loads datasets
- Vision model training → Vision experiment artifacts: reads data; saves results
- Vision model training → Conrad sampler: samples distribution
- Researcher → Pythia suite plots: runs plotting
- Pythia suite plots → Text experiment artifacts: reads results
- In-context plots → Text experiment artifacts: reads results
- Pythia suite plots → Figures: writes figures
- In-context plots → Figures: writes figures
- Researcher → Vision plots: runs plotting
- Vision plots → Vision experiment artifacts: reads results
- Vision plots → Figures: writes figures