How eleutherai/features-across-time works
features-across-time is an experiment codebase for tracking learned features and model behavior over training. Its documented Pythia workflow builds n-gram data, evaluates models at training steps, and plots the resulting measurements. The tree also supports a distinct vision-experiment branch: preparing real and synthetic datasets, training vision models, and plotting results. Sampled source confirms key evaluation, data-generation, and plotting responsibilities; some links in the vision branch are based on module roles evident from names and README-level project context, not inspected call sites.
Main entry points: ngrams.py Pythia n-gram eval · train_vision.py Vision model training · build_ngram_data.py N-gram data build · build_vision_data.py Vision data build
Components 18
Text experiments
- N-gram data build
scripts/preprocess/build_ngram_data.py - Pythia n-gram eval
scripts/inference/ngrams.py - Token-gram eval
scripts/inference/tokengrams.py - Text experiment artifacts
Vision experiments
- Vision data build
scripts/preprocess/build_vision_data.py - Third-order synthesis
scripts/preprocess/third_order_approx.py - Vision model training
scripts/inference/train_vision.py - Vision experiment artifacts
Analysis plots
- Pythia suite plots
scripts/plot/plot_single_rows.py - In-context plots
scripts/plot/plot_in_context.py - Vision plots
scripts/plot/plot_vision.py - Figures
Shared analysis
- Divergence metrics
scripts/script_utils/divergences.py - Conrad sampler
scripts/script_utils/conrad_distribution.py
Other components
- Researcher
- Pythia checkpoints
- Text datasets
- Vision datasets
Connections 25
- Researcher → N-gram data build: runs build
- Researcher → Pythia n-gram eval: runs evaluation
- N-gram data build → Text datasets: loads tokens
- N-gram data build → Text experiment artifacts: writes data
- Pythia n-gram eval → Text experiment artifacts: reads data; writes results
- Pythia n-gram eval → Pythia checkpoints: loads checkpoints
- Pythia n-gram eval → Divergence metrics: computes metrics
- Token-gram eval → Text experiment artifacts: reads n-gram artifacts
- Researcher → Vision data build: runs preparation
- Vision data build → Vision datasets: loads datasets
- Vision data build → Vision experiment artifacts: writes datasets
- Third-order synthesis → Vision datasets: loads dataset
- Third-order synthesis → Vision experiment artifacts: writes synthetic data
- Researcher → Vision model training: runs training
- Vision model training → Vision datasets: loads datasets
- Vision model training → Vision experiment artifacts: reads data; saves results
- Vision model training → Conrad sampler: samples distribution
- Researcher → Pythia suite plots: runs plotting
- Pythia suite plots → Text experiment artifacts: reads results
- In-context plots → Text experiment artifacts: reads results
- Pythia suite plots → Figures: writes figures
- In-context plots → Figures: writes figures
- Researcher → Vision plots: runs plotting
- Vision plots → Vision experiment artifacts: reads results
- Vision plots → Figures: writes figures