How xai-org/grok-1 works
Grok-1 is an example JAX inference implementation for loading the 314B-parameter model and generating text from a prompt. Its documented workflow configures the model, initializes a sharded runner, loads checkpoint weights, tokenizes the prompt, and samples generated tokens. The model includes transformer attention and MoE components; the README also documents RoPE, activation sharding, and 8-bit quantization. Evidence is limited to the provided excerpts and README, so unsampled internal calls are not expanded.
Main entry points: runners.py Inference runner · run.py Example setup · model.py Transformer stack · checkpoint.py Checkpoint restore
Components 17
Inference entry
- Example setup
run.py
Inference runtime
- Model runner
runners.py - Inference runner
runners.py - SentencePiece tokenizer
runners.py - Token sampler
runners.py
Model computation
- Language model
model.py - Transformer stack
model.py - Multi-head attention
model.py - MoE routing and experts
model.py - Sharded linear layers
model.py - RMS normalization
model.py - Token embeddings
model.py
Weights
- Checkpoint restore
checkpoint.py - Checkpoint tensors
checkpoints
Other components
- Prompt user
- Prompt text
- Generated text
Connections 18
- Prompt user → Example setup: starts example
README.md - Example setup → Model runner: configures
run.py - Example setup → Inference runner: configures
run.py - Example setup → Token sampler: calls
run.py - Inference runner → Model runner: initializes
runners.py - Inference runner → SentencePiece tokenizer: encodes prompt
runners.py - Inference runner → Token sampler: samples tokens
runners.py - Model runner → Checkpoint restore: restores state
runners.py - Checkpoint restore → Checkpoint tensors: loads tensors
checkpoint.py - Model runner → Language model: initializes model
runners.py - Language model → Token embeddings: embeds tokens
model.py - Language model → Transformer stack: runs model
model.py - Transformer stack → Multi-head attention: applies attention
model.py - Transformer stack → MoE routing and experts: applies MoE
README.md - Multi-head attention → Sharded linear layers: projects inputs
model.py - Transformer stack → RMS normalization: normalizes activations
model.py - Prompt text → Inference runner: submits prompt
runners.py - Token sampler → Generated text: returns generated text