# xai-org/grok-1 architecture

> Architecture diagram of the GitHub repository xai-org/grok-1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xai-org/grok-1
- Repository: https://github.com/xai-org/grok-1

## Overview

Grok-1 is an example JAX inference implementation for loading the 314B-parameter model and generating text from a prompt. Its documented workflow configures the model, initializes a sharded runner, loads checkpoint weights, tokenizes the prompt, and samples generated tokens. The model includes transformer attention and MoE components; the README also documents RoPE, activation sharding, and 8-bit quantization. Evidence is limited to the provided excerpts and README, so unsampled internal calls are not expanded.

Main entry points: [`runners.py`](https://github.com/xai-org/grok-1/blob/main/runners.py) (Inference runner), [`run.py`](https://github.com/xai-org/grok-1/blob/main/run.py) (Example setup), [`model.py`](https://github.com/xai-org/grok-1/blob/main/model.py) (Transformer stack), [`checkpoint.py`](https://github.com/xai-org/grok-1/blob/main/checkpoint.py) (Checkpoint restore)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Inference entry"]
  node_run["Example setup<br/>[run.py]"]
end

subgraph group_runtime["Inference runtime"]
  node_runner["Model runner<br/>[runners.py]"]
  node_inference["Inference runner<br/>[runners.py]"]
  node_tokenizer["SentencePiece tokenizer<br/>[runners.py]"]
  node_sampler["Token sampler<br/>[runners.py]"]
end

subgraph group_model["Model computation"]
  node_model_config["Language model<br/>[model.py]"]
  node_transformer["Transformer stack<br/>[model.py]"]
  node_attention["Multi-head attention<br/>[model.py]"]
  node_moe["MoE routing and experts<br/>[model.py]"]
  node_linear["Sharded linear layers<br/>[model.py]"]
  node_norm["RMS normalization<br/>[model.py]"]
  node_embedding["Token embeddings<br/>[model.py]"]
end

subgraph group_weights["Weights"]
  node_checkpoint["Checkpoint restore<br/>[checkpoint.py]"]
  node_store[("Checkpoint tensors")]
end

node_user(("Prompt user"))
node_prompt["Prompt text"]
node_response["Generated text"]

node_user -->|"starts example"| node_run
node_run -->|"configures"| node_runner
node_run -->|"configures"| node_inference
node_run -->|"calls"| node_sampler
node_inference -->|"initializes"| node_runner
node_inference -->|"encodes prompt"| node_tokenizer
node_inference -->|"samples tokens"| node_sampler
node_runner -->|"restores state"| node_checkpoint
node_checkpoint -->|"loads tensors"| node_store
node_runner -->|"initializes model"| node_model_config
node_model_config -->|"embeds tokens"| node_embedding
node_model_config -->|"runs model"| node_transformer
node_transformer -->|"applies attention"| node_attention
node_transformer -->|"applies MoE"| node_moe
node_attention -->|"projects inputs"| node_linear
node_transformer -.->|"normalizes activations"| node_norm
node_prompt -->|"submits prompt"| node_inference
node_sampler -->|"returns generated text"| node_response

click node_run "https://github.com/xai-org/grok-1/blob/main/run.py"
click node_runner "https://github.com/xai-org/grok-1/blob/main/runners.py"
click node_inference "https://github.com/xai-org/grok-1/blob/main/runners.py"
click node_tokenizer "https://github.com/xai-org/grok-1/blob/main/runners.py"
click node_sampler "https://github.com/xai-org/grok-1/blob/main/runners.py"
click node_checkpoint "https://github.com/xai-org/grok-1/blob/main/checkpoint.py"
click node_store "https://github.com/xai-org/grok-1/tree/main/checkpoints"
click node_model_config "https://github.com/xai-org/grok-1/blob/main/model.py"
click node_transformer "https://github.com/xai-org/grok-1/blob/main/model.py"
click node_attention "https://github.com/xai-org/grok-1/blob/main/model.py"
click node_moe "https://github.com/xai-org/grok-1/blob/main/model.py"
click node_linear "https://github.com/xai-org/grok-1/blob/main/model.py"
click node_norm "https://github.com/xai-org/grok-1/blob/main/model.py"
click node_embedding "https://github.com/xai-org/grok-1/blob/main/model.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_run,node_user toneBlue
class node_runner,node_inference,node_tokenizer,node_sampler toneAmber
class node_model_config,node_transformer,node_attention,node_moe,node_linear,node_norm,node_embedding toneMint
class node_checkpoint,node_store toneRose
class node_prompt,node_response toneIndigo
```

## Components

### Inference entry

- [Example setup](https://github.com/xai-org/grok-1/blob/main/run.py): `run.py`

### Inference runtime

- [Model runner](https://github.com/xai-org/grok-1/blob/main/runners.py): `runners.py`
- [Inference runner](https://github.com/xai-org/grok-1/blob/main/runners.py): `runners.py`
- [SentencePiece tokenizer](https://github.com/xai-org/grok-1/blob/main/runners.py): `runners.py`
- [Token sampler](https://github.com/xai-org/grok-1/blob/main/runners.py): `runners.py`

### Model computation

- [Language model](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`
- [Transformer stack](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`
- [Multi-head attention](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`
- [MoE routing and experts](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`
- [Sharded linear layers](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`
- [RMS normalization](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`
- [Token embeddings](https://github.com/xai-org/grok-1/blob/main/model.py): `model.py`

### Weights

- [Checkpoint restore](https://github.com/xai-org/grok-1/blob/main/checkpoint.py): `checkpoint.py`
- [Checkpoint tensors](https://github.com/xai-org/grok-1/tree/main/checkpoints): `checkpoints`

### Other components

- **Prompt user**
- **Prompt text**
- **Generated text**

## Connections

- Prompt user → Example setup: starts example (evidence: `README.md`)
- Example setup → Model runner: configures (evidence: `run.py`)
- Example setup → Inference runner: configures (evidence: `run.py`)
- Example setup → Token sampler: calls (evidence: `run.py`)
- Inference runner → Model runner: initializes (evidence: `runners.py`)
- Inference runner → SentencePiece tokenizer: encodes prompt (evidence: `runners.py`)
- Inference runner → Token sampler: samples tokens (evidence: `runners.py`)
- Model runner → Checkpoint restore: restores state (evidence: `runners.py`)
- Checkpoint restore → Checkpoint tensors: loads tensors (evidence: `checkpoint.py`)
- Model runner → Language model: initializes model (evidence: `runners.py`)
- Language model → Token embeddings: embeds tokens (evidence: `model.py`)
- Language model → Transformer stack: runs model (evidence: `model.py`)
- Transformer stack → Multi-head attention: applies attention (evidence: `model.py`)
- Transformer stack → MoE routing and experts: applies MoE (evidence: `README.md`)
- Multi-head attention → Sharded linear layers: projects inputs (evidence: `model.py`)
- Transformer stack → RMS normalization: normalizes activations (evidence: `model.py`)
- Prompt text → Inference runner: submits prompt (evidence: `runners.py`)
- Token sampler → Generated text: returns generated text
