# meta-llama/llama3 architecture

> Architecture diagram of the GitHub repository meta-llama/llama3, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/meta-llama/llama3
- Repository: https://github.com/meta-llama/llama3

## Overview

This deprecated repository is minimal native Llama 3 inference code. Users provide model checkpoints and a tokenizer file, then run either chat or pretrained text completion. The examples build a generator and invoke the corresponding completion method; generation tokenizes or formats prompts, runs the Transformer, and decodes results. The graph preserves both documented inference modes and the tokenizer, chat-formatting, and model layers. The excerpts do not show every detail of model loading, so those relationships rely on the README or are omitted.

Main entry points: [`llama/generation.py`](https://github.com/meta-llama/llama3/blob/main/llama/generation.py) (Llama generation), [`example_chat_completion.py`](https://github.com/meta-llama/llama3/blob/main/example_chat_completion.py) (Chat example), [`example_text_completion.py`](https://github.com/meta-llama/llama3/blob/main/example_text_completion.py) (Text example), [`llama/tokenizer.py`](https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py) (Tokenizer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_examples["Inference examples"]
  node_chat_example["Chat example"]
  node_text_example["Text example"]
end

subgraph group_runtime["Inference runtime"]
  node_generator["Llama generation<br/>[generation.py]"]
end

subgraph group_model["Model and tokenization"]
  node_tokenizer["Tokenizer<br/>[tokenizer.py]"]
  node_chat_format["Chat formatting<br/>[tokenizer.py]"]
  node_transformer["Transformer model<br/>[model.py]"]
end

node_user(("Model user"))
node_checkpoints[("Model checkpoints")]
node_tokenizer_file["Tokenizer model file"]
node_tiktoken["tiktoken"]

node_user -->|"runs chat"| node_chat_example
node_user -->|"runs completion"| node_text_example
node_chat_example -->|"builds and calls"| node_generator
node_text_example -->|"builds and calls"| node_generator
node_generator -->|"encodes and decodes"| node_tokenizer
node_generator -->|"formats dialogs"| node_chat_format
node_generator -->|"runs inference"| node_transformer
node_generator -->|"loads weights"| node_checkpoints
node_generator -->|"loads vocabulary"| node_tokenizer_file
node_tokenizer -->|"encodes tokens"| node_tiktoken
node_chat_example -->|"prints responses"| node_user
node_text_example -->|"prints completions"| node_user

click node_chat_example "https://github.com/meta-llama/llama3/blob/main/example_chat_completion.py"
click node_text_example "https://github.com/meta-llama/llama3/blob/main/example_text_completion.py"
click node_generator "https://github.com/meta-llama/llama3/blob/main/llama/generation.py"
click node_tokenizer "https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py"
click node_chat_format "https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py"
click node_transformer "https://github.com/meta-llama/llama3/blob/main/llama/model.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat_example,node_text_example,node_user toneBlue
class node_generator,node_checkpoints toneAmber
class node_tokenizer,node_chat_format,node_transformer toneMint
class node_tokenizer_file,node_tiktoken toneIndigo
```

## Components

### Inference examples

- [Chat example](https://github.com/meta-llama/llama3/blob/main/example_chat_completion.py): `example_chat_completion.py`
- [Text example](https://github.com/meta-llama/llama3/blob/main/example_text_completion.py): `example_text_completion.py`

### Inference runtime

- [Llama generation](https://github.com/meta-llama/llama3/blob/main/llama/generation.py): `llama/generation.py`

### Model and tokenization

- [Tokenizer](https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py): `llama/tokenizer.py`
- [Chat formatting](https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py): `llama/tokenizer.py`
- [Transformer model](https://github.com/meta-llama/llama3/blob/main/llama/model.py): `llama/model.py`

### Other components

- **Model user**
- **Model checkpoints**
- **Tokenizer model file**
- **tiktoken**

## Connections

- Model user → Chat example: runs chat (evidence: `README.md`)
- Model user → Text example: runs completion (evidence: `README.md`)
- Chat example → Llama generation: builds and calls (evidence: `example_chat_completion.py`)
- Text example → Llama generation: builds and calls (evidence: `example_text_completion.py`)
- Llama generation → Tokenizer: encodes and decodes (evidence: `llama/generation.py`)
- Llama generation → Chat formatting: formats dialogs (evidence: `llama/generation.py`)
- Llama generation → Transformer model: runs inference (evidence: `llama/generation.py`)
- Llama generation → Model checkpoints: loads weights (evidence: `llama/generation.py`)
- Llama generation → Tokenizer model file: loads vocabulary (evidence: `README.md`)
- Tokenizer → tiktoken: encodes tokens (evidence: `llama/tokenizer.py`)
- Chat example → Model user: prints responses (evidence: `example_chat_completion.py`)
- Text example → Model user: prints completions (evidence: `example_text_completion.py`)
