# meta-llama/llama-models architecture

> Architecture diagram of the GitHub repository meta-llama/llama-models, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/meta-llama/llama-models
- Repository: https://github.com/meta-llama/llama-models

## Overview

This repository distributes Llama model assets and provides a CLI for cataloguing, downloading, verifying, and removing them, plus native Llama 3 and Llama 4 inference code. The main usage paths are CLI request → model catalog/download → local checkpoint, and prompt → model-specific inference entry point → generated text. Llama 4 also includes image preprocessing and vision components; quantization is an optional inference path. The README documents these capabilities; sampled source confirms selected implementations, so unsampled wiring is left unspecified.

Main entry points: [`models/cli`](https://github.com/meta-llama/llama-models/tree/main/models/cli) (Model CLI Commands), [`models/llama3/generation.py`](https://github.com/meta-llama/llama-models/blob/main/models/llama3/generation.py) (Llama 3 Generation), [`models/checkpoint.py`](https://github.com/meta-llama/llama-models/blob/main/models/checkpoint.py) (Local Checkpoints), [`models/cli/download.py`](https://github.com/meta-llama/llama-models/blob/main/models/cli/download.py) (Download Command)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["Model CLI"]
  node_cli_router["CLI Router<br/>[llama.py]"]
  node_download["Download Command<br/>[download.py]"]
  node_verify["Verify Download<br/>[verify_download.py]"]
  node_remove["Remove Model<br/>[remove.py]"]
  node_describe["Describe Model<br/>[describe.py]"]
  node_prompt_format["Prompt Format<br/>[prompt_format.py]"]
  node_model_cli["Model CLI Commands"]
end

subgraph group_assets["Model Assets"]
  node_catalog["Model Catalog<br/>[sku_list.py]"]
  node_checkpoint[("Local Checkpoints<br/>[checkpoint.py]")]
end

subgraph group_llama3["Llama 3 Runtime"]
  node_llama3_entry["Llama 3 Scripts"]
  node_llama3_generation["Llama 3 Generation<br/>[generation.py]"]
  node_llama3_format["Chat Formatting<br/>[chat_format.py]"]
  node_llama3_tokenizer["Llama 3 Tokenizer<br/>[tokenizer.py]"]
  node_llama3_model["Llama 3 Model<br/>[model.py]"]
end

subgraph group_llama4["Llama 4 Runtime"]
  node_llama4_entry["Llama 4 Scripts"]
  node_llama4_generation["Llama 4 Generation<br/>[generation.py]"]
  node_llama4_model["Llama 4 Model<br/>[model.py]"]
  node_image_preprocess["Image Preprocessing<br/>[preprocess.py]"]
  node_vision_encoder["Vision Encoder<br/>[encoder.py]"]
  node_vision_embedding["Vision Embeddings<br/>[embedding.py]"]
end

subgraph group_shared["Quantization"]
  node_quant3["Llama 3 Quantization<br/>[loader.py]"]
  node_quant4["Llama 4 Quantization<br/>[loader.py]"]
  node_quant_impl["Quantization Kernels<br/>[quantize_impls.py]"]
end

node_developer(("Developer"))
node_model_hub(("Meta / Hugging Face"))

node_developer -->|"runs commands"| node_cli_router
node_cli_router -->|"dispatches"| node_model_cli
node_model_cli -->|"lists models"| node_catalog
node_model_cli -->|"dispatches"| node_describe
node_model_cli -->|"dispatches"| node_download
node_model_cli -->|"dispatches"| node_verify
node_model_cli -->|"dispatches"| node_remove
node_model_cli -->|"dispatches"| node_prompt_format
node_download -->|"selects model"| node_catalog
node_download -.->|"downloads from"| node_model_hub
node_download -->|"writes files"| node_checkpoint
node_verify -->|"checks files"| node_checkpoint
node_remove -->|"removes files"| node_checkpoint
node_developer -->|"runs inference"| node_llama3_entry
node_llama3_entry -->|"invokes"| node_llama3_generation
node_llama3_generation -->|"formats chat"| node_llama3_format
node_llama3_generation -->|"tokenizes prompts"| node_llama3_tokenizer
node_llama3_generation -->|"runs model"| node_llama3_model
node_llama3_generation -->|"loads weights"| node_checkpoint
node_llama3_generation -.->|"converts weights"| node_quant3
node_developer -->|"runs inference"| node_llama4_entry
node_llama4_entry -->|"invokes"| node_llama4_generation
node_llama4_generation -->|"runs model"| node_llama4_model
node_llama4_generation -->|"loads weights"| node_checkpoint
node_llama4_generation -.->|"converts weights"| node_quant4
node_quant3 -->|"uses kernels"| node_quant_impl
node_quant4 -->|"uses kernels"| node_quant_impl
node_image_preprocess -->|"prepares image input"| node_vision_embedding
node_vision_embedding -->|"encodes patches"| node_vision_encoder

click node_cli_router "https://github.com/meta-llama/llama-models/blob/main/models/cli/llama.py"
click node_catalog "https://github.com/meta-llama/llama-models/blob/main/models/sku_list.py"
click node_download "https://github.com/meta-llama/llama-models/blob/main/models/cli/download.py"
click node_verify "https://github.com/meta-llama/llama-models/blob/main/models/cli/verify_download.py"
click node_remove "https://github.com/meta-llama/llama-models/blob/main/models/cli/remove.py"
click node_describe "https://github.com/meta-llama/llama-models/blob/main/models/cli/describe.py"
click node_prompt_format "https://github.com/meta-llama/llama-models/blob/main/models/cli/prompt_format.py"
click node_model_cli "https://github.com/meta-llama/llama-models/tree/main/models/cli"
click node_checkpoint "https://github.com/meta-llama/llama-models/blob/main/models/checkpoint.py"
click node_llama3_entry "https://github.com/meta-llama/llama-models/tree/main/models/llama3/scripts"
click node_llama3_generation "https://github.com/meta-llama/llama-models/blob/main/models/llama3/generation.py"
click node_llama3_format "https://github.com/meta-llama/llama-models/blob/main/models/llama3/chat_format.py"
click node_llama3_tokenizer "https://github.com/meta-llama/llama-models/blob/main/models/llama3/tokenizer.py"
click node_llama3_model "https://github.com/meta-llama/llama-models/blob/main/models/llama3/model.py"
click node_llama4_entry "https://github.com/meta-llama/llama-models/tree/main/models/llama4/scripts"
click node_llama4_generation "https://github.com/meta-llama/llama-models/blob/main/models/llama4/generation.py"
click node_llama4_model "https://github.com/meta-llama/llama-models/blob/main/models/llama4/model.py"
click node_image_preprocess "https://github.com/meta-llama/llama-models/blob/main/models/llama4/preprocess.py"
click node_vision_encoder "https://github.com/meta-llama/llama-models/blob/main/models/llama4/vision/encoder.py"
click node_vision_embedding "https://github.com/meta-llama/llama-models/blob/main/models/llama4/vision/embedding.py"
click node_quant3 "https://github.com/meta-llama/llama-models/blob/main/models/llama3/quantization/loader.py"
click node_quant4 "https://github.com/meta-llama/llama-models/blob/main/models/llama4/quantization/loader.py"
click node_quant_impl "https://github.com/meta-llama/llama-models/blob/main/models/quantize_impls.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli_router,node_download,node_verify,node_remove,node_describe,node_prompt_format,node_model_cli toneBlue
class node_catalog,node_checkpoint toneAmber
class node_llama3_entry,node_llama3_generation,node_llama3_format,node_llama3_tokenizer,node_llama3_model toneMint
class node_llama4_entry,node_llama4_generation,node_llama4_model,node_image_preprocess,node_vision_encoder,node_vision_embedding toneRose
class node_quant3,node_quant4,node_quant_impl,node_developer,node_model_hub toneIndigo
```

## Components

### Model CLI

- [CLI Router](https://github.com/meta-llama/llama-models/blob/main/models/cli/llama.py): `models/cli/llama.py`
- [Download Command](https://github.com/meta-llama/llama-models/blob/main/models/cli/download.py): `models/cli/download.py`
- [Verify Download](https://github.com/meta-llama/llama-models/blob/main/models/cli/verify_download.py): `models/cli/verify_download.py`
- [Remove Model](https://github.com/meta-llama/llama-models/blob/main/models/cli/remove.py): `models/cli/remove.py`
- [Describe Model](https://github.com/meta-llama/llama-models/blob/main/models/cli/describe.py): `models/cli/describe.py`
- [Prompt Format](https://github.com/meta-llama/llama-models/blob/main/models/cli/prompt_format.py): `models/cli/prompt_format.py`
- [Model CLI Commands](https://github.com/meta-llama/llama-models/tree/main/models/cli): `models/cli`

### Model Assets

- [Model Catalog](https://github.com/meta-llama/llama-models/blob/main/models/sku_list.py): `models/sku_list.py`
- [Local Checkpoints](https://github.com/meta-llama/llama-models/blob/main/models/checkpoint.py): `models/checkpoint.py`

### Llama 3 Runtime

- [Llama 3 Scripts](https://github.com/meta-llama/llama-models/tree/main/models/llama3/scripts): `models/llama3/scripts`
- [Llama 3 Generation](https://github.com/meta-llama/llama-models/blob/main/models/llama3/generation.py): `models/llama3/generation.py`
- [Chat Formatting](https://github.com/meta-llama/llama-models/blob/main/models/llama3/chat_format.py): `models/llama3/chat_format.py`
- [Llama 3 Tokenizer](https://github.com/meta-llama/llama-models/blob/main/models/llama3/tokenizer.py): `models/llama3/tokenizer.py`
- [Llama 3 Model](https://github.com/meta-llama/llama-models/blob/main/models/llama3/model.py): `models/llama3/model.py`

### Llama 4 Runtime

- [Llama 4 Scripts](https://github.com/meta-llama/llama-models/tree/main/models/llama4/scripts): `models/llama4/scripts`
- [Llama 4 Generation](https://github.com/meta-llama/llama-models/blob/main/models/llama4/generation.py): `models/llama4/generation.py`
- [Llama 4 Model](https://github.com/meta-llama/llama-models/blob/main/models/llama4/model.py): `models/llama4/model.py`
- [Image Preprocessing](https://github.com/meta-llama/llama-models/blob/main/models/llama4/preprocess.py): `models/llama4/preprocess.py`
- [Vision Encoder](https://github.com/meta-llama/llama-models/blob/main/models/llama4/vision/encoder.py): `models/llama4/vision/encoder.py`
- [Vision Embeddings](https://github.com/meta-llama/llama-models/blob/main/models/llama4/vision/embedding.py): `models/llama4/vision/embedding.py`

### Quantization

- [Llama 3 Quantization](https://github.com/meta-llama/llama-models/blob/main/models/llama3/quantization/loader.py): `models/llama3/quantization/loader.py`
- [Llama 4 Quantization](https://github.com/meta-llama/llama-models/blob/main/models/llama4/quantization/loader.py): `models/llama4/quantization/loader.py`
- [Quantization Kernels](https://github.com/meta-llama/llama-models/blob/main/models/quantize_impls.py): `models/quantize_impls.py`

### Other components

- **Developer**
- **Meta / Hugging Face**

## Connections

- Developer → CLI Router: runs commands
- CLI Router → Model CLI Commands: dispatches
- Model CLI Commands → Model Catalog: lists models
- Model CLI Commands → Describe Model: dispatches
- Model CLI Commands → Download Command: dispatches
- Model CLI Commands → Verify Download: dispatches
- Model CLI Commands → Remove Model: dispatches
- Model CLI Commands → Prompt Format: dispatches
- Download Command → Model Catalog: selects model
- Download Command → Meta / Hugging Face: downloads from
- Download Command → Local Checkpoints: writes files
- Verify Download → Local Checkpoints: checks files
- Remove Model → Local Checkpoints: removes files
- Developer → Llama 3 Scripts: runs inference
- Llama 3 Scripts → Llama 3 Generation: invokes
- Llama 3 Generation → Chat Formatting: formats chat
- Llama 3 Generation → Llama 3 Tokenizer: tokenizes prompts
- Llama 3 Generation → Llama 3 Model: runs model
- Llama 3 Generation → Local Checkpoints: loads weights
- Llama 3 Generation → Llama 3 Quantization: converts weights
- Developer → Llama 4 Scripts: runs inference
- Llama 4 Scripts → Llama 4 Generation: invokes
- Llama 4 Generation → Llama 4 Model: runs model
- Llama 4 Generation → Local Checkpoints: loads weights
- Llama 4 Generation → Llama 4 Quantization: converts weights
- Llama 3 Quantization → Quantization Kernels: uses kernels
- Llama 4 Quantization → Quantization Kernels: uses kernels
- Image Preprocessing → Vision Embeddings: prepares image input
- Vision Embeddings → Vision Encoder: encodes patches