How meta-llama/llama3 works
This deprecated repository is minimal native Llama 3 inference code. Users provide model checkpoints and a tokenizer file, then run either chat or pretrained text completion. The examples build a generator and invoke the corresponding completion method; generation tokenizes or formats prompts, runs the Transformer, and decodes results. The graph preserves both documented inference modes and the tokenizer, chat-formatting, and model layers. The excerpts do not show every detail of model loading, so those relationships rely on the README or are omitted.
Main entry points: generation.py Llama generation · example_chat_completion.py Chat example · example_text_completion.py Text example · tokenizer.py Tokenizer
Components 10
Inference examples
- Chat example
example_chat_completion.py - Text example
example_text_completion.py
Inference runtime
- Llama generation
llama/generation.py
Model and tokenization
- Tokenizer
llama/tokenizer.py - Chat formatting
llama/tokenizer.py - Transformer model
llama/model.py
Other components
- Model user
- Model checkpoints
- Tokenizer model file
- tiktoken
Connections 12
- Model user → Chat example: runs chat
README.md - Model user → Text example: runs completion
README.md - Chat example → Llama generation: builds and calls
example_chat_completion.py - Text example → Llama generation: builds and calls
example_text_completion.py - Llama generation → Tokenizer: encodes and decodes
llama/generation.py - Llama generation → Chat formatting: formats dialogs
llama/generation.py - Llama generation → Transformer model: runs inference
llama/generation.py - Llama generation → Model checkpoints: loads weights
llama/generation.py - Llama generation → Tokenizer model file: loads vocabulary
README.md - Tokenizer → tiktoken: encodes tokens
llama/tokenizer.py - Chat example → Model user: prints responses
example_chat_completion.py - Text example → Model user: prints completions
example_text_completion.py