# google-deepmind/ai-foundations architecture

> Architecture diagram of the GitHub repository google-deepmind/ai-foundations, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/google-deepmind/ai-foundations
- Repository: https://github.com/google-deepmind/ai-foundations

## Overview

AI Research Foundations is a repository of practical labs and utilities for an eight-course language-modeling curriculum. A learner’s broad workflow is to work through labs that build from tokenization and language modeling toward neural networks, transformers, fine-tuning, and model analysis; the README documents these course stages but not a single executable end-to-end application flow. The graph includes the sampled runtime components and the documented course-feedback branch. Several course subsystems are represented only by their tree paths because their implementations were not sampled.

Main entry points: [`ai_foundations/generation/gemma.py`](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/generation/gemma.py) (Gemma Generation), [`ai_foundations/training/model.py`](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/model.py) (Training Model), [`ai_foundations/attention/modules.py`](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/attention/modules.py) (Gemma Attention), [`ai_foundations/feedback`](https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/feedback) (Course Feedback Labs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_language["Language Modeling"]
  node_tokenizer["BPE Tokenizer<br/>[bpe_tokenizer.py]"]
  node_ngram["N-gram Model<br/>[model.py]"]
  node_gemma["Gemma Generation<br/>[gemma.py]"]
  node_generation_loading["Model Loading<br/>[loaders.py]"]
end

subgraph group_neural["Neural Models"]
  node_mlp["MLP Utilities<br/>[mlp.py]"]
  node_transformer["Transformer Layers<br/>[layers.py]"]
  node_attention["Gemma Attention<br/>[modules.py]"]
  node_attention_core["Attention Operations<br/>[attention.py]"]
end

subgraph group_learning["Training and Feedback"]
  node_training["Training Model<br/>[model.py]"]
  node_losses["Training Losses<br/>[losses.py]"]
  node_callbacks["Training Callbacks<br/>[callbacks.py]"]
  node_course_feedback["Course Feedback Labs"]
  node_feedback_render["Feedback Renderer<br/>[utils.py]"]
  node_qa_format["QA Formatting<br/>[format_qa.py]"]
end

subgraph group_analysis["Analysis Tools"]
  node_embedding_tools["Embedding Tools"]
  node_visualization["Plots and Visualizations"]
  node_attention_visualization["Attention Visualization"]
  node_plotting["Plotting Utilities<br/>[plots.py]"]
  node_formatting_utils["Formatting Utilities<br/>[formatting.py]"]
end

node_learner(("Learner"))

node_learner -->|"works through labs"| node_tokenizer
node_learner -->|"works through labs"| node_ngram
node_learner -->|"works through labs"| node_mlp
node_learner -->|"works through labs"| node_transformer
node_learner -->|"uses course labs"| node_course_feedback
node_generation_loading -->|"loads model"| node_gemma
node_gemma -->|"uses attention model"| node_attention
node_attention -->|"delegates attention"| node_attention_core
node_training -->|"uses losses"| node_losses
node_training -->|"uses layers"| node_transformer
node_callbacks -->|"uses generation"| node_gemma
node_course_feedback -->|"renders feedback"| node_feedback_render
node_embedding_tools -.->|"analyzes embeddings"| node_visualization
node_attention_visualization -.->|"uses plots"| node_plotting

click node_tokenizer "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/tokenization/bpe_tokenizer.py"
click node_ngram "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/ngram/model.py"
click node_gemma "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/generation/gemma.py"
click node_generation_loading "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/generation/loaders.py"
click node_mlp "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/machine_learning/mlp.py"
click node_transformer "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/transformers/layers.py"
click node_attention "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/attention/modules.py"
click node_attention_core "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/attention/attention.py"
click node_training "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/model.py"
click node_losses "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/losses.py"
click node_callbacks "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/callbacks.py"
click node_course_feedback "https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/feedback"
click node_feedback_render "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/feedback/utils.py"
click node_qa_format "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/formatting/format_qa.py"
click node_embedding_tools "https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/embeddings"
click node_visualization "https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/visualizations"
click node_attention_visualization "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/visualizations/attention_visualization.py"
click node_plotting "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/visualizations/plots.py"
click node_formatting_utils "https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/utils/formatting.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tokenizer,node_ngram,node_gemma,node_generation_loading toneBlue
class node_mlp,node_transformer,node_attention,node_attention_core toneAmber
class node_training,node_losses,node_callbacks,node_course_feedback,node_feedback_render,node_qa_format toneMint
class node_embedding_tools,node_visualization,node_attention_visualization,node_plotting,node_formatting_utils toneRose
class node_learner toneIndigo
```

## Components

### Language Modeling

- [BPE Tokenizer](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/tokenization/bpe_tokenizer.py): `ai_foundations/tokenization/bpe_tokenizer.py`
- [N-gram Model](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/ngram/model.py): `ai_foundations/ngram/model.py`
- [Gemma Generation](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/generation/gemma.py): `ai_foundations/generation/gemma.py`
- [Model Loading](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/generation/loaders.py): `ai_foundations/generation/loaders.py`

### Neural Models

- [MLP Utilities](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/machine_learning/mlp.py): `ai_foundations/machine_learning/mlp.py`
- [Transformer Layers](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/transformers/layers.py): `ai_foundations/transformers/layers.py`
- [Gemma Attention](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/attention/modules.py): `ai_foundations/attention/modules.py`
- [Attention Operations](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/attention/attention.py): `ai_foundations/attention/attention.py`

### Training and Feedback

- [Training Model](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/model.py): `ai_foundations/training/model.py`
- [Training Losses](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/losses.py): `ai_foundations/training/losses.py`
- [Training Callbacks](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/training/callbacks.py): `ai_foundations/training/callbacks.py`
- [Course Feedback Labs](https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/feedback): `ai_foundations/feedback`
- [Feedback Renderer](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/feedback/utils.py): `ai_foundations/feedback/utils.py`
- [QA Formatting](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/formatting/format_qa.py): `ai_foundations/formatting/format_qa.py`

### Analysis Tools

- [Embedding Tools](https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/embeddings): `ai_foundations/embeddings`
- [Plots and Visualizations](https://github.com/google-deepmind/ai-foundations/tree/main/ai_foundations/visualizations): `ai_foundations/visualizations`
- [Attention Visualization](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/visualizations/attention_visualization.py): `ai_foundations/visualizations/attention_visualization.py`
- [Plotting Utilities](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/visualizations/plots.py): `ai_foundations/visualizations/plots.py`
- [Formatting Utilities](https://github.com/google-deepmind/ai-foundations/blob/main/ai_foundations/utils/formatting.py): `ai_foundations/utils/formatting.py`

### Other components

- **Learner**

## Connections

- Learner → BPE Tokenizer: works through labs (evidence: `README.md`)
- Learner → N-gram Model: works through labs (evidence: `README.md`)
- Learner → MLP Utilities: works through labs (evidence: `README.md`)
- Learner → Transformer Layers: works through labs (evidence: `README.md`)
- Learner → Course Feedback Labs: uses course labs (evidence: `README.md`)
- Model Loading → Gemma Generation: loads model (evidence: `ai_foundations/generation/loaders.py`)
- Gemma Generation → Gemma Attention: uses attention model (evidence: `ai_foundations/generation/gemma.py`)
- Gemma Attention → Attention Operations: delegates attention (evidence: `ai_foundations/attention/modules.py`)
- Training Model → Training Losses: uses losses (evidence: `ai_foundations/training/model.py`)
- Training Model → Transformer Layers: uses layers (evidence: `ai_foundations/training/model.py`)
- Training Callbacks → Gemma Generation: uses generation (evidence: `ai_foundations/training/callbacks.py`)
- Course Feedback Labs → Feedback Renderer: renders feedback (evidence: `ai_foundations/feedback/course_2/embeddings/embedding_tests.py`)
- Embedding Tools → Plots and Visualizations: analyzes embeddings
- Attention Visualization → Plotting Utilities: uses plots
