# michaelyliu6/annotated-gemma architecture

> Architecture diagram of the GitHub repository michaelyliu6/annotated-gemma, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/michaelyliu6/annotated-gemma
- Repository: https://github.com/michaelyliu6/annotated-gemma

## Overview

Based on our review of the file tree and README, the Gemma project is an open‐source implementation of a family of decoder-only large language models (both text-only and multimodal) built in PyTorch. It supports multiple inference environments (CPU, GPU, TPU) using both standard PyTorch and PyTorch/XLA, and it is containerized using Docker. The primary purpose is to offer state‑of‑the‑art models (inspired by Google Gemini) with flexible deployment options.

##### Main Components
- Core Model Library – Represented by the “gemma” package. It includes:
– Core models (e.g., gemma3_model.py, model.py, model_xla.py)
– Preprocessing components (e.g., gemma3_preprocessor.py, siglip_vision/preprocessor.py)
– Vision module under “gemma/siglip_vision”, indicating the multimodal capabilities.
– Configuration files (config.py in both gemma and siglip_vision)
- Tokenizer – Present in a separate “tokenizer” folder (contains the tokenizer model files) used for processing text.
- Scripts – The “scripts” directory includes entry-point scripts for running inference with different configurations:
– run.py (possibly for a default/single-modality run)
– run_multimodal.py (for handling multimodal inference)
– run_xla.py (for PyTorch/XLA specific execution)
- Docker Setup – The “docker” directory has multiple Dockerfiles (Dockerfile, xla.Dockerfile, xla_gpu.Dockerfile). These represent different container setups for CPU, GPU, and TPU (or XLA-specific deployments).

##### Relationships and Interactions
- The user/deployment environment interacts with the system via Docker containers. Each container runs one of the provided Docker images (standard or XLA variants).
- Inside these containers, the “scripts” act as the entry points that load checkpoints (provided via mounted volumes) and trigger the appropriate model (choosing variants based on CLI flags).
- The “gemma” package modules work together to load model weights, preprocess input data (using both text and vision preprocessor modules), and run inference.
- The “tokenizer” module is utilized during preprocessing to transform raw text into token sequences.
- Conditional branches (for text-only vs. multimodal processing) might be indicated by how different scripts call their respective preprocessor and model modules.

##### Architectural Patterns and Design Principles
- Modular Content – Code is organized into clearly delineated modules (model, preprocessor, tokenizer, vision).
- Separation via Containers – Docker containerization encapsulates runtime environments for standard PyTorch and PyTorch/XLA, ensuring that each hardware target (CPU/GPU/TPU) is addressed separately.
- Reusability – The repository supports multiple variants and has clear separation between inference logic (scripts) and core model definitions.
- Configurability – Command-line flags (e.g., --ckpt, --variant, --device) allow external configuration of model behavior and environment.

– Another for core Python modules.
– A third for supporting modules such as the tokenizer.

This approach will help in creating a detailed, component-wise representation of the Gemma project’s software architecture, ensuring that every essential element—from Docker runtime considerations to detailed internal module organization—is clearly communicated.

## Diagram (Mermaid)

```mermaid
flowchart TD
    ExternalInput["User Input/Checkpoint Volume"]:::external

    subgraph "Deployment Environment"
        D1["Docker: gemma (CPU/PyTorch)"]:::docker
        D2["Docker: gemma_xla (PyTorch/XLA)"]:::docker
        D3["Docker: gemma_xla_gpu (GPU/XLA)"]:::docker
    end

    subgraph "Inference Scripts Layer"
        S1["scripts/run.py"]:::scripts
        S2["scripts/run_multimodal.py"]:::scripts
        S3["scripts/run_xla.py"]:::scripts
    end

    subgraph "Core Model Library (Gemma Package)"
        G1["gemma3_model.py"]:::core
        G2["model.py"]:::core
        G3["model_xla.py"]:::core
        G4["gemma3_preprocessor.py"]:::preproc
        G5["tokenizer.py"]:::core
        subgraph "Vision Module"
            V1["siglip_vision_model.py"]:::core
            V2["preprocessor.py"]:::preproc
            V3["pan_and_scan.py"]:::core
            V4["config.py"]:::config
        end
    end

    subgraph "Tokenizer Module"
        T1["tokenizer.model"]:::tokenizer
        T2["gemma3_cleaned_262144_v2.spiece.model"]:::tokenizer
    end

    %% External input feeds Docker containers
    ExternalInput --> D1
    ExternalInput --> D2
    ExternalInput --> D3

    %% Docker containers execute inference scripts
    D1 --> S1
    D1 --> S2
    D1 --> S3
    D2 --> S1
    D2 --> S2
    D2 --> S3
    D3 --> S1
    D3 --> S2
    D3 --> S3

    %% Inference scripts call core modules
    S1 -->|"calls"| G1
    S1 -->|"calls"| G4
    S1 -->|"calls"| G5

    S2 -->|"calls"| V2
    S2 -->|"calls"| V1

    S3 -->|"calls"| G3

    %% Tokenizer module preprocesses textual inputs, feeding into preprocessor modules
    T1 -->|"tokenizes"| G4
    T1 -->|"tokenizes"| V2

    %% Preprocessing in Gemma Package funnels into model inference
    G4 -->|"processes"| G1
    V2 -->|"processes"| V1

    %% Style definitions for color coding
    classDef docker fill:#FDEBD0,stroke:#000,stroke-width:2px;
    classDef scripts fill:#D6EAF8,stroke:#000,stroke-width:2px;
    classDef core fill:#D5F5E3,stroke:#000,stroke-width:2px;
    classDef preproc fill:#FCF3CF,stroke:#000,stroke-width:2px;
    classDef tokenizer fill:#FADBD8,stroke:#000,stroke-width:2px;
    classDef external fill:#D2B4DE,stroke:#000,stroke-width:2px;
    classDef config fill:#D6EAF8,stroke:#000,stroke-width:2px;

    %% Click events for Docker Setup
    click D1 "https://github.com/michaelyliu6/annotated-gemma/tree/main/docker/Dockerfile"
    click D2 "https://github.com/michaelyliu6/annotated-gemma/blob/main/docker/xla.Dockerfile"
    click D3 "https://github.com/michaelyliu6/annotated-gemma/blob/main/docker/xla_gpu.Dockerfile"

    %% Click events for Inference Scripts
    click S1 "https://github.com/michaelyliu6/annotated-gemma/blob/main/scripts/run.py"
    click S2 "https://github.com/michaelyliu6/annotated-gemma/blob/main/scripts/run_multimodal.py"
    click S3 "https://github.com/michaelyliu6/annotated-gemma/blob/main/scripts/run_xla.py"

    %% Click events for Gemma Package components
    click G1 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/gemma3_model.py"
    click G2 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/model.py"
    click G3 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/model_xla.py"
    click G4 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/gemma3_preprocessor.py"
    click G5 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/tokenizer.py"

    %% Click events for Vision Module components
    click V1 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/siglip_vision/siglip_vision_model.py"
    click V2 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/siglip_vision/preprocessor.py"
    click V3 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/siglip_vision/pan_and_scan.py"
    click V4 "https://github.com/michaelyliu6/annotated-gemma/blob/main/gemma/siglip_vision/config.py"

    %% Click events for Tokenizer Module components
    click T1 "https://github.com/michaelyliu6/annotated-gemma/blob/main/tokenizer/tokenizer.model"
    click T2 "https://github.com/michaelyliu6/annotated-gemma/blob/main/tokenizer/gemma3_cleaned_262144_v2.spiece.model"
```
