# google-deepmind/kinetics-i3d architecture

> Architecture diagram of the GitHub repository google-deepmind/kinetics-i3d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/google-deepmind/kinetics-i3d
- Repository: https://github.com/google-deepmind/kinetics-i3d

## Overview

##### Project type and purpose
- This is an open-source research codebase providing pre-trained Inflated 3D ConvNet (I3D) models for video action recognition on the Kinetics dataset.
- It supplies model definitions, checkpoints, evaluation scripts, sample preprocessed data, and tests.

##### File structure
- Top level:
- i3d.py ‑ model architecture (Sonnet + TensorFlow)
- evaluate_sample.py ‑ example inference pipeline
- multi_evaluate.sh ‑ batch runner for different checkpoint combinations
- data/ ‑ checkpoints (rgb and flow streams, ImageNet-initialized and scratch), sample `.npy` arrays and label maps
- out/ ‑ text outputs from multi_evaluate.sh
- No separate “frontend” or “backend” – this is a library/tool for model inference & evaluation.
- Key external configuration: use of TensorFlow, DeepMind Sonnet, NumPy, MediaPipe/OpenCV for preprocessing.

##### README for additional insights
- Two-stream I3D architecture: separate RGB and optical-flow networks, combined by adding logits.
- Sample workflow:
a) build TensorFlow graph (calls i3d.InceptionI3d)
b) restore checkpoint from data/checkpoints
c) load preprocessed `.npy` video clips
d) run session to obtain logits and probabilities
e) output top-20 classes
- multi_evaluate.sh automates inference over four variants (RGB/flow × ImageNet-initialized/scratch) and writes results in out/.

##### Elements
- Blue for data stores
- Green for processing modules (preprocessor, model builder)
- Orange for orchestration scripts
- Gray for external dependencies

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Storage
    subgraph "Data Storage" 
        direction TB
        CP["Checkpoints"]:::data
        subgraph "Checkpoint Variants"
            direction TB
            CP1["rgb_imagenet"]:::data
            CP2["rgb_scratch"]:::data
            CP3["flow_imagenet"]:::data
            CP4["flow_scratch"]:::data
            CP5["rgb_scratch_kin600"]:::data
        end
        SV1["v_CricketShot_g04_c01_flow.npy"]:::data
        SV2["v_CricketShot_g04_c01_rgb.npy"]:::data
        LM1["label_map.txt"]:::data
        LM2["label_map_600.txt"]:::data
    end

    %% Preprocessing
    Pre["MediaPipe/OpenCV Preprocessor"]:::process

    %% Model Definition
    ModelDef["i3d.py"]:::process

    %% Inference Pipeline
    Eval["evaluate_sample.py"]:::script

    %% Orchestration
    Multi["multi_evaluate.sh"]:::script

    %% Testing
    Test["i3d_test.py"]:::test

    %% Outputs
    subgraph "Outputs" 
        direction TB
        OutDir["out/"]:::output
        O1["imagenet_rgb.txt"]:::output
        O2["imagenet_flow.txt"]:::output
        O3["imagenet_joint.txt"]:::output
        O4["no_imagenet_rgb.txt"]:::output
        O5["no_imagenet_flow.txt"]:::output
        O6["no_imagenet_joint.txt"]:::output
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        TF["TensorFlow"]:::external
        Sonnet["DeepMind Sonnet"]:::external
        NumPy["NumPy"]:::external
        Bash["Bash"]:::external
    end

    %% Data Flow
    Pre --> SV1
    Pre --> SV2
    SV1 --> Eval
    SV2 --> Eval
    CP1 --> Eval
    CP2 --> Eval
    CP3 --> Eval
    CP4 --> Eval
    CP5 --> Eval
    LM1 --> Eval
    LM2 --> Eval

    Eval -->|builds TF graph| ModelDef
    ModelDef --> Eval
    Eval -->|"logits & probs"| OutDir
    Multi --> Eval
    Multi --> O1
    Multi --> O2
    Multi --> O3
    Multi --> O4
    Multi --> O5
    Multi --> O6

    Test --> ModelDef

    %% External Links
    TF --> ModelDef
    Sonnet --> ModelDef
    NumPy --> Eval
    Bash --> Multi
    NumPy --> Pre

    %% Click Events
    click CP "https://github.com/google-deepmind/kinetics-i3d/tree/master/data/checkpoints"
    click CP1 "https://github.com/google-deepmind/kinetics-i3d/tree/master/data/checkpoints/rgb_imagenet"
    click CP2 "https://github.com/google-deepmind/kinetics-i3d/tree/master/data/checkpoints/rgb_scratch"
    click CP3 "https://github.com/google-deepmind/kinetics-i3d/tree/master/data/checkpoints/flow_imagenet"
    click CP4 "https://github.com/google-deepmind/kinetics-i3d/tree/master/data/checkpoints/flow_scratch"
    click CP5 "https://github.com/google-deepmind/kinetics-i3d/tree/master/data/checkpoints/rgb_scratch_kin600"
    click SV1 "https://github.com/google-deepmind/kinetics-i3d/blob/master/data/v_CricketShot_g04_c01_flow.npy"
    click SV2 "https://github.com/google-deepmind/kinetics-i3d/blob/master/data/v_CricketShot_g04_c01_rgb.npy"
    click LM1 "https://github.com/google-deepmind/kinetics-i3d/blob/master/data/label_map.txt"
    click LM2 "https://github.com/google-deepmind/kinetics-i3d/blob/master/data/label_map_600.txt"
    click ModelDef "https://github.com/google-deepmind/kinetics-i3d/blob/master/i3d.py"
    click Eval "https://github.com/google-deepmind/kinetics-i3d/blob/master/evaluate_sample.py"
    click Multi "https://github.com/google-deepmind/kinetics-i3d/blob/master/multi_evaluate.sh"
    click Test "https://github.com/google-deepmind/kinetics-i3d/blob/master/i3d_test.py"
    click OutDir "https://github.com/google-deepmind/kinetics-i3d/tree/master/out"
    click O1 "https://github.com/google-deepmind/kinetics-i3d/blob/master/out/imagenet_rgb.txt"
    click O2 "https://github.com/google-deepmind/kinetics-i3d/blob/master/out/imagenet_flow.txt"
    click O3 "https://github.com/google-deepmind/kinetics-i3d/blob/master/out/imagenet_joint.txt"
    click O4 "https://github.com/google-deepmind/kinetics-i3d/blob/master/out/no_imagenet_rgb.txt"
    click O5 "https://github.com/google-deepmind/kinetics-i3d/blob/master/out/no_imagenet_flow.txt"
    click O6 "https://github.com/google-deepmind/kinetics-i3d/blob/master/out/no_imagenet_joint.txt"

    %% Styles
    classDef data fill:#cce5ff,stroke:#1f78b4
    classDef process fill:#d4edda,stroke:#2e7d32
    classDef script fill:#fff3cd,stroke:#ff8f00
    classDef test fill:#f8d7da,stroke:#c62828
    classDef output fill:#e1bee7,stroke:#6a1b9a
    classDef external fill:#f0f0f0,stroke:#555555,stroke-dasharray:5 5
```
