# facebookresearch/slowfast architecture

> Architecture diagram of the GitHub repository facebookresearch/slowfast, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookresearch/slowfast
- Repository: https://github.com/facebookresearch/slowfast

## Overview

##### Project type and purpose
- PySlowFast is an open-source video understanding framework built on PyTorch.
- It provides end-to-end video model backbones, training, evaluation, and visualization tools for video classification/detection research.
- Key use cases: training new models, running inference/demo, evaluating on AVA/Kinetics datasets, visualizing results.

##### File structure
- Top-level components:
- slowfast/                – Core library: config, datasets, models, utils, visualization
- configs/                 – YAML configurations for experiments (various datasets, backbones)
- tools/                   – CLI entrypoints: train_net.py, test_net.py, run_net.py, demo_net.py, benchmark.py, visualization.py
- ava_evaluation/          – Evaluation code for AVA action detection
- demo/                    – Demo YAMLs and visualization assets
- projects/                – Research projects/extensions (MAE, maskfeat, MViTv2, etc.)
- setup.py, setup.cfg      – Package installation
- scripts & docs: INSTALL.md, GETTING_STARTED.md, MODEL_ZOO.md, VISUALIZATION_TOOLS.md, etc.

##### README for additional insights
- Architecture is modular:
- Config system decouples code from experiment settings (slowfast/config)
- Datasets abstraction (slowfast/datasets)
- Model builder pattern (slowfast/models/video_model_builder, custom_video_model_builder, ptv_model_builder)
- Utilities for checkpointing, distributed training, logging, lr policy (slowfast/utils)
- Visualization components (slowfast/visualization, tools/visualization.py)
- Supports multiple backbones: SlowFast, I3D, X3D, MViT, Rev-ViT, contrastive SSL, masked SSL.

##### Final advice
- Be very detailed: break “slowfast/models” further into builders, backbone definitions, loss.
- Essential components; show both training and inference paths.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Subgraphs for layers
    subgraph "User/Researcher CLI Layer" 
        direction TB
        TrainNet["train_net.py"]:::cli
        TestNet["test_net.py"]:::cli
        RunNet["run_net.py"]:::cli
        DemoNet["demo_net.py"]:::cli
        Benchmark["benchmark.py"]:::cli
        ToolsVis["visualization.py"]:::cli
    end

    subgraph "Configuration Management" 
        direction TB
        ConfigYAML["YAML Configs"]:::config
        ConfigAPI1["defaults.py"]:::config
        ConfigAPI2["custom_config.py"]:::config
    end

    subgraph "Data Pipeline" 
        direction TB
        Registry["build.py"]:::core
        Decoder["decoder.py"]:::core
        VideoContainer["video_container.py"]:::core
        Transforms["Transforms & Augmentations"]:::core
        Loader["loader.py"]:::core
        Sampler["multigrid_helper.py"]:::core
        Datasets["Dataset Definitions"]:::core
    end

    subgraph "Model Factory" 
        direction TB
        VideoBuilder["video_model_builder.py"]:::core
        CustomBuilder["custom_video_model_builder.py"]:::core
        PTVBuilder["ptv_model_builder.py"]:::core
        ModelReg["build.py"]:::core
        BackboneHelpers["Backbone Helpers"]:::core
        Operators["operators.py"]:::core
        Attention["attention.py"]:::core
        BatchNorm["batchnorm_helper.py"]:::core
        HeadsLosses["Heads & Losses"]:::core
        OptimizerUtils["optimizer.py & utils.py"]:::core
    end

    subgraph "Training & Evaluation Loop"
        direction TB
        TrainScript["tools/train_net.py"]:::cli
        EvalTest["tools/test_net.py"]:::cli
        EvalBench["tools/benchmark.py"]:::cli
        AVAEval["AVA Eval Modules"]:::core
        Utils["Utils (checkpoint, distributed, lr_policy, metrics)"]:::core
        Checkpoints["Checkpoints/Logs"]:::storage
    end

    subgraph "Visualization & Logging"
        direction TB
        TBVis["tensorboard_vis.py"]:::core
        VidVis["video_visualizer.py"]:::core
        PredVis["prediction_vis.py"]:::core
        GradCAM["gradcam_utils.py"]:::core
        DemoTools["tools/visualization.py & demo assets"]:::cli
    end

    subgraph "Projects & Model Zoo Extensions"
        direction TB
        ProjectsDir["projects/"]:::external
        ModelZoo["MODEL_ZOO.md"]:::external
    end

    %% Data stores and external
    VideoFiles["Video Files"]:::storage
    ExternalDS["External Datasets"]:::external
    PyTorch["PyTorch & DDP"]:::external
    Tensorboard["Tensorboard"]:::external

    %% Connections
    TrainNet -->|parses args| ConfigYAML
    TestNet -->|parses args| ConfigYAML
    RunNet -->|parses args| ConfigYAML
    DemoNet -->|parses args| ConfigYAML
    Benchmark -->|parses args| ConfigYAML
    ToolsVis -->|parses args| ConfigYAML

    ConfigYAML -->|loads defaults| ConfigAPI1
    ConfigYAML -->|loads custom| ConfigAPI2

    ConfigAPI1 --> Registry
    ConfigAPI2 --> Registry

    VideoFiles --> Loader
    ExternalDS --> Loader
    Loader -->|batches| Registry
    Registry --> VideoBuilder
    Registry --> CustomBuilder
    Registry --> PTVBuilder

    VideoBuilder -->|uses| BackboneHelpers
    VideoBuilder --> Operators
    VideoBuilder --> Attention
    VideoBuilder --> BatchNorm
    VideoBuilder --> HeadsLosses
    VideoBuilder --> OptimizerUtils

    TrainScript --> Registry
    TrainScript --> Utils
    TrainScript --> Checkpoints
    EvalTest --> Loader
    EvalTest --> Checkpoints
    EvalBench --> Loader
    EvalBench --> Checkpoints

    Checkpoints -->|visualize| TBVis
    Checkpoints -->|visualize| VidVis
    Checkpoints -->|visualize| PredVis
    Checkpoints -->|visualize| GradCAM
    DemoTools --> DemoNet

    ProjectsDir -.-> VideoBuilder
    ProjectsDir -.-> Registry
    ModelZoo -.-> VideoBuilder

    PyTorch --> Utils
    PyTorch --> VideoBuilder
    Tensorboard --> TBVis

    %% Click Events
    click TrainNet "https://github.com/facebookresearch/slowfast/blob/main/tools/train_net.py"
    click TestNet "https://github.com/facebookresearch/slowfast/blob/main/tools/test_net.py"
    click RunNet "https://github.com/facebookresearch/slowfast/blob/main/tools/run_net.py"
    click DemoNet "https://github.com/facebookresearch/slowfast/blob/main/tools/demo_net.py"
    click Benchmark "https://github.com/facebookresearch/slowfast/blob/main/tools/benchmark.py"
    click ToolsVis "https://github.com/facebookresearch/slowfast/blob/main/tools/visualization.py"
    click ConfigYAML "https://github.com/facebookresearch/slowfast/tree/main/configs/"
    click ConfigAPI1 "https://github.com/facebookresearch/slowfast/blob/main/slowfast/config/defaults.py"
    click ConfigAPI2 "https://github.com/facebookresearch/slowfast/blob/main/slowfast/config/custom_config.py"
    click Registry "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/build.py"
    click Decoder "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/decoder.py"
    click VideoContainer "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/video_container.py"
    click Transforms "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/transform.py"
    click Sampler "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/multigrid_helper.py"
    click Loader "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/loader.py"
    click Datasets "https://github.com/facebookresearch/slowfast/blob/main/slowfast/datasets/imagenet.py"
    click VideoBuilder "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/video_model_builder.py"
    click CustomBuilder "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/custom_video_model_builder.py"
    click PTVBuilder "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/ptv_model_builder.py"
    click ModelReg "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/build.py"
    click BackboneHelpers "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/resnet_helper.py"
    click Operators "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/operators.py"
    click Attention "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/attention.py"
    click BatchNorm "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/batchnorm_helper.py"
    click HeadsLosses "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/losses.py"
    click OptimizerUtils "https://github.com/facebookresearch/slowfast/blob/main/slowfast/models/optimizer.py"
    click AVAEval "https://github.com/facebookresearch/slowfast/blob/main/ava_evaluation/metrics.py"
    click Utils "https://github.com/facebookresearch/slowfast/blob/main/slowfast/utils/checkpoint.py"
    click TBVis "https://github.com/facebookresearch/slowfast/blob/main/slowfast/visualization/tensorboard_vis.py"
    click VidVis "https://github.com/facebookresearch/slowfast/blob/main/slowfast/visualization/video_visualizer.py"
    click PredVis "https://github.com/facebookresearch/slowfast/blob/main/slowfast/visualization/prediction_vis.py"
    click GradCAM "https://github.com/facebookresearch/slowfast/blob/main/slowfast/visualization/gradcam_utils.py"
    click ProjectsDir "https://github.com/facebookresearch/slowfast/tree/main/projects/"
    click ModelZoo "https://github.com/facebookresearch/slowfast/blob/main/MODEL_ZOO.md"

    %% Styles
    classDef cli fill:#b3e2cd,stroke:#006d2c,color:#000
    classDef config fill:#fed976,stroke:#f16913,color:#000
    classDef core fill:#9ecae1,stroke:#2171b5,color:#000
    classDef external fill:#d9d9d9,stroke:#636363,color:#000
    classDef storage fill:#f0f0f0,stroke:#969696,color:#000
```
