# microsoft/mage architecture

> Architecture diagram of the GitHub repository microsoft/mage, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/mage
- Repository: https://github.com/microsoft/mage

## Overview

Mage is a two-branch multimodal model family: Mage-Flow generates images from prompts and edits supplied images, while Mage-VL handles image/video understanding and proactive streaming commentary. The Flow app and inference interface feed a shared pipeline that screens and conditions requests, runs the generative model, and returns images. Its model combines a text encoder, VAE, diffusion transformer, and attention kernels. Mage-VL’s two inference scripts are represented as separate user-facing capabilities; the README documents their behavior, but their internal call structure was not sampled.

Main entry points: [`mage_flow/pipeline.py`](https://github.com/microsoft/mage/blob/main/mage_flow/pipeline.py) (Generation pipeline), [`mage_flow/models/mage_flow.py`](https://github.com/microsoft/mage/blob/main/mage_flow/models/mage_flow.py) (Flow model), [`mage_flow/models/modules/text_encoder.py`](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/text_encoder.py) (Text encoder), [`mage_flow/app.py`](https://github.com/microsoft/mage/blob/main/mage_flow/app.py) (Gradio app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_flow_entry["Mage-Flow interfaces"]
  node_flow_app["Gradio app<br/>[app.py]"]
  node_flow_cli["CLI inference<br/>[inference.py]"]
end

subgraph group_flow_runtime["Flow generation runtime"]
  node_flow_pipeline["Generation pipeline<br/>[pipeline.py]"]
end

subgraph group_flow_model["Flow model stack"]
  node_flow_model["Flow model<br/>[mage_flow.py]"]
  node_text_policy["Text screening<br/>[text_encoder.py]"]
  node_text_encoder["Text encoder<br/>[text_encoder.py]"]
  node_vae["Mage VAE<br/>[mage_vae.py]"]
  node_transformer_layers["Transformer layers<br/>[mage_layers.py]"]
  node_attention["Attention backend<br/>[_attn_backend.py]"]
  node_text_components["Text components<br/>[mage_text.py]"]
  node_latent_tools["Latent tools<br/>[mage_latent.py]"]
  node_model_utils["Model utilities<br/>[utils.py]"]
end

subgraph group_vl_runtime["Mage-VL inference"]
  node_vl_understanding["Image/video understanding<br/>[inference_base.py]"]
  node_vl_streaming["Proactive streaming"]
end

node_creator(("Image creator"))
node_vl_user(("Image/video user"))

node_creator -->|"uses"| node_flow_app
node_creator -->|"invokes"| node_flow_cli
node_flow_app -->|"calls"| node_flow_pipeline
node_flow_pipeline -->|"screens prompts"| node_text_policy
node_flow_pipeline -->|"encodes conditioning"| node_text_encoder
node_flow_pipeline -->|"loads model"| node_flow_model
node_flow_model -->|"uses"| node_text_encoder
node_flow_model -->|"loads"| node_vae
node_flow_model -->|"uses"| node_transformer_layers
node_flow_model -->|"selects backend"| node_attention
node_text_encoder -->|"uses"| node_text_components
node_transformer_layers -->|"dispatches attention"| node_attention
node_flow_pipeline -->|"uses"| node_latent_tools
node_flow_pipeline -->|"uses"| node_model_utils
node_vl_user -->|"requests understanding"| node_vl_understanding
node_vl_user -->|"requests commentary"| node_vl_streaming

click node_flow_app "https://github.com/microsoft/mage/blob/main/mage_flow/app.py"
click node_flow_cli "https://github.com/microsoft/mage/blob/main/mage_flow/inference.py"
click node_flow_pipeline "https://github.com/microsoft/mage/blob/main/mage_flow/pipeline.py"
click node_flow_model "https://github.com/microsoft/mage/blob/main/mage_flow/models/mage_flow.py"
click node_text_policy "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/text_encoder.py"
click node_text_encoder "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/text_encoder.py"
click node_vae "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_vae.py"
click node_transformer_layers "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_layers.py"
click node_attention "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/_attn_backend.py"
click node_text_components "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_text.py"
click node_latent_tools "https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_latent.py"
click node_model_utils "https://github.com/microsoft/mage/blob/main/mage_flow/models/utils.py"
click node_vl_understanding "https://github.com/microsoft/mage/blob/main/mage_vl/inference_base.py"
click node_vl_streaming "https://github.com/microsoft/mage/blob/main/mage_vl/inference_streaming.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_flow_app,node_flow_cli,node_vl_user toneBlue
class node_flow_pipeline toneAmber
class node_flow_model,node_text_policy,node_text_encoder,node_vae,node_transformer_layers,node_attention,node_text_components,node_latent_tools,node_model_utils toneMint
class node_vl_understanding,node_vl_streaming toneRose
class node_creator toneIndigo
```

## Components

### Mage-Flow interfaces

- [Gradio app](https://github.com/microsoft/mage/blob/main/mage_flow/app.py): `mage_flow/app.py`
- [CLI inference](https://github.com/microsoft/mage/blob/main/mage_flow/inference.py): `mage_flow/inference.py`

### Flow generation runtime

- [Generation pipeline](https://github.com/microsoft/mage/blob/main/mage_flow/pipeline.py): `mage_flow/pipeline.py`

### Flow model stack

- [Flow model](https://github.com/microsoft/mage/blob/main/mage_flow/models/mage_flow.py): `mage_flow/models/mage_flow.py`
- [Text screening](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/text_encoder.py): `mage_flow/models/modules/text_encoder.py`
- [Text encoder](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/text_encoder.py): `mage_flow/models/modules/text_encoder.py`
- [Mage VAE](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_vae.py): `mage_flow/models/modules/mage_vae.py`
- [Transformer layers](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_layers.py): `mage_flow/models/modules/mage_layers.py`
- [Attention backend](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/_attn_backend.py): `mage_flow/models/modules/_attn_backend.py`
- [Text components](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_text.py): `mage_flow/models/modules/mage_text.py`
- [Latent tools](https://github.com/microsoft/mage/blob/main/mage_flow/models/modules/mage_latent.py): `mage_flow/models/modules/mage_latent.py`
- [Model utilities](https://github.com/microsoft/mage/blob/main/mage_flow/models/utils.py): `mage_flow/models/utils.py`

### Mage-VL inference

- [Image/video understanding](https://github.com/microsoft/mage/blob/main/mage_vl/inference_base.py): `mage_vl/inference_base.py`
- [Proactive streaming](https://github.com/microsoft/mage/blob/main/mage_vl/inference_streaming.py): `mage_vl/inference_streaming.py`

### Other components

- **Image creator**
- **Image/video user**

## Connections

- Image creator → Gradio app: uses
- Image creator → CLI inference: invokes
- Gradio app → Generation pipeline: calls (evidence: `mage_flow/app.py`)
- Generation pipeline → Text screening: screens prompts (evidence: `mage_flow/pipeline.py`)
- Generation pipeline → Text encoder: encodes conditioning (evidence: `mage_flow/pipeline.py`)
- Generation pipeline → Flow model: loads model (evidence: `mage_flow/pipeline.py`)
- Flow model → Text encoder: uses (evidence: `mage_flow/models/mage_flow.py`)
- Flow model → Mage VAE: loads (evidence: `mage_flow/models/mage_flow.py`)
- Flow model → Transformer layers: uses (evidence: `mage_flow/models/mage_flow.py`)
- Flow model → Attention backend: selects backend (evidence: `mage_flow/models/mage_flow.py`)
- Text encoder → Text components: uses (evidence: `mage_flow/models/modules/text_encoder.py`)
- Transformer layers → Attention backend: dispatches attention (evidence: `mage_flow/models/modules/mage_layers.py`)
- Generation pipeline → Latent tools: uses (evidence: `mage_flow/pipeline.py`)
- Generation pipeline → Model utilities: uses (evidence: `mage_flow/pipeline.py`)
- Image/video user → Image/video understanding: requests understanding (evidence: `README.md`)
- Image/video user → Proactive streaming: requests commentary (evidence: `README.md`)