# pq-yang/matanyone2 architecture

> Architecture diagram of the GitHub repository pq-yang/matanyone2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pq-yang/matanyone2
- Repository: https://github.com/pq-yang/matanyone2

## Overview

MatAnyone 2 is a human video-matting framework. Its main workflow takes a video and first-frame target mask, processes frames with persistent object and memory state, and produces foreground and alpha videos. Users can run this through the CLI/Python inference path or the interactive demo, which adds click-based mask creation; evaluation scripts are a separate capability. The graph reflects documented workflows and indexed source references; some internal implementations and demo wiring were not excerpted, so unsupported links are omitted.

Main entry points: [`matanyone2/inference/inference_core.py`](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/inference_core.py) (Inference orchestration), [`hugging_face/app.py`](https://github.com/pq-yang/matanyone2/blob/main/hugging_face/app.py) (Gradio demo), [`matanyone2/model/matanyone2.py`](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/matanyone2.py) (MatAnyone2 network), [`matanyone2/inference/memory_manager.py`](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/memory_manager.py) (Memory management)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User entry points"]
  node_video_mask["Video and mask"]
  node_cli["CLI<br/>[cli.py]"]
  node_script["Inference script"]
  node_api["Python API<br/>[inference_core.py]"]
end

subgraph group_demo["Interactive mask demo"]
  node_app["Gradio demo<br/>[app.py]"]
  node_interaction["Click interaction<br/>[interact_tools.py]"]
  node_mask_tools["Mask painting<br/>[mask_painter.py]"]
  node_painter["Result painter<br/>[painter.py]"]
  node_wrapper["Demo model wrapper"]
end

subgraph group_runtime["Video inference runtime"]
  node_frame_io["Frame input<br/>[inference_utils.py]"]
  node_core["Inference orchestration<br/>[inference_core.py]"]
  node_features[("Image features")]
  node_memory["Memory management<br/>[memory_manager.py]"]
  node_kv_store[("Key-value memory<br/>[kv_memory_store.py]")]
  node_objects["Object tracking<br/>[object_manager.py]"]
  node_outputs["Foreground and alpha"]
end

subgraph group_model["Matting model"]
  node_model_net["MatAnyone2 network<br/>[matanyone2.py]"]
  node_encoder["Pixel and mask encoding<br/>[big_modules.py]"]
  node_group_ops["Grouped features<br/>[group_modules.py]"]
  node_query_transformer["Object query transformer"]
  node_summarizer["Object summarizer"]
end

subgraph group_evaluation["Evaluation"]
  node_eval_suite["Evaluation scripts<br/>[eval_crgnn.py]"]
  node_eval_hr["High-resolution evaluation<br/>[eval_hr.py]"]
  node_eval_lr["Low-resolution evaluation<br/>[eval_lr.py]"]
  node_eval_inference["Inference evaluation"]
end

node_user(("Video user"))

node_user -->|"provides"| node_video_mask
node_video_mask -->|"runs with"| node_cli
node_video_mask -->|"runs with"| node_script
node_user -->|"opens"| node_app
node_cli -->|"invokes"| node_core
node_script -->|"invokes"| node_core
node_api -->|"processes video"| node_core
node_app -->|"uses"| node_interaction
node_app -->|"uses"| node_mask_tools
node_app -->|"uses"| node_painter
node_app -->|"uses"| node_wrapper
node_wrapper -->|"wraps inference"| node_core
node_interaction -->|"uses"| node_mask_tools
node_interaction -->|"uses"| node_painter
node_core -.->|"reads frames"| node_frame_io
node_core -->|"uses"| node_features
node_core -->|"manages memory"| node_memory
node_core -->|"tracks objects"| node_objects
node_core -->|"runs network"| node_model_net
node_core -->|"writes videos"| node_outputs
node_memory -->|"stores values"| node_kv_store
node_memory -->|"consults objects"| node_objects
node_memory -->|"uses network"| node_model_net
node_model_net -->|"encodes features"| node_encoder
node_model_net -->|"reads queries"| node_query_transformer
node_model_net -->|"uses summaries"| node_summarizer
node_encoder -->|"uses grouped ops"| node_group_ops
node_query_transformer -->|"uses grouped ops"| node_group_ops
node_eval_suite -.->|"includes"| node_eval_hr
node_eval_suite -.->|"includes"| node_eval_lr
node_eval_inference -->|"evaluates inference"| node_core

click node_cli "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/cli.py"
click node_script "https://github.com/pq-yang/matanyone2/blob/main/inference_matanyone2.py"
click node_api "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/inference_core.py"
click node_app "https://github.com/pq-yang/matanyone2/blob/main/hugging_face/app.py"
click node_interaction "https://github.com/pq-yang/matanyone2/blob/main/hugging_face/tools/interact_tools.py"
click node_mask_tools "https://github.com/pq-yang/matanyone2/blob/main/hugging_face/tools/mask_painter.py"
click node_painter "https://github.com/pq-yang/matanyone2/blob/main/hugging_face/tools/painter.py"
click node_wrapper "https://github.com/pq-yang/matanyone2/blob/main/hugging_face/matanyone2_wrapper.py"
click node_frame_io "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/utils/inference_utils.py"
click node_core "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/inference_core.py"
click node_features "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/image_feature_store.py"
click node_memory "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/memory_manager.py"
click node_kv_store "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/kv_memory_store.py"
click node_objects "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/object_manager.py"
click node_model_net "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/matanyone2.py"
click node_encoder "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/big_modules.py"
click node_group_ops "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/group_modules.py"
click node_query_transformer "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/transformer/object_transformer.py"
click node_summarizer "https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/transformer/object_summarizer.py"
click node_eval_suite "https://github.com/pq-yang/matanyone2/blob/main/evaluation/eval_crgnn.py"
click node_eval_hr "https://github.com/pq-yang/matanyone2/blob/main/evaluation/eval_hr.py"
click node_eval_lr "https://github.com/pq-yang/matanyone2/blob/main/evaluation/eval_lr.py"
click node_eval_inference "https://github.com/pq-yang/matanyone2/blob/main/evaluation/inference_matanyone_eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_video_mask,node_cli,node_script,node_api,node_user toneBlue
class node_app,node_interaction,node_mask_tools,node_painter,node_wrapper toneAmber
class node_frame_io,node_core,node_features,node_memory,node_kv_store,node_objects,node_outputs toneMint
class node_model_net,node_encoder,node_group_ops,node_query_transformer,node_summarizer toneRose
class node_eval_suite,node_eval_hr,node_eval_lr,node_eval_inference toneIndigo
```

## Components

### User entry points

- **Video and mask**
- [CLI](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/cli.py): `matanyone2/cli.py`
- [Inference script](https://github.com/pq-yang/matanyone2/blob/main/inference_matanyone2.py): `inference_matanyone2.py`
- [Python API](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/inference_core.py): `matanyone2/inference/inference_core.py`

### Interactive mask demo

- [Gradio demo](https://github.com/pq-yang/matanyone2/blob/main/hugging_face/app.py): `hugging_face/app.py`
- [Click interaction](https://github.com/pq-yang/matanyone2/blob/main/hugging_face/tools/interact_tools.py): `hugging_face/tools/interact_tools.py`
- [Mask painting](https://github.com/pq-yang/matanyone2/blob/main/hugging_face/tools/mask_painter.py): `hugging_face/tools/mask_painter.py`
- [Result painter](https://github.com/pq-yang/matanyone2/blob/main/hugging_face/tools/painter.py): `hugging_face/tools/painter.py`
- [Demo model wrapper](https://github.com/pq-yang/matanyone2/blob/main/hugging_face/matanyone2_wrapper.py): `hugging_face/matanyone2_wrapper.py`

### Video inference runtime

- [Frame input](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/utils/inference_utils.py): `matanyone2/utils/inference_utils.py`
- [Inference orchestration](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/inference_core.py): `matanyone2/inference/inference_core.py`
- [Image features](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/image_feature_store.py): `matanyone2/inference/image_feature_store.py`
- [Memory management](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/memory_manager.py): `matanyone2/inference/memory_manager.py`
- [Key-value memory](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/kv_memory_store.py): `matanyone2/inference/kv_memory_store.py`
- [Object tracking](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/inference/object_manager.py): `matanyone2/inference/object_manager.py`
- **Foreground and alpha**

### Matting model

- [MatAnyone2 network](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/matanyone2.py): `matanyone2/model/matanyone2.py`
- [Pixel and mask encoding](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/big_modules.py): `matanyone2/model/big_modules.py`
- [Grouped features](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/group_modules.py): `matanyone2/model/group_modules.py`
- [Object query transformer](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/transformer/object_transformer.py): `matanyone2/model/transformer/object_transformer.py`
- [Object summarizer](https://github.com/pq-yang/matanyone2/blob/main/matanyone2/model/transformer/object_summarizer.py): `matanyone2/model/transformer/object_summarizer.py`

### Evaluation

- [Evaluation scripts](https://github.com/pq-yang/matanyone2/blob/main/evaluation/eval_crgnn.py): `evaluation/eval_crgnn.py`
- [High-resolution evaluation](https://github.com/pq-yang/matanyone2/blob/main/evaluation/eval_hr.py): `evaluation/eval_hr.py`
- [Low-resolution evaluation](https://github.com/pq-yang/matanyone2/blob/main/evaluation/eval_lr.py): `evaluation/eval_lr.py`
- [Inference evaluation](https://github.com/pq-yang/matanyone2/blob/main/evaluation/inference_matanyone_eval.py): `evaluation/inference_matanyone_eval.py`

### Other components

- **Video user**

## Connections

- Video user → Video and mask: provides (evidence: `README.md`)
- Video and mask → CLI: runs with (evidence: `README.md`)
- Video and mask → Inference script: runs with (evidence: `README.md`)
- Video user → Gradio demo: opens (evidence: `README.md`)
- CLI → Inference orchestration: invokes (evidence: `matanyone2/cli.py`)
- Inference script → Inference orchestration: invokes (evidence: `inference_matanyone2.py`)
- Python API → Inference orchestration: processes video (evidence: `README.md`)
- Gradio demo → Click interaction: uses (evidence: `hugging_face/app.py`)
- Gradio demo → Mask painting: uses (evidence: `hugging_face/app.py`)
- Gradio demo → Result painter: uses (evidence: `hugging_face/app.py`)
- Gradio demo → Demo model wrapper: uses (evidence: `hugging_face/app.py`)
- Demo model wrapper → Inference orchestration: wraps inference (evidence: `hugging_face/matanyone2_wrapper.py`)
- Click interaction → Mask painting: uses (evidence: `hugging_face/tools/interact_tools.py`)
- Click interaction → Result painter: uses (evidence: `hugging_face/tools/interact_tools.py`)
- Inference orchestration → Frame input: reads frames (evidence: `matanyone2/inference/inference_core.py`)
- Inference orchestration → Image features: uses (evidence: `matanyone2/inference/inference_core.py`)
- Inference orchestration → Memory management: manages memory (evidence: `matanyone2/inference/inference_core.py`)
- Inference orchestration → Object tracking: tracks objects (evidence: `matanyone2/inference/inference_core.py`)
- Inference orchestration → MatAnyone2 network: runs network (evidence: `matanyone2/inference/inference_core.py`)
- Inference orchestration → Foreground and alpha: writes videos (evidence: `README.md`)
- Memory management → Key-value memory: stores values (evidence: `matanyone2/inference/memory_manager.py`)
- Memory management → Object tracking: consults objects (evidence: `matanyone2/inference/memory_manager.py`)
- Memory management → MatAnyone2 network: uses network (evidence: `matanyone2/inference/memory_manager.py`)
- MatAnyone2 network → Pixel and mask encoding: encodes features (evidence: `matanyone2/model/matanyone2.py`)
- MatAnyone2 network → Object query transformer: reads queries (evidence: `matanyone2/model/matanyone2.py`)
- MatAnyone2 network → Object summarizer: uses summaries (evidence: `matanyone2/model/matanyone2.py`)
- Pixel and mask encoding → Grouped features: uses grouped ops (evidence: `matanyone2/model/big_modules.py`)
- Object query transformer → Grouped features: uses grouped ops (evidence: `matanyone2/model/transformer/object_transformer.py`)
- Evaluation scripts → High-resolution evaluation: includes
- Evaluation scripts → Low-resolution evaluation: includes
- Inference evaluation → Inference orchestration: evaluates inference (evidence: `evaluation/inference_matanyone_eval.py`)
