# hitachinsk/fgt architecture

> Architecture diagram of the GitHub repository hitachinsk/fgt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hitachinsk/fgt
- Repository: https://github.com/hitachinsk/fgt

## Overview

This repository implements flow-guided video inpainting, with object removal, watermark removal, and video extrapolation modes. The user supplies video frames and masks; the inference tool prepares flows, completes corrupted flows with LAFC, then uses FGT to fill missing video content and saves results. Training is a separate documented two-stage workflow: train LAFC, then FGT. The graph preserves the documented flow-completion and transformer stages; source excerpts are partial, so some runtime links rely on README statements rather than inspected call sites.

Main entry points: [`tool/video_inpainting.py`](https://github.com/hitachinsk/fgt/blob/master/tool/video_inpainting.py) (Inpainting CLI), [`FGT/models/model.py`](https://github.com/hitachinsk/fgt/blob/master/FGT/models/model.py) (FGT Transformer), [`LAFC/trainer.py`](https://github.com/hitachinsk/fgt/blob/master/LAFC/trainer.py) (LAFC Trainer), [`FGT/trainer.py`](https://github.com/hitachinsk/fgt/blob/master/FGT/trainer.py) (FGT Trainer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Inference Workflow"]
  node_inference["Inpainting CLI"]
  node_inputs["Frames and Masks<br/>[inputs.py]"]
  node_output["Completed Video"]
end

subgraph group_flow["Flow Completion"]
  node_lafc["LAFC Completion<br/>[lafc.py]"]
  node_flowdata["Flow Dataset"]
  node_flowloss["Flow Losses<br/>[flow_losses.py]"]
  node_regionfill["Flow Region Fill<br/>[region_fill.py]"]
end

subgraph group_fgt["FGT Inpainting"]
  node_fgtmodel["FGT Transformer<br/>[model.py]"]
  node_attention["Flow-Guided Attention<br/>[attention_flow.py]"]
  node_temporal["Temporal Attention<br/>[attention_base.py]"]
  node_feedforward["Transformer Feedforward<br/>[ffn_base.py]"]
end

subgraph group_training["Training and Evaluation"]
  node_fgtdata["Video Training Data<br/>[train_dataset.py]"]
  node_fgttrainer["FGT Trainer<br/>[trainer.py]"]
  node_fgttrainnet["FGT Training Network<br/>[network.py]"]
  node_flowtrainer["LAFC Trainer<br/>[trainer.py]"]
  node_flowtrainnet["LAFC Training Network<br/>[network.py]"]
  node_metrics["Image Metrics<br/>[ssim.py]"]
end

subgraph group_raft["Optical Flow"]
  node_raft["RAFT Estimator<br/>[raft.py]"]
  node_flowextract["Flow Extraction Tool<br/>[flow_extract.py]"]
end

node_user(("Video User"))

node_user -->|"invokes"| node_inference
node_inference -->|"loads inputs"| node_inputs
node_inference -.->|"estimates flow"| node_raft
node_flowextract -.->|"runs estimator"| node_raft
node_inference -->|"completes flow"| node_lafc
node_inference -->|"inpaints video"| node_fgtmodel
node_lafc -->|"fills flow regions"| node_regionfill
node_fgtmodel -->|"uses flow attention"| node_attention
node_fgtmodel -->|"uses temporal attention"| node_temporal
node_fgtmodel -->|"uses feedforward"| node_feedforward
node_inference -->|"saves results"| node_output
node_flowtrainer -->|"trains from"| node_flowdata
node_flowtrainnet -->|"extends trainer"| node_flowtrainer
node_fgttrainer -->|"loads batches"| node_fgtdata
node_fgttrainnet -->|"extends trainer"| node_fgttrainer
node_fgttrainnet -->|"evaluates with"| node_metrics
node_user -->|"prepares training flow"| node_flowextract
node_flowtrainer -->|"precedes stage"| node_fgttrainer

click node_inference "https://github.com/hitachinsk/fgt/blob/master/tool/video_inpainting.py"
click node_inputs "https://github.com/hitachinsk/fgt/blob/master/FGT/inputs.py"
click node_output "https://github.com/hitachinsk/fgt/blob/master/tool/video_inpainting.py"
click node_lafc "https://github.com/hitachinsk/fgt/blob/master/LAFC/models/lafc.py"
click node_flowdata "https://github.com/hitachinsk/fgt/blob/master/LAFC/data/train_dataset_edge.py"
click node_flowloss "https://github.com/hitachinsk/fgt/blob/master/LAFC/models/utils/flow_losses.py"
click node_regionfill "https://github.com/hitachinsk/fgt/blob/master/FGT/data/util/flow_utils/region_fill.py"
click node_fgtmodel "https://github.com/hitachinsk/fgt/blob/master/FGT/models/model.py"
click node_attention "https://github.com/hitachinsk/fgt/blob/master/FGT/models/transformer_base/attention_flow.py"
click node_temporal "https://github.com/hitachinsk/fgt/blob/master/FGT/models/transformer_base/attention_base.py"
click node_feedforward "https://github.com/hitachinsk/fgt/blob/master/FGT/models/transformer_base/ffn_base.py"
click node_fgtdata "https://github.com/hitachinsk/fgt/blob/master/FGT/data/train_dataset.py"
click node_fgttrainer "https://github.com/hitachinsk/fgt/blob/master/FGT/trainer.py"
click node_fgttrainnet "https://github.com/hitachinsk/fgt/blob/master/FGT/networks/network.py"
click node_flowtrainer "https://github.com/hitachinsk/fgt/blob/master/LAFC/trainer.py"
click node_flowtrainnet "https://github.com/hitachinsk/fgt/blob/master/LAFC/networks/network.py"
click node_metrics "https://github.com/hitachinsk/fgt/blob/master/FGT/metrics/ssim.py"
click node_raft "https://github.com/hitachinsk/fgt/blob/master/RAFT/raft.py"
click node_flowextract "https://github.com/hitachinsk/fgt/blob/master/tool/flow_extract.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_inference,node_inputs,node_output,node_user toneBlue
class node_lafc,node_flowdata,node_flowloss,node_regionfill toneAmber
class node_fgtmodel,node_attention,node_temporal,node_feedforward toneMint
class node_fgtdata,node_fgttrainer,node_fgttrainnet,node_flowtrainer,node_flowtrainnet,node_metrics toneRose
class node_raft,node_flowextract toneIndigo
```

## Components

### Inference Workflow

- [Inpainting CLI](https://github.com/hitachinsk/fgt/blob/master/tool/video_inpainting.py): `tool/video_inpainting.py`
- [Frames and Masks](https://github.com/hitachinsk/fgt/blob/master/FGT/inputs.py): `FGT/inputs.py`
- [Completed Video](https://github.com/hitachinsk/fgt/blob/master/tool/video_inpainting.py): `tool/video_inpainting.py`

### Flow Completion

- [LAFC Completion](https://github.com/hitachinsk/fgt/blob/master/LAFC/models/lafc.py): `LAFC/models/lafc.py`
- [Flow Dataset](https://github.com/hitachinsk/fgt/blob/master/LAFC/data/train_dataset_edge.py): `LAFC/data/train_dataset_edge.py`
- [Flow Losses](https://github.com/hitachinsk/fgt/blob/master/LAFC/models/utils/flow_losses.py): `LAFC/models/utils/flow_losses.py`
- [Flow Region Fill](https://github.com/hitachinsk/fgt/blob/master/FGT/data/util/flow_utils/region_fill.py): `FGT/data/util/flow_utils/region_fill.py`

### FGT Inpainting

- [FGT Transformer](https://github.com/hitachinsk/fgt/blob/master/FGT/models/model.py): `FGT/models/model.py`
- [Flow-Guided Attention](https://github.com/hitachinsk/fgt/blob/master/FGT/models/transformer_base/attention_flow.py): `FGT/models/transformer_base/attention_flow.py`
- [Temporal Attention](https://github.com/hitachinsk/fgt/blob/master/FGT/models/transformer_base/attention_base.py): `FGT/models/transformer_base/attention_base.py`
- [Transformer Feedforward](https://github.com/hitachinsk/fgt/blob/master/FGT/models/transformer_base/ffn_base.py): `FGT/models/transformer_base/ffn_base.py`

### Training and Evaluation

- [Video Training Data](https://github.com/hitachinsk/fgt/blob/master/FGT/data/train_dataset.py): `FGT/data/train_dataset.py`
- [FGT Trainer](https://github.com/hitachinsk/fgt/blob/master/FGT/trainer.py): `FGT/trainer.py`
- [FGT Training Network](https://github.com/hitachinsk/fgt/blob/master/FGT/networks/network.py): `FGT/networks/network.py`
- [LAFC Trainer](https://github.com/hitachinsk/fgt/blob/master/LAFC/trainer.py): `LAFC/trainer.py`
- [LAFC Training Network](https://github.com/hitachinsk/fgt/blob/master/LAFC/networks/network.py): `LAFC/networks/network.py`
- [Image Metrics](https://github.com/hitachinsk/fgt/blob/master/FGT/metrics/ssim.py): `FGT/metrics/ssim.py`

### Optical Flow

- [RAFT Estimator](https://github.com/hitachinsk/fgt/blob/master/RAFT/raft.py): `RAFT/raft.py`
- [Flow Extraction Tool](https://github.com/hitachinsk/fgt/blob/master/tool/flow_extract.py): `tool/flow_extract.py`

### Other components

- **Video User**

## Connections

- Video User → Inpainting CLI: invokes (evidence: `README.md`)
- Inpainting CLI → Frames and Masks: loads inputs (evidence: `README.md`)
- Inpainting CLI → RAFT Estimator: estimates flow
- Flow Extraction Tool → RAFT Estimator: runs estimator
- Inpainting CLI → LAFC Completion: completes flow (evidence: `README.md`)
- Inpainting CLI → FGT Transformer: inpaints video (evidence: `README.md`)
- LAFC Completion → Flow Region Fill: fills flow regions
- FGT Transformer → Flow-Guided Attention: uses flow attention (evidence: `FGT/models/model.py`)
- FGT Transformer → Temporal Attention: uses temporal attention (evidence: `FGT/models/model.py`)
- FGT Transformer → Transformer Feedforward: uses feedforward (evidence: `FGT/models/model.py`)
- Inpainting CLI → Completed Video: saves results (evidence: `README.md`)
- LAFC Trainer → Flow Dataset: trains from (evidence: `README.md`)
- LAFC Training Network → LAFC Trainer: extends trainer (evidence: `LAFC/networks/network.py`)
- FGT Trainer → Video Training Data: loads batches (evidence: `FGT/trainer.py`)
- FGT Training Network → FGT Trainer: extends trainer (evidence: `FGT/networks/network.py`)
- FGT Training Network → Image Metrics: evaluates with (evidence: `FGT/networks/network.py`)
- Video User → Flow Extraction Tool: prepares training flow (evidence: `README.md`)
- LAFC Trainer → FGT Trainer: precedes stage (evidence: `README.md`)
