# xinyusun/fgprompt architecture

> Architecture diagram of the GitHub repository xinyusun/fgprompt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xinyusun/fgprompt
- Repository: https://github.com/xinyusun/fgprompt

## Overview

FGPrompt is an image-goal navigation agent for Habitat: it trains and evaluates policies that use goal-image cues to navigate Gibson, MP3D, and HM3D scenes. The documented workflow starts with a configured training or evaluation run, then uses a navigation policy and visual encoders to select actions in the simulated environment; task measures provide reward and success signals. The graph includes the documented early- and mid-fusion variants and evaluation outputs. Several internal links rely on README or SOURCE INDEX references because their implementation excerpts are incomplete.

Main entry points: [`src/trainer.py`](https://github.com/xinyusun/fgprompt/blob/main/src/trainer.py) (Trainer), [`src/policy.py`](https://github.com/xinyusun/fgprompt/blob/main/src/policy.py) (Navigation policy), [`src/environment.py`](https://github.com/xinyusun/fgprompt/blob/main/src/environment.py) (Habitat environment), [`src/models.py`](https://github.com/xinyusun/fgprompt/blob/main/src/models.py) (Visual encoder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Run and training"]
  node_entry["Run entry<br/>[run.py]"]
  node_trainer["Trainer<br/>[trainer.py]"]
  node_configs["Experiment configs"]
end

subgraph group_task["Navigation task"]
  node_datasets["Navigation episodes<br/>[dataset.py]"]
  node_environment["Habitat environment<br/>[environment.py]"]
  node_metrics["Task measures<br/>[metric.py]"]
  node_reward["RL reward<br/>[environment.py]"]
end

subgraph group_perception["Goal perception"]
  node_sensors["Goal and view sensors<br/>[sensor.py]"]
  node_transforms["Image transforms<br/>[transforms.py]"]
  node_encoder["Visual encoder<br/>[models.py]"]
  node_resnet["ResNet backbone<br/>[resnet.py]"]
  node_midfusion["Mid-fusion FiLM<br/>[midfusion.py]"]
end

subgraph group_learning["Policy and learning"]
  node_policy["Navigation policy<br/>[policy.py]"]
end

subgraph group_outputs["Evaluation outputs"]
  node_visualization["Episode visualization<br/>[visualization.py]"]
  node_results["Checkpoints and metrics<br/>[trainer.py]"]
end

node_operator(("Experimenter"))
node_scene_data[("Habitat scene datasets")]

node_operator -->|"starts run"| node_entry
node_configs -->|"configures"| node_entry
node_entry -->|"dispatches train/eval"| node_trainer
node_trainer -->|"resolves policy"| node_policy
node_trainer -->|"uses"| node_visualization
node_datasets -.->|"uses scenes"| node_scene_data
node_sensors -.->|"provides observations"| node_environment
node_environment -->|"reads task measures"| node_metrics
node_environment -->|"computes reward"| node_reward
node_policy -->|"consumes observations"| node_sensors
node_policy -->|"uses transforms"| node_transforms
node_policy -->|"constructs encoder"| node_encoder
node_encoder -->|"uses backbone"| node_resnet
node_encoder -->|"supports fusion"| node_midfusion
node_trainer -.->|"runs episodes"| node_environment
node_trainer -.->|"loads episodes"| node_datasets
node_trainer -->|"saves evaluation"| node_results
node_trainer -.->|"reports measures"| node_metrics
node_policy -.->|"selects actions"| node_environment

click node_entry "https://github.com/xinyusun/fgprompt/blob/main/run.py"
click node_trainer "https://github.com/xinyusun/fgprompt/blob/main/src/trainer.py"
click node_configs "https://github.com/xinyusun/fgprompt/tree/main/exp_config"
click node_datasets "https://github.com/xinyusun/fgprompt/blob/main/src/dataset.py"
click node_environment "https://github.com/xinyusun/fgprompt/blob/main/src/environment.py"
click node_metrics "https://github.com/xinyusun/fgprompt/blob/main/src/metric.py"
click node_sensors "https://github.com/xinyusun/fgprompt/blob/main/src/sensor.py"
click node_transforms "https://github.com/xinyusun/fgprompt/blob/main/src/transforms.py"
click node_encoder "https://github.com/xinyusun/fgprompt/blob/main/src/models.py"
click node_resnet "https://github.com/xinyusun/fgprompt/blob/main/src/resnet.py"
click node_midfusion "https://github.com/xinyusun/fgprompt/blob/main/src/midfusion.py"
click node_policy "https://github.com/xinyusun/fgprompt/blob/main/src/policy.py"
click node_reward "https://github.com/xinyusun/fgprompt/blob/main/src/environment.py"
click node_visualization "https://github.com/xinyusun/fgprompt/blob/main/src/visualization.py"
click node_results "https://github.com/xinyusun/fgprompt/blob/main/src/trainer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_trainer,node_configs toneBlue
class node_datasets,node_environment,node_metrics,node_reward,node_scene_data toneAmber
class node_sensors,node_transforms,node_encoder,node_resnet,node_midfusion toneMint
class node_policy toneRose
class node_visualization,node_results,node_operator toneIndigo
```

## Components

### Run and training

- [Run entry](https://github.com/xinyusun/fgprompt/blob/main/run.py): `run.py`
- [Trainer](https://github.com/xinyusun/fgprompt/blob/main/src/trainer.py): `src/trainer.py`
- [Experiment configs](https://github.com/xinyusun/fgprompt/tree/main/exp_config): `exp_config`

### Navigation task

- [Navigation episodes](https://github.com/xinyusun/fgprompt/blob/main/src/dataset.py): `src/dataset.py`
- [Habitat environment](https://github.com/xinyusun/fgprompt/blob/main/src/environment.py): `src/environment.py`
- [Task measures](https://github.com/xinyusun/fgprompt/blob/main/src/metric.py): `src/metric.py`
- [RL reward](https://github.com/xinyusun/fgprompt/blob/main/src/environment.py): `src/environment.py`

### Goal perception

- [Goal and view sensors](https://github.com/xinyusun/fgprompt/blob/main/src/sensor.py): `src/sensor.py`
- [Image transforms](https://github.com/xinyusun/fgprompt/blob/main/src/transforms.py): `src/transforms.py`
- [Visual encoder](https://github.com/xinyusun/fgprompt/blob/main/src/models.py): `src/models.py`
- [ResNet backbone](https://github.com/xinyusun/fgprompt/blob/main/src/resnet.py): `src/resnet.py`
- [Mid-fusion FiLM](https://github.com/xinyusun/fgprompt/blob/main/src/midfusion.py): `src/midfusion.py`

### Policy and learning

- [Navigation policy](https://github.com/xinyusun/fgprompt/blob/main/src/policy.py): `src/policy.py`

### Evaluation outputs

- [Episode visualization](https://github.com/xinyusun/fgprompt/blob/main/src/visualization.py): `src/visualization.py`
- [Checkpoints and metrics](https://github.com/xinyusun/fgprompt/blob/main/src/trainer.py): `src/trainer.py`

### Other components

- **Experimenter**
- **Habitat scene datasets**

## Connections

- Experimenter → Run entry: starts run (evidence: `README.md`)
- Experiment configs → Run entry: configures (evidence: `README.md`)
- Run entry → Trainer: dispatches train/eval (evidence: `README.md`)
- Trainer → Navigation policy: resolves policy (evidence: `src/trainer.py`)
- Trainer → Episode visualization: uses (evidence: `src/trainer.py`)
- Navigation episodes → Habitat scene datasets: uses scenes (evidence: `README.md`)
- Goal and view sensors → Habitat environment: provides observations
- Habitat environment → Task measures: reads task measures (evidence: `src/environment.py`)
- Habitat environment → RL reward: computes reward (evidence: `src/environment.py`)
- Navigation policy → Goal and view sensors: consumes observations (evidence: `src/policy.py`)
- Navigation policy → Image transforms: uses transforms (evidence: `src/policy.py`)
- Navigation policy → Visual encoder: constructs encoder (evidence: `src/policy.py`)
- Visual encoder → ResNet backbone: uses backbone (evidence: `src/models.py`)
- Visual encoder → Mid-fusion FiLM: supports fusion (evidence: `src/models.py`)
- Trainer → Habitat environment: runs episodes
- Trainer → Navigation episodes: loads episodes
- Trainer → Checkpoints and metrics: saves evaluation (evidence: `README.md`)
- Trainer → Task measures: reports measures
- Navigation policy → Habitat environment: selects actions
