How xinyusun/fgprompt works
FGPrompt is an image-goal navigation agent for Habitat: it trains and evaluates policies that use goal-image cues to navigate Gibson, MP3D, and HM3D scenes. The documented workflow starts with a configured training or evaluation run, then uses a navigation policy and visual encoders to select actions in the simulated environment; task measures provide reward and success signals. The graph includes the documented early- and mid-fusion variants and evaluation outputs. Several internal links rely on README or SOURCE INDEX references because their implementation excerpts are incomplete.
Main entry points: trainer.py Trainer · policy.py Navigation policy · environment.py Habitat environment · models.py Visual encoder
Components 17
Run and training
- Run entry
run.py - Trainer
src/trainer.py - Experiment configs
exp_config
Navigation task
- Navigation episodes
src/dataset.py - Habitat environment
src/environment.py - Task measures
src/metric.py - RL reward
src/environment.py
Goal perception
- Goal and view sensors
src/sensor.py - Image transforms
src/transforms.py - Visual encoder
src/models.py - ResNet backbone
src/resnet.py - Mid-fusion FiLM
src/midfusion.py
Policy and learning
- Navigation policy
src/policy.py
Evaluation outputs
- Episode visualization
src/visualization.py - Checkpoints and metrics
src/trainer.py
Other components
- Experimenter
- Habitat scene datasets
Connections 19
- Experimenter → Run entry: starts run
README.md - Experiment configs → Run entry: configures
README.md - Run entry → Trainer: dispatches train/eval
README.md - Trainer → Navigation policy: resolves policy
src/trainer.py - Trainer → Episode visualization: uses
src/trainer.py - Navigation episodes → Habitat scene datasets: uses scenes
README.md - Goal and view sensors → Habitat environment: provides observations
- Habitat environment → Task measures: reads task measures
src/environment.py - Habitat environment → RL reward: computes reward
src/environment.py - Navigation policy → Goal and view sensors: consumes observations
src/policy.py - Navigation policy → Image transforms: uses transforms
src/policy.py - Navigation policy → Visual encoder: constructs encoder
src/policy.py - Visual encoder → ResNet backbone: uses backbone
src/models.py - Visual encoder → Mid-fusion FiLM: supports fusion
src/models.py - Trainer → Habitat environment: runs episodes
- Trainer → Navigation episodes: loads episodes
- Trainer → Checkpoints and metrics: saves evaluation
README.md - Trainer → Task measures: reports measures
- Navigation policy → Habitat environment: selects actions