How dipin-adhikari/see2seek works
See2Seek trains and evaluates a recurrent agent for zero-shot RoboTHOR navigation: visual observations and image or text goals feed a multimodal policy, which selects actions in simulated episodes; evaluation reports navigation performance. Supporting capabilities include episodic spatial memory, goal-image/embedding preparation, and training-curve plotting. The graph uses README statements and SOURCE INDEX references for relationships; unsampled implementations are represented only where the tree or documentation supports them.
Main entry points: ppo_trainer.py PPO trainer · evaluator.py Evaluator · clip_encoder.py CLIP encoder · generate_data.py Goal preparation
Components 17
Goal data
- Goal preparation
see2seek/utils/generate_data.py - Angle augmentation
see2seek/utils/augment_goal_angles.py - Goal cache
see2seek/utils/goal_cache.py
Navigation runtime
- Configuration
see2seek/utils/config.py - RoboTHOR environment
see2seek/envs/robothor_env.py - Vector environments
see2seek/envs/vec_env.py
Policy learning
- DINOv2 encoder
see2seek/models/encoders/dino_encoder.py - CLIP encoder
see2seek/models/encoders/clip_encoder.py - GRU policy
see2seek/agents/gru_policy.py - Episodic memory
see2seek/agents/gru_policy.py - Rollout buffer
see2seek/buffers/rollout_buffer.py - PPO trainer
see2seek/trainers/ppo_trainer.py
Evaluation
- Navigation metrics
see2seek/evaluation/metrics.py - Evaluator
see2seek/evaluation/evaluator.py - Training plots
scripts/plot_training.py
Other components
- Experimenter
- AI2-THOR
Connections 24
- Experimenter → Goal preparation: prepares goals
README.md - Goal preparation → CLIP encoder: encodes goals
see2seek/utils/generate_data.py - Goal preparation → Goal cache: uses cache
see2seek/utils/generate_data.py - Experimenter → Angle augmentation: augments views
README.md - Angle augmentation → CLIP encoder: encodes views
see2seek/utils/augment_goal_angles.py - Experimenter → PPO trainer: trains agent
README.md - PPO trainer → Configuration: loads settings
see2seek/trainers/ppo_trainer.py - PPO trainer → Vector environments: collects rollouts
see2seek/trainers/ppo_trainer.py - PPO trainer → DINOv2 encoder: encodes observations
see2seek/trainers/ppo_trainer.py - PPO trainer → CLIP encoder: encodes goals
see2seek/trainers/ppo_trainer.py - PPO trainer → GRU policy: optimizes policy
see2seek/trainers/ppo_trainer.py - PPO trainer → Rollout buffer: stores experience
see2seek/trainers/ppo_trainer.py - GRU policy → Episodic memory: uses memory
README.md - Vector environments → RoboTHOR environment: runs episodes
- RoboTHOR environment → AI2-THOR: controls scenes
see2seek/envs/robothor_env.py - Experimenter → Evaluator: evaluates checkpoint
README.md - Evaluator → Configuration: loads settings
see2seek/evaluation/evaluator.py - Evaluator → Vector environments: runs evaluations
see2seek/evaluation/evaluator.py - Evaluator → RoboTHOR environment: uses episodes
see2seek/evaluation/evaluator.py - Evaluator → DINOv2 encoder: encodes observations
see2seek/evaluation/evaluator.py - Evaluator → CLIP encoder: encodes goals
see2seek/evaluation/evaluator.py - Evaluator → GRU policy: runs policy
see2seek/evaluation/evaluator.py - Experimenter → Training plots: plots training
README.md - Training plots → Configuration: reads settings
scripts/plot_training.py