How internrobotics/internnav works
Project Type and Purpose
• InternNav is an all-in-one open-source toolbox for embodied navigation. It provides a modular framework—built on PyTorch, Habitat, and Isaac Sim—for training, evaluating, and deploying vision-language and visual navigation agents.
Top-Level Components (from file structure & README)
a. CLI / Scripting Layer
• scripts/train/train.py + configs → Training entrypoint
• scripts/eval/eval.py + configs → Evaluation entrypoint
• challenge/health_check.sh, start_eval_iros.sh → Challenge evaluation drivers
• scripts/realworld/http_internvla_{client,server}.py → Real-world HTTP service
b. Core Library (internnav/)
• configs/ → Hydra-style configuration modules (agent, trainer, model, evaluator)
• dataset/ → Data loaders & adapters (LMDB, custom dataset formats)
• env/ → Environment interfaces (VLN-multi, VLN-PE)
• habitat_extensions/ → Extensions to Habitat for custom measures, maps, fonts
• agent/ → High-level agent orchestrators (cma_agent, rdp_agent, seq2seq_agent)
• model/ →
– basemodel/ → Policy implementations for different algorithm backbones (CMA, RDP, Seq2Seq, diffusion-based)
– encoder/ → Vision & language encoders (CLIP, BERT, ResNet, depth models, diffusion policy encoders)
– utils/ → Shared utilities across models
• trainer/ → Trainer classes (cma_trainer, navdp_trainer, rdp_trainer, etc.)
• evaluator/ → Evaluator classes & utilities (vln_multi_evaluator, vln_pe_evaluator, metrics collection)
• utils/ → Logging, checkpoint I/O, tensorboard, distributed utils, visualization
c. Data Conversion & Miscellaneous
• scripts/dataset_converters/ → Convert between VLN-CE, LeRobot formats
• internnav/dist.py → Distributed training helpers
d. Tests
• tests/ → Unit & end-to-end functional tests
External Dependencies & Services
• PyTorch: model implementation, training loops
• Habitat & Isaac Sim: simulators for virtual environments
• LMDB: high-throughput dataset storage
• Gradio (via backend script): web UI for evaluation demo
• HTTP server/client: real-world deployment interface