# prince687028/traveluav architecture

> Architecture diagram of the GitHub repository prince687028/traveluav, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/prince687028/traveluav
- Repository: https://github.com/prince687028/traveluav

## Overview

##### Main Components
- Simulation Environment:
- AirSim Simulator – represented by the airsim_plugin, which provides both server (AirVLNSimulatorServerTool.py) and client (AirVLNSimulatorClientTool.py) functionalities.
- External environments (Carla, closeloop, extra) that supply the simulation maps.
- UAV Vision-Language Model (LLaMA-UAV):
- Core model components reside in Model/LLaMA-UAV. This includes:
– The primary language model implementations (e.g., llamavid, language_model submodules).
– Multimodal components including the multimodal encoder and projector – key for combining visual and language modalities.
– Training scripts and utilities for model tuning (in llamavid/train and scripts/llm/train_uav_llm.sh).
- Auxiliary Computer Vision Modules:
- GroundingDINO integration, which can be found under src/model_wrapper/utils/GroundingDINO. This part handles object detection and vision grounding, contributing to scene analysis.
- Pipeline Orchestration and Evaluation:
- Simulation orchestration and experimental routines in src/vlnce_src (e.g., eval.py, dagger.py) and scripts (e.g., dagger_NYC.sh, eval.sh, metric.sh).
- Common utilities and configurations in src/common, utils, and configuration scripts.
- Data and Environment Setup:
- Dataset and environment download paths – as referenced in the README – for preparing both training data and simulation maps.

##### Relationships and Interactions
- The Simulation Environment module (AirSim server and external simulator maps) interacts with the Model via simulation control scripts and API clients. Data flows from the simulation (environment state or sensor data) to the model.
- The LLaMA-UAV model receives visual inputs and language instructions. Within its architecture, the language model and multimodal encoder project features into a common space, using components from both the “llamavid” package and the “GroundingDINO” module.
- Pipeline orchestration layers coordinate the data flow:
- Preprocessing scripts and utility modules convert simulation data for model consumption.
- Training scripts (and associated configurations) trigger the model’s forward and backward passes, while evaluation scripts (in src/vlnce_src and scripts) manage inference and metric calculation.
- The GroundingDINO module supports the main model by providing visual grounding information which augments the decision-making process in the UAV navigation tasks.

##### Architectural Patterns and Design Principles
- Modularity: Each major function (simulation, model training/inference, vision-processing) is organized into its own directory and module. This enables independent development and testing.
- Plug-and-Play Components: Emphasis on how external modules (like AirSim APIs and GroundingDINO) integrate with the core navigation algorithm, promoting extensibility of the platform.

##### Relevant Technologies, Frameworks, and Libraries
- PyTorch (for model training/inference as installed in the environment setup).
- Conda and pip for environment management and dependency installation.
- Docker and configuration files (e.g., Dockerfile for GroundingDINO) that support deployment.
- Shell scripts for execution, training, and evaluation.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Simulation Environment Layer
    subgraph "Simulation Environment Layer"
        A["AirSim Simulator Server"]:::simulation
        B["AirSim Simulator Client"]:::simulation
        C["External Simulation Maps\n(Carla, Closeloop, Extra)"]:::simulation
    end

    %% Pipeline Orchestration & Evaluation Layer
    subgraph "Pipeline Orchestration & Evaluation"
        D1["Orchestration & Evaluation Scripts\n(src/vlnce_src)"]:::orchestration
        D2["Shell Scripts for Orchestration\n(scripts: dagger_NYC.sh, eval.sh, metric.sh)"]:::orchestration
        E["Common Utilities & Configurations\n(src/common, utils)"]:::orchestration
    end

    %% UAV Vision-Language Model (LLaMA-UAV) Layer
    subgraph "UAV Vision-Language Model (LLaMA-UAV)"
        F["Language Model Component"]:::model
        G["Multimodal Encoder & Projector"]:::model
        H1["Training Directory"]:::model
        H2["Training Script"]:::model
    end

    %% Auxiliary Computer Vision Layer
    subgraph "Auxiliary Computer Vision"
        I["GroundingDINO Visual Grounding"]:::auxiliary
    end

    %% Data Flow Relationships
    C -->|"provides map data"| A
    A -->|"sends sensor data"| D1
    B -->|"sends client commands"| D1
    D1 -->|"triggers training"| H1
    D1 -->|"triggers training"| H2
    D1 -->|"initiates model inference"| F
    D1 -->|"initiates model inference"| G
    I -->|"supports feature extraction"| G
    F -->|"outputs decisions"| D1
    G -->|"outputs decisions"| D1

    %% Click Events
    click A "https://github.com/prince687028/traveluav/blob/main/airsim_plugin/AirVLNSimulatorServerTool.py"
    click B "https://github.com/prince687028/traveluav/blob/main/airsim_plugin/AirVLNSimulatorClientTool.py"
    click C "https://github.com/prince687028/traveluav/tree/main/src/vlnce_src"
    click F "https://github.com/prince687028/traveluav/tree/main/Model/LLaMA-UAV/llamavid/model/language_model/"
    click H1 "https://github.com/prince687028/traveluav/tree/main/Model/LLaMA-UAV/llamavid/train/"
    click H2 "https://github.com/prince687028/traveluav/blob/main/Model/LLaMA-UAV/scripts/llm/train_uav_llm.sh"
    click I "https://github.com/prince687028/traveluav/tree/main/src/model_wrapper/utils/GroundingDINO"
    click D1 "https://github.com/prince687028/traveluav/tree/main/src/vlnce_src"
    click D2 "https://github.com/prince687028/traveluav/blob/main/scripts/dagger_NYC.sh"
    click D2 "https://github.com/prince687028/traveluav/blob/main/scripts/eval.sh"
    click D2 "https://github.com/prince687028/traveluav/blob/main/scripts/metric.sh"
    click E "https://github.com/prince687028/traveluav/tree/main/src/common"
    click E "https://github.com/prince687028/traveluav/tree/main/utils"

    %% Styles
    classDef simulation fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef orchestration fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef model fill:#98FB98,stroke:#000,stroke-width:2px;
    classDef auxiliary fill:#FFA07A,stroke:#000,stroke-width:2px;
```
