# nvlabs/eagle architecture

> Architecture diagram of the GitHub repository nvlabs/eagle, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/eagle
- Repository: https://github.com/nvlabs/eagle

## Overview

Eagle is a family of vision-language models spanning general image understanding (Eagle), long-context image/video understanding (Eagle 2.5), and grounding, detection, and pointing (LocateAnything). The user workflow is to submit an image or video with a prompt through a demo or inference entry point, have the model process visual and language inputs, and receive generated text or localized results; evaluation scripts assess model outputs on benchmark tasks. The graph includes principal model, training, and evaluation branches. Some subsystem paths and interactions are inferred only from the tree and README, so uncertain wiring is omitted.

Main entry points: [`Eagle2_5/eaglevl/model/eagle2_5/modeling_eagle2_5_vl.py`](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/eaglevl/model/eagle2_5/modeling_eagle2_5_vl.py) (Eagle 2.5 VLM), [`Embodied/eaglevl/model/locany/modeling_locateanything.py`](https://github.com/nvlabs/eagle/blob/main/Embodied/eaglevl/model/locany/modeling_locateanything.py) (LocateAnything Model), [`Eagle2_5/streamlit_demo/app.py`](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/app.py) (Eagle 2.5 Demo), [`Eagle2_5/streamlit_demo/model_worker.py`](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/model_worker.py) (Model Worker)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User Interfaces"]
  node_eagle2_demo["Eagle 2.5 Demo<br/>[app.py]"]
  node_controller["Demo Controller<br/>[controller.py]"]
  node_model_worker["Model Worker<br/>[model_worker.py]"]
  node_locany_worker["LocateAnything Worker"]
end

subgraph group_runtime["Model Runtime"]
  node_eagle_legacy["Eagle VLM<br/>[eagle_arch.py]"]
  node_eagle2_model["Eagle 2.5 VLM"]
  node_vision_encoders["Vision Encoders"]
  node_language_models["Language Models"]
  node_locany_model["LocateAnything Model"]
end

subgraph group_training["Training"]
  node_eagle2_training["Eagle 2.5 Training"]
  node_embodied_training["LocateAnything Training"]
end

subgraph group_evaluation["Evaluation"]
  node_lmms_eval["Multimodal Evaluator<br/>[evaluator.py]"]
  node_task_metrics["Benchmark Tasks"]
  node_grounding_eval["Grounding Evaluation"]
  node_detection_eval["Detection Evaluation"]
  node_sspro_eval["ScreenSpot Evaluation"]
  node_eval_metrics["Evaluation Metrics"]
end

node_user(("User"))
node_image_video(("Image / Video Input"))
node_training_data(("Training Data"))

node_user -->|"submits prompt"| node_eagle2_demo
node_image_video -->|"uploads media"| node_eagle2_demo
node_eagle2_demo -->|"requests model"| node_controller
node_eagle2_demo -->|"streams generation"| node_model_worker
node_controller -->|"dispatches request"| node_model_worker
node_model_worker -->|"generates response"| node_eagle2_model
node_eagle2_model -->|"encodes visuals"| node_vision_encoders
node_eagle2_model -->|"processes language"| node_language_models
node_user -->|"submits query"| node_locany_worker
node_image_video -->|"provides image"| node_locany_worker
node_locany_worker -->|"generates grounding"| node_locany_model
node_training_data -->|"provides examples"| node_eagle2_training
node_eagle2_training -->|"fine-tunes"| node_eagle2_model
node_training_data -->|"provides examples"| node_embodied_training
node_embodied_training -->|"fine-tunes"| node_locany_model
node_eagle_legacy -.->|"uses encoders"| node_vision_encoders
node_eagle2_model -->|"supplies model"| node_lmms_eval
node_lmms_eval -->|"evaluates tasks"| node_task_metrics
node_locany_model -->|"supplies predictions"| node_grounding_eval
node_locany_model -->|"supplies predictions"| node_detection_eval
node_locany_model -->|"supplies predictions"| node_sspro_eval
node_grounding_eval -->|"scores results"| node_eval_metrics
node_detection_eval -->|"scores results"| node_eval_metrics
node_sspro_eval -->|"scores results"| node_eval_metrics

click node_eagle2_demo "https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/app.py"
click node_controller "https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/controller.py"
click node_model_worker "https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/model_worker.py"
click node_locany_worker "https://github.com/nvlabs/eagle/blob/main/Embodied/locateanything_worker.py"
click node_eagle_legacy "https://github.com/nvlabs/eagle/blob/main/Eagle/eagle/model/eagle_arch.py"
click node_eagle2_model "https://github.com/nvlabs/eagle/blob/main/Eagle2_5/eaglevl/model/eagle2_5/modeling_eagle2_5_vl.py"
click node_vision_encoders "https://github.com/nvlabs/eagle/tree/main/Eagle2_5/eaglevl/model/multimodal_encoder"
click node_language_models "https://github.com/nvlabs/eagle/tree/main/Eagle2_5/eaglevl/model/llama"
click node_locany_model "https://github.com/nvlabs/eagle/blob/main/Embodied/eaglevl/model/locany/modeling_locateanything.py"
click node_eagle2_training "https://github.com/nvlabs/eagle/blob/main/Eagle2_5/eaglevl/train/eagle_2_5_vl_finetune.py"
click node_embodied_training "https://github.com/nvlabs/eagle/blob/main/Embodied/eaglevl/train/locany_finetune_magi_stream.py"
click node_lmms_eval "https://github.com/nvlabs/eagle/blob/main/Eagle/lmms_eval/evaluator.py"
click node_task_metrics "https://github.com/nvlabs/eagle/tree/main/Eagle/lmms_eval/tasks"
click node_grounding_eval "https://github.com/nvlabs/eagle/blob/main/Embodied/evaluation/inference_grounding_ddp.py"
click node_detection_eval "https://github.com/nvlabs/eagle/blob/main/Embodied/evaluation/inference_detection_ddp.py"
click node_sspro_eval "https://github.com/nvlabs/eagle/blob/main/Embodied/evaluation/inference_sspro_ddp.py"
click node_eval_metrics "https://github.com/nvlabs/eagle/tree/main/Embodied/evaluation/metrics"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_eagle2_demo,node_controller,node_model_worker,node_locany_worker,node_user toneBlue
class node_eagle_legacy,node_eagle2_model,node_vision_encoders,node_language_models,node_locany_model toneAmber
class node_eagle2_training,node_embodied_training toneMint
class node_lmms_eval,node_task_metrics,node_grounding_eval,node_detection_eval,node_sspro_eval,node_eval_metrics toneRose
class node_image_video,node_training_data toneIndigo
```

## Components

### User Interfaces

- [Eagle 2.5 Demo](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/app.py): `Eagle2_5/streamlit_demo/app.py`
- [Demo Controller](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/controller.py): `Eagle2_5/streamlit_demo/controller.py`
- [Model Worker](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/streamlit_demo/model_worker.py): `Eagle2_5/streamlit_demo/model_worker.py`
- [LocateAnything Worker](https://github.com/nvlabs/eagle/blob/main/Embodied/locateanything_worker.py): `Embodied/locateanything_worker.py`

### Model Runtime

- [Eagle VLM](https://github.com/nvlabs/eagle/blob/main/Eagle/eagle/model/eagle_arch.py): `Eagle/eagle/model/eagle_arch.py`
- [Eagle 2.5 VLM](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/eaglevl/model/eagle2_5/modeling_eagle2_5_vl.py): `Eagle2_5/eaglevl/model/eagle2_5/modeling_eagle2_5_vl.py`
- [Vision Encoders](https://github.com/nvlabs/eagle/tree/main/Eagle2_5/eaglevl/model/multimodal_encoder): `Eagle2_5/eaglevl/model/multimodal_encoder`
- [Language Models](https://github.com/nvlabs/eagle/tree/main/Eagle2_5/eaglevl/model/llama): `Eagle2_5/eaglevl/model/llama`
- [LocateAnything Model](https://github.com/nvlabs/eagle/blob/main/Embodied/eaglevl/model/locany/modeling_locateanything.py): `Embodied/eaglevl/model/locany/modeling_locateanything.py`

### Training

- [Eagle 2.5 Training](https://github.com/nvlabs/eagle/blob/main/Eagle2_5/eaglevl/train/eagle_2_5_vl_finetune.py): `Eagle2_5/eaglevl/train/eagle_2_5_vl_finetune.py`
- [LocateAnything Training](https://github.com/nvlabs/eagle/blob/main/Embodied/eaglevl/train/locany_finetune_magi_stream.py): `Embodied/eaglevl/train/locany_finetune_magi_stream.py`

### Evaluation

- [Multimodal Evaluator](https://github.com/nvlabs/eagle/blob/main/Eagle/lmms_eval/evaluator.py): `Eagle/lmms_eval/evaluator.py`
- [Benchmark Tasks](https://github.com/nvlabs/eagle/tree/main/Eagle/lmms_eval/tasks): `Eagle/lmms_eval/tasks`
- [Grounding Evaluation](https://github.com/nvlabs/eagle/blob/main/Embodied/evaluation/inference_grounding_ddp.py): `Embodied/evaluation/inference_grounding_ddp.py`
- [Detection Evaluation](https://github.com/nvlabs/eagle/blob/main/Embodied/evaluation/inference_detection_ddp.py): `Embodied/evaluation/inference_detection_ddp.py`
- [ScreenSpot Evaluation](https://github.com/nvlabs/eagle/blob/main/Embodied/evaluation/inference_sspro_ddp.py): `Embodied/evaluation/inference_sspro_ddp.py`
- [Evaluation Metrics](https://github.com/nvlabs/eagle/tree/main/Embodied/evaluation/metrics): `Embodied/evaluation/metrics`

### Other components

- **User**
- **Image / Video Input**
- **Training Data**

## Connections

- User → Eagle 2.5 Demo: submits prompt
- Image / Video Input → Eagle 2.5 Demo: uploads media
- Eagle 2.5 Demo → Demo Controller: requests model
- Eagle 2.5 Demo → Model Worker: streams generation
- Demo Controller → Model Worker: dispatches request
- Model Worker → Eagle 2.5 VLM: generates response
- Eagle 2.5 VLM → Vision Encoders: encodes visuals
- Eagle 2.5 VLM → Language Models: processes language
- User → LocateAnything Worker: submits query
- Image / Video Input → LocateAnything Worker: provides image
- LocateAnything Worker → LocateAnything Model: generates grounding
- Training Data → Eagle 2.5 Training: provides examples
- Eagle 2.5 Training → Eagle 2.5 VLM: fine-tunes
- Training Data → LocateAnything Training: provides examples
- LocateAnything Training → LocateAnything Model: fine-tunes
- Eagle VLM → Vision Encoders: uses encoders
- Eagle 2.5 VLM → Multimodal Evaluator: supplies model
- Multimodal Evaluator → Benchmark Tasks: evaluates tasks
- LocateAnything Model → Grounding Evaluation: supplies predictions
- LocateAnything Model → Detection Evaluation: supplies predictions
- LocateAnything Model → ScreenSpot Evaluation: supplies predictions
- Grounding Evaluation → Evaluation Metrics: scores results
- Detection Evaluation → Evaluation Metrics: scores results
- ScreenSpot Evaluation → Evaluation Metrics: scores results