# curryqka/agentthink architecture

> Architecture diagram of the GitHub repository curryqka/agentthink, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/curryqka/agentthink
- Repository: https://github.com/curryqka/agentthink
- Video tour (about a minute): https://gitdiagram.com/curryqka/agentthink/video

## Overview

AgentThink is a tool-augmented vision-language reasoning framework for autonomous-driving scenarios. Its main workflow prepares driving examples and tool-augmented reasoning data, runs model inference with optional perception tools, and evaluates predictions. The repository also includes demo inference paths and tool-result preparation. The README documents the major capabilities; relationships between sampled modules are shown only where the excerpts, source index, or README support them. Several evaluation and tool implementations are unsampled, so their internal links are omitted.

Main entry points: [`scripts/tools/tool_libraries.py`](https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_libraries.py) (Driving tool library), [`scripts/tools/agentthink_data_generater_pipeline.py`](https://github.com/curryqka/agentthink/blob/main/scripts/tools/agentthink_data_generater_pipeline.py) (CoT data generation), [`evaluation/inference_withtool.py`](https://github.com/curryqka/agentthink/blob/main/evaluation/inference_withtool.py) (Tool-augmented inference), [`scripts/tools/gen_tool_result_data.py`](https://github.com/curryqka/agentthink/blob/main/scripts/tools/gen_tool_result_data.py) (Tool-result assembly)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_prepare_data["Tool-result assembly"]
  node_image_convert["Multi-view image conversion"]
  node_cot_generate["CoT data generation"]
end

subgraph group_reasoning["Reasoning and tools"]
  node_tool_lib["Driving tool library<br/>[tool_libraries.py]"]
  node_simple_tool_lib["Visual tool library"]
  node_tool_prompts["Tool prompts<br/>[tool_prompts.py]"]
  node_geometry["BEV geometry<br/>[geometry.py]"]
  node_box_distance["Box distances<br/>[box_distance.py]"]
end

subgraph group_inference["Inference"]
  node_tool_inference["Tool-augmented inference"]
  node_agentthink_inference["AgentThink inference"]
  node_demo_drivelmm["DriveLMM demo"]
  node_demo_drivemllm["DriveMLLM demo"]
end

subgraph group_evaluation["Evaluation"]
  node_scoring["LLM-as-judge scoring"]
  node_evaluation_inference["Baseline inference<br/>[inference.py]"]
end

node_researcher(("Researcher"))
node_driving_data["Driving data"]
node_openai{{"OpenAI API"}}
node_tool_models{{"Vision tool models"}}
node_results["Prediction results"]

node_researcher -->|"generates data"| node_cot_generate
node_driving_data -->|"provides samples"| node_cot_generate
node_cot_generate -->|"requests completions"| node_openai
node_researcher -->|"prepares images"| node_image_convert
node_driving_data -->|"provides images"| node_image_convert
node_driving_data -.->|"provides records"| node_prepare_data
node_prepare_data -->|"executes tools"| node_tool_lib
node_prepare_data -->|"uses prompts"| node_tool_prompts
node_cot_generate -->|"uses tool definitions"| node_tool_lib
node_tool_lib -->|"uses coordinates"| node_geometry
node_tool_lib -->|"checks distances"| node_box_distance
node_tool_lib -.->|"invokes perception"| node_tool_models
node_simple_tool_lib -.->|"invokes perception"| node_tool_models
node_researcher -->|"runs inference"| node_tool_inference
node_driving_data -.->|"supplies examples"| node_tool_inference
node_tool_inference -->|"dispatches tools"| node_simple_tool_lib
node_researcher -->|"runs inference"| node_agentthink_inference
node_researcher -->|"runs demo"| node_demo_drivelmm
node_demo_drivelmm -->|"uses driving tools"| node_tool_lib
node_researcher -.->|"runs demo"| node_demo_drivemllm
node_demo_drivemllm -->|"dispatches tools"| node_simple_tool_lib
node_researcher -.->|"runs baseline"| node_evaluation_inference
node_tool_inference -.->|"writes predictions"| node_results
node_agentthink_inference -.->|"writes predictions"| node_results
node_results -.->|"scores predictions"| node_scoring
node_researcher -->|"evaluates results"| node_scoring

click node_prepare_data "https://github.com/curryqka/agentthink/blob/main/scripts/tools/gen_tool_result_data.py"
click node_image_convert "https://github.com/curryqka/agentthink/blob/main/scripts/tools/data_image_converter.py"
click node_cot_generate "https://github.com/curryqka/agentthink/blob/main/scripts/tools/agentthink_data_generater_pipeline.py"
click node_tool_lib "https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_libraries.py"
click node_simple_tool_lib "https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_libraries_simple.py"
click node_tool_prompts "https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_prompts.py"
click node_geometry "https://github.com/curryqka/agentthink/blob/main/scripts/tools/utils/geometry.py"
click node_box_distance "https://github.com/curryqka/agentthink/blob/main/scripts/tools/utils/box_distance.py"
click node_tool_inference "https://github.com/curryqka/agentthink/blob/main/evaluation/inference_withtool.py"
click node_agentthink_inference "https://github.com/curryqka/agentthink/blob/main/evaluation/inference_agentthink.py"
click node_demo_drivelmm "https://github.com/curryqka/agentthink/blob/main/Inference/inference_demo_drivelmm.py"
click node_demo_drivemllm "https://github.com/curryqka/agentthink/blob/main/Inference/inference_demo_drivemllm.py"
click node_scoring "https://github.com/curryqka/agentthink/blob/main/evaluation/evaluation_script.py"
click node_evaluation_inference "https://github.com/curryqka/agentthink/blob/main/evaluation/inference.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepare_data,node_image_convert,node_cot_generate toneBlue
class node_tool_lib,node_simple_tool_lib,node_tool_prompts,node_geometry,node_box_distance toneAmber
class node_tool_inference,node_agentthink_inference,node_demo_drivelmm,node_demo_drivemllm,node_openai toneMint
class node_scoring,node_evaluation_inference toneRose
class node_researcher,node_driving_data,node_tool_models,node_results toneIndigo
```

## Components

### Data preparation

- [Tool-result assembly](https://github.com/curryqka/agentthink/blob/main/scripts/tools/gen_tool_result_data.py): `scripts/tools/gen_tool_result_data.py`
- [Multi-view image conversion](https://github.com/curryqka/agentthink/blob/main/scripts/tools/data_image_converter.py): `scripts/tools/data_image_converter.py`
- [CoT data generation](https://github.com/curryqka/agentthink/blob/main/scripts/tools/agentthink_data_generater_pipeline.py): `scripts/tools/agentthink_data_generater_pipeline.py`

### Reasoning and tools

- [Driving tool library](https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_libraries.py): `scripts/tools/tool_libraries.py`
- [Visual tool library](https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_libraries_simple.py): `scripts/tools/tool_libraries_simple.py`
- [Tool prompts](https://github.com/curryqka/agentthink/blob/main/scripts/tools/tool_prompts.py): `scripts/tools/tool_prompts.py`
- [BEV geometry](https://github.com/curryqka/agentthink/blob/main/scripts/tools/utils/geometry.py): `scripts/tools/utils/geometry.py`
- [Box distances](https://github.com/curryqka/agentthink/blob/main/scripts/tools/utils/box_distance.py): `scripts/tools/utils/box_distance.py`

### Inference

- [Tool-augmented inference](https://github.com/curryqka/agentthink/blob/main/evaluation/inference_withtool.py): `evaluation/inference_withtool.py`
- [AgentThink inference](https://github.com/curryqka/agentthink/blob/main/evaluation/inference_agentthink.py): `evaluation/inference_agentthink.py`
- [DriveLMM demo](https://github.com/curryqka/agentthink/blob/main/Inference/inference_demo_drivelmm.py): `Inference/inference_demo_drivelmm.py`
- [DriveMLLM demo](https://github.com/curryqka/agentthink/blob/main/Inference/inference_demo_drivemllm.py): `Inference/inference_demo_drivemllm.py`

### Evaluation

- [LLM-as-judge scoring](https://github.com/curryqka/agentthink/blob/main/evaluation/evaluation_script.py): `evaluation/evaluation_script.py`
- [Baseline inference](https://github.com/curryqka/agentthink/blob/main/evaluation/inference.py): `evaluation/inference.py`

### Other components

- **Researcher**
- **Driving data**
- **OpenAI API**
- **Vision tool models**
- **Prediction results**

## Connections

- Researcher → CoT data generation: generates data (evidence: `README.md`)
- Driving data → CoT data generation: provides samples (evidence: `README.md`)
- CoT data generation → OpenAI API: requests completions (evidence: `scripts/tools/agentthink_data_generater_pipeline.py`)
- Researcher → Multi-view image conversion: prepares images (evidence: `README.md`)
- Driving data → Multi-view image conversion: provides images (evidence: `README.md`)
- Driving data → Tool-result assembly: provides records
- Tool-result assembly → Driving tool library: executes tools (evidence: `scripts/tools/gen_tool_result_data.py`)
- Tool-result assembly → Tool prompts: uses prompts (evidence: `scripts/tools/gen_tool_result_data.py`)
- CoT data generation → Driving tool library: uses tool definitions (evidence: `scripts/tools/agentthink_data_generater_pipeline.py`)
- Driving tool library → BEV geometry: uses coordinates (evidence: `scripts/tools/tool_libraries.py`)
- Driving tool library → Box distances: checks distances (evidence: `scripts/tools/tool_libraries.py`)
- Driving tool library → Vision tool models: invokes perception
- Visual tool library → Vision tool models: invokes perception
- Researcher → Tool-augmented inference: runs inference (evidence: `README.md`)
- Driving data → Tool-augmented inference: supplies examples
- Tool-augmented inference → Visual tool library: dispatches tools (evidence: `evaluation/inference_withtool.py`)
- Researcher → AgentThink inference: runs inference (evidence: `README.md`)
- Researcher → DriveLMM demo: runs demo (evidence: `README.md`)
- DriveLMM demo → Driving tool library: uses driving tools (evidence: `Inference/inference_demo_drivelmm.py`)
- Researcher → DriveMLLM demo: runs demo
- DriveMLLM demo → Visual tool library: dispatches tools (evidence: `Inference/inference_demo_drivemllm.py`)
- Researcher → Baseline inference: runs baseline
- Tool-augmented inference → Prediction results: writes predictions
- AgentThink inference → Prediction results: writes predictions
- Prediction results → LLM-as-judge scoring: scores predictions
- Researcher → LLM-as-judge scoring: evaluates results (evidence: `README.md`)
