How curryqka/agentthink works
AgentThink is a tool-augmented vision-language reasoning framework for autonomous-driving scenarios. Its main workflow prepares driving examples and tool-augmented reasoning data, runs model inference with optional perception tools, and evaluates predictions. The repository also includes demo inference paths and tool-result preparation. The README documents the major capabilities; relationships between sampled modules are shown only where the excerpts, source index, or README support them. Several evaluation and tool implementations are unsampled, so their internal links are omitted.
Main entry points: tool_libraries.py Driving tool library · agentthink_data_generater_pipeline.py CoT data generation · inference_withtool.py Tool-augmented inference · gen_tool_result_data.py Tool-result assembly
Components 19
Data preparation
- Tool-result assembly
scripts/tools/gen_tool_result_data.py - Multi-view image conversion
scripts/tools/data_image_converter.py - CoT data generation
scripts/tools/agentthink_data_generater_pipeline.py
Reasoning and tools
- Driving tool library
scripts/tools/tool_libraries.py - Visual tool library
scripts/tools/tool_libraries_simple.py - Tool prompts
scripts/tools/tool_prompts.py - BEV geometry
scripts/tools/utils/geometry.py - Box distances
scripts/tools/utils/box_distance.py
Inference
- Tool-augmented inference
evaluation/inference_withtool.py - AgentThink inference
evaluation/inference_agentthink.py - DriveLMM demo
Inference/inference_demo_drivelmm.py - DriveMLLM demo
Inference/inference_demo_drivemllm.py
Evaluation
- LLM-as-judge scoring
evaluation/evaluation_script.py - Baseline inference
evaluation/inference.py
Other components
- Researcher
- Driving data
- OpenAI API
- Vision tool models
- Prediction results
Connections 26
- Researcher → CoT data generation: generates data
README.md - Driving data → CoT data generation: provides samples
README.md - CoT data generation → OpenAI API: requests completions
scripts/tools/agentthink_data_generater_pipeline.py - Researcher → Multi-view image conversion: prepares images
README.md - Driving data → Multi-view image conversion: provides images
README.md - Driving data → Tool-result assembly: provides records
- Tool-result assembly → Driving tool library: executes tools
scripts/tools/gen_tool_result_data.py - Tool-result assembly → Tool prompts: uses prompts
scripts/tools/gen_tool_result_data.py - CoT data generation → Driving tool library: uses tool definitions
scripts/tools/agentthink_data_generater_pipeline.py - Driving tool library → BEV geometry: uses coordinates
scripts/tools/tool_libraries.py - Driving tool library → Box distances: checks distances
scripts/tools/tool_libraries.py - Driving tool library → Vision tool models: invokes perception
- Visual tool library → Vision tool models: invokes perception
- Researcher → Tool-augmented inference: runs inference
README.md - Driving data → Tool-augmented inference: supplies examples
- Tool-augmented inference → Visual tool library: dispatches tools
evaluation/inference_withtool.py - Researcher → AgentThink inference: runs inference
README.md - Researcher → DriveLMM demo: runs demo
README.md - DriveLMM demo → Driving tool library: uses driving tools
Inference/inference_demo_drivelmm.py - Researcher → DriveMLLM demo: runs demo
- DriveMLLM demo → Visual tool library: dispatches tools
Inference/inference_demo_drivemllm.py - Researcher → Baseline inference: runs baseline
- Tool-augmented inference → Prediction results: writes predictions
- AgentThink inference → Prediction results: writes predictions
- Prediction results → LLM-as-judge scoring: scores predictions
- Researcher → LLM-as-judge scoring: evaluates results
README.md