# adobe-research/dynasaur architecture

> Architecture diagram of the GitHub repository adobe-research/dynasaur, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adobe-research/dynasaur
- Repository: https://github.com/adobe-research/dynasaur

## Overview

##### Main Components

a) Core Components:
- LLM Interface Layer (Azure OpenAI integration)
- Agent System
- Dynamic Action Generator
- Action Library Manager
- Embedding System

b) Supporting Components:
- Environment Handler
- Tools Collection
- Prompt Management System

**Top Layer: Client Interface**

**Middle Layer: Core Processing**

**Bottom Layer: External Services & Tools**

##### Key Relationships to Highlight

- User Input → Environment Handler
- Environment Handler → Agent System
- Agent System ↔ LLM Interface
- Agent System ↔ Dynamic Action Generator
- Dynamic Action Generator ↔ Action Library
- Action Library ↔ Tools Collection

##### Special Architectural Elements

Unique aspects:
- Dynamic Action Generation Loop
- Action Library Extension Process
- Embedding-based Action Retrieval System

##### Component Details

Key files to represent:
- dynasaur.py (Main System Controller)
- agents.py (Agent Implementation)
- actions.py (Action Management)
- prompts.py (Prompt Templates)
- scripts/tools/* (Tool Implementation)

##### External Dependencies

Connections to:
- Azure OpenAI API
- Azure Embeddings API
- Optional APIs (SERPAPI, GPT4V)

##### Implementation Notes

Annotations for:
- Python code generation capability
- Action composition mechanism
- Error recovery system
- GAIA benchmark integration

##### Data Flow Indicators

- → Solid: Direct calls
- ⇄ Double: Bidirectional communication

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Client["Client Layer"]
        UI[User Interface]
        AR["Agent Runner"]:::runner
    end

    subgraph Core["Core Processing Layer"]
        MSC["Main System Controller"]:::controller
        AS["Agent System"]:::agent
        DAG["Dynamic Action Generator"]:::action
        ALM["Action Library Manager"]:::action
        EH["Environment Handler"]:::env
        PMS["Prompt Management System"]:::prompt
        RS["Reformulator System"]:::reformulator
    end

    subgraph External["External Services & Tools Layer"]
        subgraph LLM["LLM Services"]
            AOAI["Azure OpenAI Interface"]:::llm
            AE["Azure Embeddings"]:::llm
        end

        subgraph Tools["Tools Collection"]
            WIT["Web Interaction Tools"]:::tool
            VPT["Visual Processing Tool"]:::tool
            DPT["Document Processing Tool"]:::tool
            CMT["Cookie Management Tool"]:::tool
            UF["Utility Functions"]:::util
        end
    end

    %% Connections
    UI --> AR
    AR --> MSC
    MSC --> AS
    AS <--> DAG
    DAG <--> ALM
    ALM <--> Tools
    AS <--> AOAI
    AS <--> AE
    EH --> AS
    PMS --> AS
    RS --> AS
    
    %% Click Events
    click MSC "https://github.com/adobe-research/dynasaur/blob/main/dynasaur.py"
    click AOAI "https://github.com/adobe-research/dynasaur/blob/main/scripts/llm_engines.py"
    click AS "https://github.com/adobe-research/dynasaur/blob/main/agents.py"
    click DAG "https://github.com/adobe-research/dynasaur/blob/main/actions.py"
    click ALM "https://github.com/adobe-research/dynasaur/blob/main/actions.py"
    click EH "https://github.com/adobe-research/dynasaur/blob/main/env.py"
    click PMS "https://github.com/adobe-research/dynasaur/blob/main/prompts.py"
    click Tools "https://github.com/adobe-research/dynasaur/tree/main/scripts/tools/"
    click RS "https://github.com/adobe-research/dynasaur/blob/main/scripts/reformulator.py"
    click AR "https://github.com/adobe-research/dynasaur/blob/main/scripts/run_agents.py"
    click WIT "https://github.com/adobe-research/dynasaur/blob/main/scripts/tools/web_surfer.py"
    click VPT "https://github.com/adobe-research/dynasaur/blob/main/scripts/tools/visual_qa.py"
    click DPT "https://github.com/adobe-research/dynasaur/blob/main/scripts/tools/mdconvert.py"
    click CMT "https://github.com/adobe-research/dynasaur/blob/main/scripts/tools/cookies.py"
    click UF "https://github.com/adobe-research/dynasaur/blob/main/utils.py"

    %% Styles
    classDef controller fill:#1f77b4
    classDef agent fill:#2ca02c
    classDef action fill:#ff7f0e
    classDef env fill:#9467bd
    classDef prompt fill:#8c564b
    classDef llm fill:#e377c2
    classDef tool fill:#7f7f7f
    classDef util fill:#bcbd22
    classDef runner fill:#17becf
    classDef reformulator fill:#d62728

    %% Legend
    subgraph Legend
        L1["Controller"]:::controller
        L2["Agent"]:::agent
        L3["Action"]:::action
        L4["Environment"]:::env
        L5["Prompt"]:::prompt
        L6["LLM Service"]:::llm
        L7["Tool"]:::tool
        L8["Utility"]:::util
        L9["Runner"]:::runner
        L10["Reformulator"]:::reformulator
    end
```
