# snowflake-labs/orchestration-framework architecture

> Architecture diagram of the GitHub repository snowflake-labs/orchestration-framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/snowflake-labs/orchestration-framework
- Repository: https://github.com/snowflake-labs/orchestration-framework

## Overview

##### Main Components
- Agent Gateway Core – This is the heart of the framework. It includes modules found under agent_gateway/gateway such as:
- gateway.py (orchestrates the request flow)
- planner.py (breaks down requests and generates a task execution graph)
- task_processor.py (coordinates asynchronous, multi-tool function calls)
- output_parser.py (handles the results and formatting)
- constants.py (holds key configuration constants)
- Tools Module – Under agent_gateway/tools, this folder holds the implementations for:
- Base logic (base.py)
- Utility functions (utils.py)
- Logging, Schema, and Snowflake integration (logger.py, schema.py, snowflake_prompts.py, snowflake_tools.py)
- Data files (e.g., sec_chunk_search.parquet, sp500.parquet, sp500_semantic_model.yaml) that are needed for tool configurations.
- Demo App – The demo_app folder includes a Streamlit application (demo_app.py, truagent_demo_app.py) that demonstrates how a client application might interact with the Agent Gateway.
- Testing Suite – The tests directory contains integration and unit tests that validate the orchestration logic and tool interactions.
- External Integrations – The framework is configured to work with external services such as:
- Snowflake (using Snowpark sessions for compute and data retrieval)
- External REST APIs (as demonstrated by the Python Tool example)
- Possibly additional cloud and API integrations (managed via agent configurations).

##### Relationships and Interactions
- The Agent Gateway serves as the central coordinator. When a user submits a query, it delegates the request to the appropriate tool based on the tool’s configuration (as described in the README).
- The planner component decomposes the request, constructing a “task graph” where tasks are executed asynchronously or in parallel.
- Once the planner creates the task graph, the task_processor invokes the individual tools. Each tool (Cortex Search, Cortex Analyst, Python Tool, SQL Tool) communicates with external systems (such as executing SQL queries on Snowflake or fetching HTML content via an external API).
- The output_parser collects and standardizes the results which are then sent back to the caller.
- The demo app interacts with the Agent Gateway to showcase orchestration via a user-friendly Streamlit UI.

##### Architectural Patterns and Design Principles
- Modular Design: Separation into distinct modules for the gateway orchestration and tool implementations allows the system to be easily extended with additional tool types.
- Asynchronous Execution: The design leverages parallel asynchronous calls to support multi-step workflows, making the overall responsiveness and scalability better.
- Configuration-Driven: Tools are configured with metadata so that the agent can “route” work correctly based on mutually exclusive language hints.
- Client-Side Orchestration: The orchestration logic is executed on the client side while leveraging Snowflake compute for the heavy lifting.
- Integration Flexibility: The use of a dedicated planner (inspired by the LLM Compiler architecture) means that additional tools or external systems can be integrated without major changes to core logic.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Agent Gateway Core"
        AG1["Gateway (Orchestration)"]:::core
        AG2["Planner (Task Breakdown)"]:::core
        AG3["Task Processor (Async Execution)"]:::core
        AG4["Output Parser (Results Formatting)"]:::core
        AG5["Constants (Configuration)"]:::core
    end

    subgraph "Tool Layer"
        TM1["Base"]:::tools
        TM2["Utils"]:::tools
        TM3["Logger"]:::tools
        TM4["Schema"]:::tools
        TM5["Snowflake Prompts"]:::tools
        TM6["Snowflake Tools"]:::tools
        TM7["Data Resources"]:::tools
        subgraph "Service Tools"
            ST1["Cortex Search"]:::tools
            ST2["Cortex Analyst"]:::tools
            ST3["Python Tool"]:::tools
            ST4["SQL Tool"]:::tools
        end
    end

    subgraph "Demo Application"
        DA1["Demo App"]:::demo
        DA2["Truagent Demo"]:::demo
    end

    subgraph "External Services"
        ES1["Snowflake (Snowpark Session)"]:::external
        ES2["External REST API"]:::external
    end

    TS["Testing Suite"]:::tests

    %% Connections from Demo to Agent Gateway
    DA1 -->|"submits_request"| AG1
    AG1 -->|"delegates_to"| AG2
    AG2 -->|"generates_task_graph"| AG3
    AG3 -->|"invokes_tool"| ST1
    AG3 -->|"invokes_tool"| ST2
    AG3 -->|"invokes_tool"| ST3
    AG3 -->|"invokes_tool"| ST4
    ST1 -->|"returns_result"| AG4
    ST2 -->|"returns_result"| AG4
    ST3 -->|"returns_result"| AG4
    ST4 -->|"returns_result"| AG4
    AG4 -->|"formats_output"| DA1

    %% Internal Agent Gateway flows
    AG1 --> AG2
    AG2 --> AG3
    AG3 --> AG4

    %% External integrations
    AG3 -->|"integrates_with"| ES1
    ST3 -->|"calls"| ES2

    %% Testing Suite interactions
    TS -->|"validates"| AG1
    DA1 -->|"triggers_tests"| TS

    %% Click Events for Agent Gateway Core
    click AG1 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/gateway/gateway.py"
    click AG2 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/gateway/planner.py"
    click AG3 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/gateway/task_processor.py"
    click AG4 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/gateway/output_parser.py"
    click AG5 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/gateway/constants.py"

    %% Click Events for Tools Module
    click TM1 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/tools/base.py"
    click TM2 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/tools/utils.py"
    click TM3 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/tools/logger.py"
    click TM4 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/tools/schema.py"
    click TM5 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/tools/snowflake_prompts.py"
    click TM6 "https://github.com/snowflake-labs/orchestration-framework/blob/main/agent_gateway/tools/snowflake_tools.py"
    click TM7 "https://github.com/snowflake-labs/orchestration-framework/tree/main/agent_gateway/tools/data"

    %% Click Events for Demo Application and Testing
    click DA1 "https://github.com/snowflake-labs/orchestration-framework/blob/main/demo_app/demo_app.py"
    click DA2 "https://github.com/snowflake-labs/orchestration-framework/blob/main/demo_app/truagent_demo_app.py"
    click TS "https://github.com/snowflake-labs/orchestration-framework/tree/main/tests/"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef tools fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef demo fill:#FFD700,stroke:#000,stroke-width:2px;
    classDef external fill:#DDA0DD,stroke:#000,stroke-width:2px;
    classDef tests fill:#D3D3D3,stroke:#000,stroke-width:2px;
```
