How snowflake-labs/orchestration-framework works
Main Components
• Agent Gateway Core – This is the heart of the framework. It includes modules found under agent_gateway/gateway such as:
• gateway.py (orchestrates the request flow)
• planner.py (breaks down requests and generates a task execution graph)
• task_processor.py (coordinates asynchronous, multi-tool function calls)
• output_parser.py (handles the results and formatting)
• constants.py (holds key configuration constants)
• Tools Module – Under agent_gateway/tools, this folder holds the implementations for:
• Base logic (base.py)
• Utility functions (utils.py)
• Logging, Schema, and Snowflake integration (logger.py, schema.py, snowflake_prompts.py, snowflake_tools.py)
• Data files (e.g., sec_chunk_search.parquet, sp500.parquet, sp500_semantic_model.yaml) that are needed for tool configurations.
• Demo App – The demo_app folder includes a Streamlit application (demo_app.py, truagent_demo_app.py) that demonstrates how a client application might interact with the Agent Gateway.
• Testing Suite – The tests directory contains integration and unit tests that validate the orchestration logic and tool interactions.
• External Integrations – The framework is configured to work with external services such as:
• Snowflake (using Snowpark sessions for compute and data retrieval)
• External REST APIs (as demonstrated by the Python Tool example)
• Possibly additional cloud and API integrations (managed via agent configurations).
Relationships and Interactions
• The Agent Gateway serves as the central coordinator. When a user submits a query, it delegates the request to the appropriate tool based on the tool’s configuration (as described in the README).
• The planner component decomposes the request, constructing a “task graph” where tasks are executed asynchronously or in parallel.
• Once the planner creates the task graph, the task_processor invokes the individual tools. Each tool (Cortex Search, Cortex Analyst, Python Tool, SQL Tool) communicates with external systems (such as executing SQL queries on Snowflake or fetching HTML content via an external API).
• The output_parser collects and standardizes the results which are then sent back to the caller.
• The demo app interacts with the Agent Gateway to showcase orchestration via a user-friendly Streamlit UI.
Architectural Patterns and Design Principles
• Modular Design: Separation into distinct modules for the gateway orchestration and tool implementations allows the system to be easily extended with additional tool types.
• Asynchronous Execution: The design leverages parallel asynchronous calls to support multi-step workflows, making the overall responsiveness and scalability better.
• Configuration-Driven: Tools are configured with metadata so that the agent can “route” work correctly based on mutually exclusive language hints.
• Client-Side Orchestration: The orchestration logic is executed on the client side while leveraging Snowflake compute for the heavy lifting.
• Integration Flexibility: The use of a dedicated planner (inspired by the LLM Compiler architecture) means that additional tools or external systems can be integrated without major changes to core logic.