# vectara/py-vectara-agentic architecture

> Architecture diagram of the GitHub repository vectara/py-vectara-agentic, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vectara/py-vectara-agentic
- Repository: https://github.com/vectara/py-vectara-agentic

## Overview

1. **What this repo is**
   `vectara-agentic` is a Python library for building Vectara-backed AI agents. The main architectural idea is: combine an LLM-driven agent loop with a catalog of tools, where some tools query Vectara corpora and others provide external capabilities or utility behaviors.

2. **Public entry points**
   The primary user-facing surface is `Agent` plus `AgentConfig` and the tool factory classes exposed from the top-level package. Those objects define the agent’s behavior, model/provider selection, tool set, topic/instructions, and optional workflow or endpoint behavior.

3. **Agent runtime core**
   The `agent_core/` package holds the internal execution machinery: prompt construction, agent factory logic, streaming response handling, serialization, and utility helpers. This is the boundary where chat requests are turned into agent plans, tool calls, and final responses.

4. **Configuration and model routing**
   `agent_config.py` and `llm_utils.py` are the main configuration layer for choosing LLM providers and models. The repo supports multiple providers, including OpenAI, Anthropic, Gemini, Groq, Together, Cohere, Bedrock, and a private OpenAI-compatible endpoint. Configuration can come from explicit arguments or environment variables, which matters for how the agent is deployed.

5. **Tooling subsystem**
   Tool construction is centered in `tools.py`, `tools_catalog.py`, `tool_utils.py`, and `db_tools.py`. This layer produces LlamaIndex-compatible tools from Python functions, Vectara queries, and prebuilt domain integrations. Tool metadata includes descriptions, argument schemas, and whether a tool is eligible to contribute factual context for hallucination correction.

6. **Vectara retrieval flow**
   The core data path for grounded answers is: user query → Vectara tool → Vectara corpus API → retrieved documents or summarized answer → agent response. `create_rag_tool()` adds a query plus metadata-filtered retrieval over one or more corpora, while `create_search_tool()` returns matching documents. Fixed filters and typed filter arguments shape the retrieval boundary.

7. **Response generation and streaming**
   `streaming.py` and the agent methods for sync/async chat support both full-response and token-streaming modes. The stream is not just output transport; it is part of the execution model, because tool calls, partial generation, and final response assembly all happen through the same agent lifecycle.

8. **Memory and state**
   The test suite shows explicit coverage for session memory, fallback memory, and ReAct memory consistency. Architecturally, that means the agent maintains conversational state across turns and uses that state during tool selection and response synthesis, even though memory is not exposed as a standalone top-level subsystem in the README.

9. **Hallucination correction boundary**
   `_observability.py`, `_callback.py`, and `agent_core/utils/hallucination.py` support Vectara Hallucination Correction and progress reporting. VHC analyzes the final answer against factual tool outputs from VHC-eligible tools, then can produce a corrected response. This creates a clear distinction between factual retrieval tools and utility tools such as summarization or formatting.

10. **Workflows as an alternate execution path**
   `sub_query_workflow.py` and the `run()`-based workflow support provide a second major execution path besides normal chat. Built-in workflows decompose complex questions into sub-questions, execute them in parallel or sequence, and then synthesize results. This is an orchestration layer on top of the agent/tool/LLM stack, not a separate product.

11. **API and deployment boundary**
   `agent_endpoint.py` and the Docker files indicate the library can be exposed as a service endpoint, guarded by an endpoint API key. The repo also includes a simple Docker agent example, which suggests the architecture is designed to be embedded in apps or run as a hosted service, not only used as an in-process library.

12. **Serialization and portability**
   `agent_core/serialization.py` and the related tests show that agents and tools can be serialized/deserialized. That matters for packaging, remote execution, and endpoint-style use cases, and it also explains why tool functions need to be defined at module scope rather than nested.

13. **Integration surface**
   Beyond Vectara, the library integrates with external tool ecosystems such as finance data, web search, Google services, Slack, graph databases, and SQL/database inspection. These integrations are intentionally isolated behind tool abstractions so the agent loop stays generic while capabilities remain pluggable.

14. **Validation and quality gates**
   The tests cover provider integrations, streaming, fallback behavior, endpoint behavior, serialization, workflows, memory, and tool execution. This tells you the main architectural seams are the agent loop, retrieval/tool boundary, provider adapters, and workflow orchestration.

Main entry points: [`vectara_agentic/tools.py`](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tools.py) (Tools API), [`vectara_agentic/agent.py`](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent.py) (Agent), [`tests/test_agent.py`](https://github.com/vectara/py-vectara-agentic/blob/main/tests/test_agent.py) (Tests), [`vectara_agentic/agent_core/factory.py`](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/factory.py) (Factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Agent core"]
  node_pkg["vectara_agentic<br/>[__init__.py]"]
  node_agent["Agent<br/>runtime API<br/>[agent.py]"]
  node_config["Config<br/>settings<br/>[agent_config.py]"]
  node_llm["LLM routing<br/>provider adapters<br/>[llm_utils.py]"]
  node_factory["Factory<br/>agent build<br/>[factory.py]"]
  node_prompts["Prompts<br/>prompting<br/>[prompts.py]"]
  node_stream["Streaming<br/>response flow<br/>[streaming.py]"]
  node_ser["Serialization<br/>state portability<br/>[serialization.py]"]
  node_hc{{"Hallucination<br/>verification<br/>[hallucination.py]"}}
  node_types["Types<br/>shared models<br/>[types.py]"]
  node_utils["Utils<br/>shared helpers<br/>[utils.py]"]
end

subgraph group_tools["Tooling"]
  node_tools_api["Tools API<br/>tool facade<br/>[tools.py]"]
  node_catalog["Tool catalog<br/>tool registry<br/>[tools_catalog.py]"]
  node_tool_utils["Tool utils<br/>tool schema<br/>[tool_utils.py]"]
  node_db_tools["DB tools<br/>integration tools<br/>[db_tools.py]"]
end

subgraph group_integration["Integration boundaries"]
  node_callbacks["Callbacks<br/>observability<br/>[_callback.py]"]
  node_obs["Observability<br/>telemetry<br/>[_observability.py]"]
  node_workflow["Workflows<br/>orchestration"]
  node_endpoint["Endpoint<br/>service boundary<br/>[agent_endpoint.py]"]
end

subgraph group_docs["Docs and tests"]
  node_tests["Tests<br/>validation<br/>[test_agent.py]"]
end

node_pkg -->|"exports"| node_agent
node_pkg -->|"exports"| node_config
node_pkg -->|"exports"| node_tools_api
node_agent -->|"uses"| node_factory
node_agent -->|"reads"| node_config
node_config -->|"routes"| node_llm
node_factory -->|"builds"| node_prompts
node_factory -->|"attaches"| node_tools_api
node_factory -->|"emits"| node_stream
node_stream -->|"reports"| node_callbacks
node_callbacks -->|"feeds"| node_obs
node_tools_api -->|"selects"| node_catalog
node_tools_api -->|"describes"| node_tool_utils
node_catalog -->|"includes"| node_db_tools
node_tools_api -->|"marks"| node_hc
node_hc -->|"signals"| node_callbacks
node_workflow -->|"orchestrates"| node_agent
node_workflow -->|"uses"| node_tools_api
node_endpoint -->|"hosts"| node_agent
node_ser -->|"restores"| node_factory
node_ser -->|"persists"| node_tools_api
node_tests -.->|"validates"| node_agent
node_tests -.->|"validates"| node_stream
node_tests -.->|"validates"| node_workflow
node_tests -.->|"validates"| node_endpoint
node_tests -.->|"validates"| node_ser

click node_pkg "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/__init__.py"
click node_agent "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent.py"
click node_config "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_config.py"
click node_llm "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/llm_utils.py"
click node_factory "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/factory.py"
click node_prompts "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/prompts.py"
click node_stream "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/streaming.py"
click node_ser "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/serialization.py"
click node_hc "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/utils/hallucination.py"
click node_callbacks "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/_callback.py"
click node_obs "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/_observability.py"
click node_tools_api "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tools.py"
click node_catalog "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tools_catalog.py"
click node_tool_utils "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tool_utils.py"
click node_db_tools "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/db_tools.py"
click node_workflow "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/sub_query_workflow.py"
click node_endpoint "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_endpoint.py"
click node_types "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/types.py"
click node_utils "https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/utils.py"
click node_tests "https://github.com/vectara/py-vectara-agentic/blob/main/tests/test_agent.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pkg,node_agent,node_config,node_llm,node_factory,node_prompts,node_stream,node_ser,node_hc,node_types,node_utils toneBlue
class node_tools_api,node_catalog,node_tool_utils,node_db_tools toneAmber
class node_callbacks,node_obs,node_workflow,node_endpoint toneMint
class node_tests toneRose
```

## Components

### Agent core

Primary agent runtime, prompts, serialization, and streaming execution.

- [vectara_agentic](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/__init__.py): package · Top-level library surface exposing the main agent API. · `vectara_agentic/__init__.py`
- [Agent](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent.py): runtime API · User-facing chat agent entry point. · `vectara_agentic/agent.py`
- [Config](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_config.py): settings · Agent configuration, model selection, and runtime options. · `vectara_agentic/agent_config.py`
- [LLM routing](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/llm_utils.py): provider adapters · Provider and model selection across supported LLM backends. · `vectara_agentic/llm_utils.py`
- [Factory](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/factory.py): agent build · Constructs agents from config, prompts, and tool sets. · `vectara_agentic/agent_core/factory.py`
- [Prompts](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/prompts.py): prompting · Prompt construction and agent instruction assembly. · `vectara_agentic/agent_core/prompts.py`
- [Streaming](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/streaming.py): response flow · Sync and async streaming response handling. · `vectara_agentic/agent_core/streaming.py`
- [Serialization](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/serialization.py): state portability · Serializes and deserializes agents and tool definitions. · `vectara_agentic/agent_core/serialization.py`
- [Hallucination](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_core/utils/hallucination.py): verification · Validates final answers against factual tool outputs. · `vectara_agentic/agent_core/utils/hallucination.py`
- [Types](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/types.py): shared models · Shared type definitions used across the package. · `vectara_agentic/types.py`
- [Utils](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/utils.py): shared helpers · General-purpose package helpers. · `vectara_agentic/utils.py`

### Tooling

Tool construction, catalog, and built-in integrations.

- [Tools API](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tools.py): tool facade · Public tool construction helpers exposed to users. · `vectara_agentic/tools.py`
- [Tool catalog](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tools_catalog.py): tool registry · Catalog of built-in tool patterns and integrations. · `vectara_agentic/tools_catalog.py`
- [Tool utils](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/tool_utils.py): tool schema · Tool metadata, argument schemas, and utility helpers. · `vectara_agentic/tool_utils.py`
- [DB tools](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/db_tools.py): integration tools · Database and SQL-oriented helper tools. · `vectara_agentic/db_tools.py`

### Integration boundaries

Endpoint, observability, and workflow orchestration paths.

- [Callbacks](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/_callback.py): observability · Progress reporting and runtime event hooks. · `vectara_agentic/_callback.py`
- [Observability](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/_observability.py): telemetry · Shared observability helpers for agent execution. · `vectara_agentic/_observability.py`
- [Workflows](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/sub_query_workflow.py): orchestration · Sub-question workflow execution path on top of the agent loop. · `vectara_agentic/sub_query_workflow.py`
- [Endpoint](https://github.com/vectara/py-vectara-agentic/blob/main/vectara_agentic/agent_endpoint.py): service boundary · Hosted agent endpoint guarded by an API key. · `vectara_agentic/agent_endpoint.py`

### Docs and tests

Documentation and validation surfaces that reflect the architecture.

- [Tests](https://github.com/vectara/py-vectara-agentic/blob/main/tests/test_agent.py): validation · Coverage for providers, streaming, memory, workflows, endpoint, and tool execution. · `tests/test_agent.py`

## Connections

- vectara_agentic → Agent: exports · Top-level package exposes the main agent API.
- vectara_agentic → Config: exports · Configuration is part of the public surface.
- vectara_agentic → Tools API: exports · Tool factories are exposed for user construction.
- Agent → Factory: uses · Agent instances are assembled through the core factory.
- Agent → Config: reads · Agent behavior is driven by configuration.
- Config → LLM routing: routes · Configuration selects provider and model backends.
- Factory → Prompts: builds · Factory composes prompts into runnable agent loops.
- Factory → Tools API: attaches · Factory binds the configured tool set.
- Factory → Streaming: emits · Factory-created agents support streaming execution.
- Streaming → Callbacks: reports · Streaming lifecycle surfaces progress and events.
- Callbacks → Observability: feeds · Callback events connect to observability helpers.
- Tools API → Tool catalog: selects · Public tool helpers are backed by the tool catalog.
- Tools API → Tool utils: describes · Tool factories rely on metadata and schema helpers.
- Tool catalog → DB tools: includes · Catalog contains domain integrations such as database tools.
- Tools API → Hallucination: marks · Some tools are eligible as factual sources for verification.
- Hallucination → Callbacks: signals · Hallucination correction participates in runtime reporting.
- Workflows → Agent: orchestrates · Workflow execution is an alternate path over the agent stack.
- Workflows → Tools API: uses · Workflows call tools while solving sub-questions.
- Endpoint → Agent: hosts · Endpoint exposes the agent as a service boundary.
- Serialization → Factory: restores · Serialization supports reconstructing agent setups.
- Serialization → Tools API: persists · Tool definitions need to be portable for serialization.
- Tests → Agent: validates · Tests cover core agent behavior and memory.
- Tests → Streaming: validates · Tests cover streaming and response assembly.
- Tests → Workflows: validates · Tests cover workflow orchestration behavior.
- Tests → Endpoint: validates · Tests cover API endpoint behavior.
- Tests → Serialization: validates · Tests cover serialization and portability.
