How vectara/py-vectara-agentic works
1. What this repo is
vectara-agentic is a Python library for building Vectara-backed AI agents. The main architectural idea is: combine an LLM-driven agent loop with a catalog of tools, where some tools query Vectara corpora and others provide external capabilities or utility behaviors.
2. Public entry points
The primary user-facing surface is Agent plus AgentConfig and the tool factory classes exposed from the top-level package. Those objects define the agent’s behavior, model/provider selection, tool set, topic/instructions, and optional workflow or endpoint behavior.
3. Agent runtime core
The agent_core/ package holds the internal execution machinery: prompt construction, agent factory logic, streaming response handling, serialization, and utility helpers. This is the boundary where chat requests are turned into agent plans, tool calls, and final responses.
4. Configuration and model routing
agent_config.py and llm_utils.py are the main configuration layer for choosing LLM providers and models. The repo supports multiple providers, including OpenAI, Anthropic, Gemini, Groq, Together, Cohere, Bedrock, and a private OpenAI-compatible endpoint. Configuration can come from explicit arguments or environment variables, which matters for how the agent is deployed.
5. Tooling subsystem
Tool construction is centered in tools.py, tools_catalog.py, tool_utils.py, and db_tools.py. This layer produces LlamaIndex-compatible tools from Python functions, Vectara queries, and prebuilt domain integrations. Tool metadata includes descriptions, argument schemas, and whether a tool is eligible to contribute factual context for hallucination correction.
6. Vectara retrieval flow
The core data path for grounded answers is: user query → Vectara tool → Vectara corpus API → retrieved documents or summarized answer → agent response. create_rag_tool() adds a query plus metadata-filtered retrieval over one or more corpora, while create_search_tool() returns matching documents. Fixed filters and typed filter arguments shape the retrieval boundary.
7. Response generation and streaming
streaming.py and the agent methods for sync/async chat support both full-response and token-streaming modes. The stream is not just output transport; it is part of the execution model, because tool calls, partial generation, and final response assembly all happen through the same agent lifecycle.
8. Memory and state
The test suite shows explicit coverage for session memory, fallback memory, and ReAct memory consistency. Architecturally, that means the agent maintains conversational state across turns and uses that state during tool selection and response synthesis, even though memory is not exposed as a standalone top-level subsystem in the README.
9. Hallucination correction boundary
_observability.py, _callback.py, and agent_core/utils/hallucination.py support Vectara Hallucination Correction and progress reporting. VHC analyzes the final answer against factual tool outputs from VHC-eligible tools, then can produce a corrected response. This creates a clear distinction between factual retrieval tools and utility tools such as summarization or formatting.
10. Workflows as an alternate execution path
sub_query_workflow.py and the run()-based workflow support provide a second major execution path besides normal chat. Built-in workflows decompose complex questions into sub-questions, execute them in parallel or sequence, and then synthesize results. This is an orchestration layer on top of the agent/tool/LLM stack, not a separate product.
11. API and deployment boundary
agent_endpoint.py and the Docker files indicate the library can be exposed as a service endpoint, guarded by an endpoint API key. The repo also includes a simple Docker agent example, which suggests the architecture is designed to be embedded in apps or run as a hosted service, not only used as an in-process library.
12. Serialization and portability
agent_core/serialization.py and the related tests show that agents and tools can be serialized/deserialized. That matters for packaging, remote execution, and endpoint-style use cases, and it also explains why tool functions need to be defined at module scope rather than nested.
13. Integration surface
Beyond Vectara, the library integrates with external tool ecosystems such as finance data, web search, Google services, Slack, graph databases, and SQL/database inspection. These integrations are intentionally isolated behind tool abstractions so the agent loop stays generic while capabilities remain pluggable.
14. Validation and quality gates
The tests cover provider integrations, streaming, fallback behavior, endpoint behavior, serialization, workflows, memory, and tool execution. This tells you the main architectural seams are the agent loop, retrieval/tool boundary, provider adapters, and workflow orchestration.
Main entry points: tools.py Tools API · agent.py Agent · test_agent.py Tests · factory.py Factory
Components 20
Agent core
Primary agent runtime, prompts, serialization, and streaming execution.
- vectara_agentic · Top-level library surface exposing the main agent API.
vectara_agentic/__init__.py - Agent · User-facing chat agent entry point.
vectara_agentic/agent.py - Config · Agent configuration, model selection, and runtime options.
vectara_agentic/agent_config.py - LLM routing · Provider and model selection across supported LLM backends.
vectara_agentic/llm_utils.py - Factory · Constructs agents from config, prompts, and tool sets.
vectara_agentic/agent_core/factory.py - Prompts · Prompt construction and agent instruction assembly.
vectara_agentic/agent_core/prompts.py - Streaming · Sync and async streaming response handling.
vectara_agentic/agent_core/streaming.py - Serialization · Serializes and deserializes agents and tool definitions.
vectara_agentic/agent_core/serialization.py - Hallucination · Validates final answers against factual tool outputs.
vectara_agentic/agent_core/utils/hallucination.py - Types · Shared type definitions used across the package.
vectara_agentic/types.py - Utils · General-purpose package helpers.
vectara_agentic/utils.py
Tooling
Tool construction, catalog, and built-in integrations.
- Tools API · Public tool construction helpers exposed to users.
vectara_agentic/tools.py - Tool catalog · Catalog of built-in tool patterns and integrations.
vectara_agentic/tools_catalog.py - Tool utils · Tool metadata, argument schemas, and utility helpers.
vectara_agentic/tool_utils.py - DB tools · Database and SQL-oriented helper tools.
vectara_agentic/db_tools.py
Integration boundaries
Endpoint, observability, and workflow orchestration paths.
- Callbacks · Progress reporting and runtime event hooks.
vectara_agentic/_callback.py - Observability · Shared observability helpers for agent execution.
vectara_agentic/_observability.py - Workflows · Sub-question workflow execution path on top of the agent loop.
vectara_agentic/sub_query_workflow.py - Endpoint · Hosted agent endpoint guarded by an API key.
vectara_agentic/agent_endpoint.py
Docs and tests
Documentation and validation surfaces that reflect the architecture.
- Tests · Coverage for providers, streaming, memory, workflows, endpoint, and tool execution.
tests/test_agent.py
Connections 26
- vectara_agentic → Agent: exports · Top-level package exposes the main agent API.
- vectara_agentic → Config: exports · Configuration is part of the public surface.
- vectara_agentic → Tools API: exports · Tool factories are exposed for user construction.
- Agent → Factory: uses · Agent instances are assembled through the core factory.
- Agent → Config: reads · Agent behavior is driven by configuration.
- Config → LLM routing: routes · Configuration selects provider and model backends.
- Factory → Prompts: builds · Factory composes prompts into runnable agent loops.
- Factory → Tools API: attaches · Factory binds the configured tool set.
- Factory → Streaming: emits · Factory-created agents support streaming execution.
- Streaming → Callbacks: reports · Streaming lifecycle surfaces progress and events.
- Callbacks → Observability: feeds · Callback events connect to observability helpers.
- Tools API → Tool catalog: selects · Public tool helpers are backed by the tool catalog.
- Tools API → Tool utils: describes · Tool factories rely on metadata and schema helpers.
- Tool catalog → DB tools: includes · Catalog contains domain integrations such as database tools.
- Tools API → Hallucination: marks · Some tools are eligible as factual sources for verification.
- Hallucination → Callbacks: signals · Hallucination correction participates in runtime reporting.
- Workflows → Agent: orchestrates · Workflow execution is an alternate path over the agent stack.
- Workflows → Tools API: uses · Workflows call tools while solving sub-questions.
- Endpoint → Agent: hosts · Endpoint exposes the agent as a service boundary.
- Serialization → Factory: restores · Serialization supports reconstructing agent setups.
- Serialization → Tools API: persists · Tool definitions need to be portable for serialization.
- Tests → Agent: validates · Tests cover core agent behavior and memory.
- Tests → Streaming: validates · Tests cover streaming and response assembly.
- Tests → Workflows: validates · Tests cover workflow orchestration behavior.
- Tests → Endpoint: validates · Tests cover API endpoint behavior.
- Tests → Serialization: validates · Tests cover serialization and portability.