How kryvokhyzha/langgraph-agent-toolkit works
Project type and purpose
• This is a full-stack Python toolkit + reference application for building, deploying, and interacting with AI agents.
• It comprises three “faces”:
– A FastAPI-based backend service (langgraph_agent_toolkit/service)
– A Streamlit frontend UI (langgraph_agent_toolkit/streamlit_app.py)
– A Python client library (langgraph_agent_toolkit/client)
• Core domain: LangGraph agent orchestration, LLM abstraction (LiteLLM), memory management, observability.
File structure
• Top level:
– configs/ + docker/ + docker-compose.yaml → deployment & env configurations
– langgraph_agent_toolkit/ → toolkit source, subdivided into agents, core, client, schema, service
– tests/ → comprehensive test suite (unit + integration + e2e)
• Patterns & modules:
– agents/blueprints/ → pre-built agent definitions (chatbot, bg tasks, supervisor, etc.)
– core/ → memory (Postgres/SQLite), LLM factory (OpenAI, fake), observability (Langfuse, Langsmith), prompt/templates
– service/ → FastAPI app, routes, middleware, error handlers, streaming vs non-streaming endpoints
– client/ → AgentClient abstraction to call service API programmatically
– streamlit_app.py → standalone UI, calls the local/remote service via the client
README for architectural clues
• Stack: LangGraph (graph-based agent flows) + FastAPI + Streamlit + Pydantic + LiteLLM + PostgreSQL/SQLite + Langfuse/Langsmith
a. Main components:
– Frontend: Streamlit UI (calls Client)
– Client: AgentClient (HTTP/gRPC)
– Backend Service: FastAPI App
– Core Toolkit Modules:
• Agent Executor (orchestrates LangGraph flows)
• LLM Adapter Layer (LiteLLM proxy)
• Memory Store (Postgres/SQLite)
• Observability (Langfuse, Langsmith)
• Prompt Manager (templates, variables)
– External Services / Infrastructure:
• PostgreSQL (persistence)
• Redis (caching, if configured)
• ClickHouse (analytics, optional)
• MinIO (artifact store, optional)
• Langfuse / Langsmith (observability backend)
b. Relationships / data-flows:
1. User interacts with Streamlit UI → issues “run agent” requests
2. UI calls AgentClient → issues HTTP request to FastAPI service
3. FastAPI service authenticates, routes request to AgentExecutor
4. AgentExecutor loads agent blueprint → enters LangGraph flow
5. At each step, LLM Adapter requests tokens from one of the configured LLM providers (via LiteLLM proxy)
6. Memory Store is read/written for context/history
7. Observability hooks log events/traces to Langfuse/Langsmith
8. Final response streams back (via FastAPI) → AgentClient → UI
c. Architectural patterns / principles:
– Layered modularity: clear separation between service, core, agents, client, UI
– Dependency injection via Pydantic settings & factory methods
– Asynchronous, event-driven flows (async/await streaming)
– Pluggable abstractions for LLM, memory, observability
d. Key technologies / libs:
– FastAPI (API + streaming)
– Streamlit (UI)
– Pydantic (settings, schemas)
– LangGraph (flow orchestration)
– LiteLLM (multi-provider LLM proxy)
– Postgres/SQLite (memory)
– Langfuse/Langsmith (observability)
– Docker + docker-compose (deployment)
Elements
– Blue: user-facing (UI, Client)
– Green: service/API layer
– Yellow: core business logic modules
– Gray: external infrastructure/datastores
– Purple: observability & analytics