How cloudwego/eino works
1. What this repo is
Eino is a Go LLM application framework, not an app. Its architecture is layered: stable domain types at the bottom, reusable component abstractions above that, then orchestration/composition, then agent frameworks and prebuilt agent patterns on top.
2. Core data model lives in schema/
schema defines the shared language of the framework: messages, documents, tools, and streams, plus serialization helpers. Most higher-level packages pass these types around, so this is the main boundary between framework logic and concrete model/provider implementations.
3. Component abstraction layer is in components/
This package family defines interfaces and options for model, tool, retriever, embedding, prompt, document, and indexer. These are the pluggable building blocks that external provider packages implement. The repo itself mostly defines contracts and callback hooks, while actual provider implementations are expected to come from eino-ext.
4. Orchestration engine is in compose/
compose is the graph/workflow runtime. It provides nodes, edges, branching, parallel execution, DAGs, state, checkpoints, resume, interrupts, and stream concatenation/copying. This is the deterministic execution layer that can run standalone or be wrapped as a tool for an agent.
5. Agent runtime is in adk/
adk is the agent development kit. It wires models, tools, flows, and callbacks into executable agents and runners. This is where you find chat-model agents, ReAct execution, retry logic, interrupt/resume support, deterministic transfer, and the runner/handler entry points used to drive agent queries.
6. Prebuilt agent patterns are in adk/prebuilt/
The repo ships opinionated higher-level agents such as deep, planexecute, and supervisor. These compose lower-level agent/tool primitives into ready-made patterns for multi-step reasoning, planning, and delegation. They are built on top of adk and compose, not separate systems.
7. flow/ contains specialized application flows
flow/agent packages agent-specific orchestration helpers, including ReAct and multi-agent host coordination. flow/retriever and flow/indexer provide retrieval/indexing workflows like parent retrieval, router-based retrieval, and multi-query retrieval. This layer sits between generic composition and concrete application patterns.
8. Callbacks are a cross-cutting subsystem
callbacks/, utils/callbacks/, and internal/callbacks/ implement aspect-style injection for start/end/error and stream events. These hooks are used across components, graphs, and agents for logging, tracing, metrics, and other observability behavior without coupling it into business logic.
9. Persistence and human-in-the-loop support
Checkpointing, interrupt, and resume are first-class in compose and adk, with storage/backends under adk/filesystem and related middleware. This is the stateful boundary that enables pausing execution, storing progress, and continuing later with preserved context.
10. Middlewares extend agent/component behavior
adk/middlewares/ contains reusable behavior modifiers such as filesystem handling, tool-call patching, dynamic tool search, task planning, reduction, skill injection, and summarization. These sit between the agent loop and underlying tools/models, usually transforming prompts, tool results, or task state.
11. Internal packages hold shared engine utilities
internal/ contains concurrency-safe helpers, serialization, generic utilities, stream/channel primitives, panic protection, and mocks. These packages are intentionally not part of the public architecture; they support the framework internals and tests.
12. Overall data flow
Typical execution starts with a schema input, flows through a components implementation such as a chat model or retriever, is orchestrated by compose or adk, and is observed by callbacks. If the path includes tools or multi-step reasoning, the same flow may branch into subgraphs, task middleware, checkpoints, and resume points before producing a final schema output or stream.
13. Repository boundary
This repo defines the framework, contracts, orchestration, and built-in agent patterns. Provider integrations, concrete LLM/tool backends, and many real-world examples are intentionally externalized to companion repos such as eino-ext and eino-examples.
Main entry points: graph.go Graph runtime · react.go Agent runtime · schema Schema · callbacks Callbacks
Components 21
Core model
Shared framework types and serialization boundary
- Schema · Shared messages, documents, tools, streams, and serialization helpers
schema
Components
Pluggable contracts for providers and adapters
- Model · Chat-model interfaces, options, and callback hooks
components/model - Tool · Tool interfaces, interrupt handling, and tool options
components/tool - Retriever · Retriever interfaces and configuration for pluggable backends
components/retriever - Embedding · Embedding interfaces and options
components/embedding - Prompt · Prompt and chat-template interfaces and options
components/prompt - Document · Document parsing and transformation interfaces
components/document - Indexer · Indexing interfaces and configuration
components/indexer
Orchestration
Graph and workflow runtime
- Graph runtime · Nodes, edges, branching, parallel execution, and graph execution
compose/graph.go - State & resume · Checkpointing, interrupts, resume, and mutable execution state
compose/state.go - Workflow · Higher-level workflow composition over the graph runtime
compose/workflow.go - Streams · Stream concatenation, readers, and copy helpers
compose/stream_concat.go
Agents
Agent runtime, runners, middlewares, and built-ins
- Agent runtime · Chat-model agents, ReAct execution, retries, and transfer logic
adk/react.go - Runner · Handlers and runner entry points for driving agent queries
adk/runner.go - Middlewares · Filesystem, tool patching, planning, reduction, skill, and summarization layers
adk/middlewares - Prebuilt agents · Deep, plan-execute, and supervisor agent compositions
adk/prebuilt
Specialized flows
Higher-level retrieval, indexing, and agent flow helpers
- Agent flows · Reusable orchestration helpers for agent execution patterns
flow/agent - Retriever flows · Parent, router, and multi-query retrieval workflows
flow/retriever - Indexer flows · Higher-level indexing workflow helpers
flow/indexer
Cross-cutting
Callbacks and observability hooks
- Callbacks · Aspect-style start, end, error, and stream event injection
callbacks
Internal
Private engine utilities and support code
- Internal utils · Concurrency, serialization, safety, and mock support for framework internals
internal
Connections 30
- Schema → Model: types · Shared message and stream types underpin model contracts
- Schema → Tool: types · Tool calls and tool results use the shared schema language
- Schema → Retriever: types · Retriever inputs and outputs are built on shared schema types
- Schema → Document: types · Document abstractions reuse the schema data model
- Model → Graph runtime: run · Model invocations are orchestrated through graph nodes
- Tool → Graph runtime: tool nodes · Tools can be embedded into composed execution graphs
- Retriever → Retriever flows: source · Retriever implementations feed retrieval workflows
- Embedding → Indexer flows: embed · Embedding backends support indexing workflows
- Document → Indexer flows: parse · Document parsing feeds indexing pipelines
- Indexer → Indexer flows: index · Indexer contracts are used by indexing flows
- Graph runtime → State & resume: checkpoint · Graph execution can pause, resume, and persist state
- Graph runtime → Streams: stream · Graph runtime moves streaming values through the pipeline
- Workflow → Graph runtime: builds on · Workflows compile down to the graph runtime
- Agent runtime → Model: calls · Agents drive chat-model execution through component contracts
- Agent runtime → Tool: uses · Agent loops invoke tools through the tool abstraction
- Agent runtime → Graph runtime: orchestrates · Agents can delegate multi-step execution to composition graphs
- Runner → Agent runtime: drives · Runners and handlers are the outer execution entrypoints
- Agent runtime → Middlewares: extends · Middleware modifies prompts, tools, tasks, and outputs
- Prebuilt agents → Agent runtime: packages · Prebuilt patterns are assembled from the core agent runtime
- Prebuilt agents → Graph runtime: composes · Built-in agent patterns rely on graph/workflow composition
- Agent flows → Agent runtime: supports · Agent flow helpers layer on the agent runtime
- Agent flows → Graph runtime: coordinates · Agent flows coordinate execution with graphs and branches
- Retriever flows → Retriever: wraps · Retrieval flows wrap retriever contracts into application patterns
- Indexer flows → Indexer: wraps · Indexing flows wrap indexer contracts into application patterns
- Callbacks → Model: observes · Callbacks attach to model lifecycle events
- Callbacks → Graph runtime: observes · Callbacks instrument graph execution and streaming
- Callbacks → Agent runtime: observes · Callbacks instrument agent execution and tool use
- Internal utils → Callbacks: supports · Internal callback infrastructure powers shared hook management
- Internal utils → State & resume: supports · Private helpers back interrupts, resume, and concurrency safety
- Internal utils → Runner: supports · Runner and framework internals rely on private utilities