# pipecat-ai/pipecat architecture

> Architecture diagram of the GitHub repository pipecat-ai/pipecat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pipecat-ai/pipecat
- Repository: https://github.com/pipecat-ai/pipecat

## Overview

#### Purpose and Entry Points
Pipecat is a Python 3.11+ framework for composing low-latency voice and multimodal AI agents. Applications primarily import the `pipecat` library and assemble a pipeline; the packaged CLI scaffolds projects and runs evaluations. Core entry points are `src/pipecat/cli/main.py`, `src/pipecat/runner/run.py`, and `src/pipecat/evals/__main__.py`.

#### Frame-Based Processing Model
The central data contract is typed frames: audio, text, control, LLM, and transport events move through processors rather than application-specific callbacks. `FrameProcessor` implementations transform, consume, or emit frames, allowing service and transport implementations to remain composable. The wire/schema boundary is represented by `src/pipecat/frames/frames.py`, `src/pipecat/frames/frames.proto`, and `src/pipecat/processors/frame_processor.py`.

#### Pipeline Execution and Concurrency
A `Pipeline` orders processors; `PipelineTask` and `PipelineRunner` manage lifecycle, scheduling, cancellation, and frame traversal. Parallel and synchronous-parallel variants support fan-out/fan-in, while switchers replace LLMs or services at runtime. This is the in-process agent execution boundary. Key paths: `src/pipecat/pipeline/pipeline.py`, `src/pipecat/pipeline/task.py`, `src/pipecat/pipeline/runner.py`.

#### Conversational Audio Path
A typical cascaded voice agent is: transport input audio → filtering/VAD and turn control → STT → LLM context/response aggregation → TTS → transport output. Audio utilities include resampling, mixers, noise filters, VAD, and local ONNX/CoreML turn models. Turn strategies explicitly decide user-turn start/stop, interruption, mute, and idle behavior rather than delegating all control to providers. Key paths: `src/pipecat/processors/audio/vad_processor.py`, `src/pipecat/turns/user_turn_controller.py`, `src/pipecat/processors/aggregators/llm_context.py`.

#### AI-Service and Tool Boundary
Abstract AI service bases normalize provider behavior across STT, TTS, LLM, vision/image, MCP, and realtime speech-to-speech services. Provider packages implement concrete HTTP/WebSocket SDK integrations; adapters reconcile provider-specific tool/function schemas and context representations. Important contracts and examples: `src/pipecat/services/ai_service.py`, `src/pipecat/services/llm_service.py`, `src/pipecat/adapters/services/open_ai_adapter.py`. External dependencies include OpenAI, Anthropic, AWS/Bedrock/Nova Sonic, Google/Gemini, Azure, Deepgram, ElevenLabs, and many others; optional extras keep the base runtime lightweight.

#### Transport, Telephony, and Serialization
Transports isolate media/session ingress and egress from pipeline logic. Supported boundaries include local devices, WebSockets/FastAPI, WebRTC platforms (Daily, LiveKit, MoQ, Vonage), WhatsApp, and video-avatar transports. Telephony providers are handled through serializer modules that translate provider payloads to/from frames. Key paths: `src/pipecat/transports/base_transport.py`, `src/pipecat/transports/websocket/fastapi.py`, `src/pipecat/serializers/twilio.py`.

#### Multi-Agent Coordination and Structured Flows
Each pipeline can act as an agent. The bus supports local async queues and distributed Redis or PGMQ backends, with bridge processors adapting messages into pipeline flow. Workers provide LLM, proxy, and UI roles. Separately, Flows provides stateful, structured conversation actions over LLM/tool interactions. Key paths: `src/pipecat/bus/bus.py`, `src/pipecat/bus/network/redis.py`, `src/pipecat/flows/manager.py`.

#### Operations, Observability, and Delivery
Metrics, observers, tracing, and Sentry integration observe frame/turn latency and service behavior without becoming business logic. Evaluation scenarios execute through a dedicated harness. The project uses Python packaging with `uv` and optional extras (`pyproject.toml`), while generated CLI templates can target Dockerized server/client deployments. Key paths: `src/pipecat/metrics/metrics.py`, `src/pipecat/utils/tracing/setup.py`, `src/pipecat/evals/harness.py`.

Main entry points: [`src/pipecat/frames/frames.py`](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/frames/frames.py) (Typed frames), [`src/pipecat/services/llm_service.py`](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/llm_service.py) (LLM service base), [`src/pipecat/pipeline/pipeline.py`](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/pipeline.py) (Pipeline), [`src/pipecat/pipeline/runner.py`](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/runner.py) (Pipeline runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry Points"]
  node_cli["CLI<br/>[main.py]"]
  node_run["Runner entry<br/>application runner<br/>[run.py]"]
  node_eval_main["Eval entry<br/>evaluation CLI<br/>[__main__.py]"]
end

subgraph group_runtime["Pipeline Runtime"]
  node_frames["Typed frames<br/>frame contract<br/>[frames.py]"]
  node_frame_schema["Frame schema<br/>protobuf schema<br/>[frames.proto]"]
  node_processor["Frame processor<br/>processor base<br/>[frame_processor.py]"]
  node_pipeline["Pipeline<br/>processor graph<br/>[pipeline.py]"]
  node_task["Pipeline task<br/>lifecycle unit<br/>[task.py]"]
  node_pipeline_runner["Pipeline runner<br/>scheduler<br/>[runner.py]"]
  node_vad["VAD processor<br/>audio processor<br/>[vad_processor.py]"]
  node_turn_controller["User turn control<br/>turn strategy"]
  node_llm_context["LLM context<br/>context aggregator<br/>[llm_context.py]"]
end

subgraph group_integration["Integrations"]
  node_ai_service["AI service base<br/>provider contract<br/>[ai_service.py]"]
  node_llm_service["LLM service base<br/>LLM contract<br/>[llm_service.py]"]
  node_openai_adapter["OpenAI adapter<br/>provider adapter<br/>[open_ai_adapter.py]"]
  node_transport["Base transport<br/>media/session boundary<br/>[base_transport.py]"]
  node_fastapi_ws["FastAPI WebSocket<br/>transport implementation<br/>[fastapi.py]"]
  node_twilio_serializer["Twilio serializer<br/>telephony serializer<br/>[twilio.py]"]
end

subgraph group_coordination["Coordination &amp; Operations"]
  node_bus["Agent bus<br/>message bus<br/>[bus.py]"]
  node_redis_bus[("Redis backend<br/>distributed bus backend<br/>[redis.py]")]
  node_flows["Flows manager<br/>structured flow engine<br/>[manager.py]"]
  node_metrics["Metrics<br/>observability<br/>[metrics.py]"]
  node_tracing["Tracing setup<br/>tracing integration<br/>[setup.py]"]
  node_eval_harness["Evaluation harness<br/>evaluation runtime<br/>[harness.py]"]
end

node_cli -->|"starts apps"| node_run
node_cli -->|"runs evals"| node_eval_main
node_eval_main -->|"executes"| node_eval_harness
node_frame_schema -->|"defines"| node_frames
node_frames -->|"flows through"| node_processor
node_processor -->|"composes into"| node_pipeline
node_pipeline -->|"is managed by"| node_task
node_task -->|"scheduled by"| node_pipeline_runner
node_transport -->|"ingress and egress"| node_frames
node_fastapi_ws -->|"implements"| node_transport
node_twilio_serializer -->|"serializes"| node_frames
node_vad -->|"speech activity"| node_turn_controller
node_turn_controller -->|"turn events"| node_llm_context
node_llm_context -->|"LLM requests"| node_llm_service
node_llm_service -->|"specializes"| node_ai_service
node_openai_adapter -->|"adapts"| node_llm_service
node_bus -->|"bridges agents"| node_pipeline
node_redis_bus -->|"distributed backend"| node_bus
node_flows -->|"orchestrates tools"| node_llm_service
node_metrics -.->|"observes"| node_pipeline_runner
node_tracing -.->|"traces"| node_ai_service
node_eval_harness -->|"evaluates"| node_pipeline_runner

click node_cli "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/cli/main.py"
click node_run "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/runner/run.py"
click node_eval_main "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/evals/__main__.py"
click node_frames "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/frames/frames.py"
click node_frame_schema "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/frames/frames.proto"
click node_processor "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/processors/frame_processor.py"
click node_pipeline "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/pipeline.py"
click node_task "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/task.py"
click node_pipeline_runner "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/runner.py"
click node_vad "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/processors/audio/vad_processor.py"
click node_turn_controller "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/turns/user_turn_controller.py"
click node_llm_context "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/processors/aggregators/llm_context.py"
click node_ai_service "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/ai_service.py"
click node_llm_service "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/llm_service.py"
click node_openai_adapter "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/adapters/services/open_ai_adapter.py"
click node_transport "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/transports/base_transport.py"
click node_fastapi_ws "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/transports/websocket/fastapi.py"
click node_twilio_serializer "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/serializers/twilio.py"
click node_bus "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/bus/bus.py"
click node_redis_bus "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/bus/network/redis.py"
click node_flows "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/flows/manager.py"
click node_metrics "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/metrics/metrics.py"
click node_tracing "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/utils/tracing/setup.py"
click node_eval_harness "https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/evals/harness.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_run,node_eval_main toneBlue
class node_frames,node_frame_schema,node_processor,node_pipeline,node_task,node_pipeline_runner,node_vad,node_turn_controller,node_llm_context toneAmber
class node_ai_service,node_llm_service,node_openai_adapter,node_transport,node_fastapi_ws,node_twilio_serializer toneMint
class node_bus,node_redis_bus,node_flows,node_metrics,node_tracing,node_eval_harness toneRose
```

## Components

### Entry Points

- [CLI](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/cli/main.py): scaffolding CLI · Scaffolds projects and runs evaluation commands. · `src/pipecat/cli/main.py`
- [Runner entry](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/runner/run.py): application runner · `src/pipecat/runner/run.py`
- [Eval entry](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/evals/__main__.py): evaluation CLI · `src/pipecat/evals/__main__.py`

### Pipeline Runtime

In-process frame execution and conversational control.

- [Typed frames](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/frames/frames.py): frame contract · Audio, text, control, LLM, and transport events. · `src/pipecat/frames/frames.py`
- [Frame schema](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/frames/frames.proto): protobuf schema · `src/pipecat/frames/frames.proto`
- [Frame processor](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/processors/frame_processor.py): processor base · Composable transform, consume, and emit boundary. · `src/pipecat/processors/frame_processor.py`
- [Pipeline](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/pipeline.py): processor graph · Orders processors, including parallel and switching variants. · `src/pipecat/pipeline/pipeline.py`
- [Pipeline task](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/task.py): lifecycle unit · `src/pipecat/pipeline/task.py`
- [Pipeline runner](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/runner.py): scheduler · Runs tasks with lifecycle, cancellation, and traversal management. · `src/pipecat/pipeline/runner.py`
- [VAD processor](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/processors/audio/vad_processor.py): audio processor · Filters speech activity for turn handling. · `src/pipecat/processors/audio/vad_processor.py`
- [User turn control](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/turns/user_turn_controller.py): turn strategy · Controls starts, stops, interruptions, mute, and idle behavior. · `src/pipecat/turns/user_turn_controller.py`
- [LLM context](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/processors/aggregators/llm_context.py): context aggregator · Aggregates conversation context and responses. · `src/pipecat/processors/aggregators/llm_context.py`

### Integrations

Provider and media boundaries.

- [AI service base](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/ai_service.py): provider contract · Normalizes STT, TTS, vision, realtime, and MCP providers. · `src/pipecat/services/ai_service.py`
- [LLM service base](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/services/llm_service.py): LLM contract · `src/pipecat/services/llm_service.py`
- [OpenAI adapter](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/adapters/services/open_ai_adapter.py): provider adapter · Reconciles provider tool and context schemas. · `src/pipecat/adapters/services/open_ai_adapter.py`
- [Base transport](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/transports/base_transport.py): media/session boundary · Isolates transport ingress and egress from pipeline logic. · `src/pipecat/transports/base_transport.py`
- [FastAPI WebSocket](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/transports/websocket/fastapi.py): transport implementation · `src/pipecat/transports/websocket/fastapi.py`
- [Twilio serializer](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/serializers/twilio.py): telephony serializer · Translates provider payloads to and from frames. · `src/pipecat/serializers/twilio.py`

### Coordination & Operations

- [Agent bus](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/bus/bus.py): message bus · Connects pipeline agents through local or distributed messaging. · `src/pipecat/bus/bus.py`
- [Redis backend](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/bus/network/redis.py): distributed bus backend · `src/pipecat/bus/network/redis.py`
- [Flows manager](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/flows/manager.py): structured flow engine · Coordinates stateful conversation actions over LLM and tools. · `src/pipecat/flows/manager.py`
- [Metrics](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/metrics/metrics.py): observability · Captures frame, turn, and service latency. · `src/pipecat/metrics/metrics.py`
- [Tracing setup](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/utils/tracing/setup.py): tracing integration · `src/pipecat/utils/tracing/setup.py`
- [Evaluation harness](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/evals/harness.py): evaluation runtime · Executes evaluation scenarios. · `src/pipecat/evals/harness.py`

## Connections

- CLI → Runner entry: starts apps
- CLI → Eval entry: runs evals
- Eval entry → Evaluation harness: executes
- Frame schema → Typed frames: defines
- Typed frames → Frame processor: flows through
- Frame processor → Pipeline: composes into
- Pipeline → Pipeline task: is managed by
- Pipeline task → Pipeline runner: scheduled by
- Base transport → Typed frames: ingress and egress
- FastAPI WebSocket → Base transport: implements
- Twilio serializer → Typed frames: serializes
- VAD processor → User turn control: speech activity
- User turn control → LLM context: turn events
- LLM context → LLM service base: LLM requests
- LLM service base → AI service base: specializes
- OpenAI adapter → LLM service base: adapts
- Agent bus → Pipeline: bridges agents
- Redis backend → Agent bus: distributed backend
- Flows manager → LLM service base: orchestrates tools
- Metrics → Pipeline runner: observes
- Tracing setup → AI service base: traces
- Evaluation harness → Pipeline runner: evaluates
