# potpie-ai/potpie architecture

> Architecture diagram of the GitHub repository potpie-ai/potpie, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/potpie-ai/potpie
- Repository: https://github.com/potpie-ai/potpie

## Overview

1. **Overall shape**
   - Potpie is a **Python monorepo** organized as a modular backend plus a few internal libraries. The main runtime is a **FastAPI API server** with **background workers** for async work.
   - The product centers on turning repositories into a **knowledge graph** and then using that graph to drive agent reasoning, code search, and contextual assistance.

2. **Primary entrypoints**
   - `app/main.py` and `app/api/router.py` are the main HTTP application entrypoints.
   - `app/celery/worker.py` runs async jobs through **Celery + Redis**.
   - `app/src/context-engine/adapters/inbound/hatchet/worker.py` and related queue wiring indicate a second worker path for context-ingestion orchestration.
   - There are also CLI entrypoints under `app/src/context-engine/adapters/inbound/cli` and MCP/server-style adapters for local tooling.

3. **Core data stores and infrastructure**
   - **PostgreSQL** stores product data: users, projects, conversations, messages, integrations, auth providers, usage, and cache/ledger tables.
   - **Neo4j** is the central knowledge graph backing code understanding and agent context.
   - **Redis** is used for job brokering and streaming-style coordination.
   - The repo also includes migrations via **Alembic**, showing the schema is tightly coupled to the backend services.
   - Some parsing/indexing flows also reference **Qdrant** and vector-style indexing services.

4. **Repository parsing and knowledge-graph construction**
   - The parsing pipeline lives in `app/modules/parsing` and `app/src/parsing`.
   - `app/src/parsing` is a **Rust-based parser/indexer** that works with tree-sitter query files to extract symbols and structure from code.
   - Python services then take those parse outputs and construct/update the graph via `graph_construction`, `knowledge_graph`, and `qdrant_indexing_service`.
   - The flow is: repository source → parser/symbol extraction → graph construction/inference → Neo4j and related indices.

5. **Context-engine as the core domain**
   - `app/src/context-engine` is the most explicit architecture boundary in the repo.
   - It follows a **hexagonal style**: `domain` defines ports and entities, `application` contains use cases/services, and `adapters` connect to HTTP, GitHub, Postgres, Neo4j, Hatchet, and LLM-backed synthesis/reconciliation.
   - This package owns ingestion, reconciliation, temporal search, context resolution, and context-graph jobs.

6. **Agent and intelligence subsystem**
   - `app/modules/intelligence` is the orchestration layer for agents, tools, prompts, providers, and tracing.
   - It contains:
     - **Prebuilt and custom agents**
     - **Multi-agent execution**
     - **Prompt and model selection**
     - **Tool registry and dispatch**
     - **LLM provider abstraction** for OpenAI, Anthropic, OpenRouter, Ollama, etc.
   - Agents do not directly “know” the repo; they depend on tools that query the code graph, file contents, sandbox, or external services.

7. **Tooling boundaries for agent actions**
   - Tools are a major architectural boundary under `app/modules/intelligence/tools`.
   - They cover:
     - code search and graph queries
     - local repo/workspace operations
     - sandbox execution
     - web tools
     - integrations like Jira, Confluence, Linear, and GitHub
   - The tool registry/service pattern means agents invoke capabilities through a curated execution layer rather than arbitrary direct access.

8. **Conversation and memory layer**
   - `app/modules/conversations` manages sessions, conversations, messages, access control, and Redis-based streaming.
   - This is the user-facing chat state layer that persists dialogue history in Postgres while coordinating live response streaming.
   - It sits between the API and the agent system: user message → conversation/session service → agent execution → streamed response → stored message history.

9. **Repository access, code providers, and sync**
   - `app/modules/code_provider` abstracts GitHub, GitBucket, local repos, and user-local tunnel access.
   - This layer is responsible for cloning, branch operations, PR workflows, and repo-safe operations.
   - `app/modules/repo_manager` and `app/modules/sandbox_repos` support lifecycle management for codebases and credentials.
   - These abstractions are what the parsing and agent tools use to fetch or modify source safely.

10. **Integrations and auth**
   - `app/modules/auth` supports API keys plus SSO/auth-provider records; the README highlights **Firebase Auth** in production and a local dummy user in development.
   - `app/src/integrations` is a dedicated bounded context for external product integrations, including OAuth flows and provider-specific adapters.
   - Supported integrations in the tree include **GitHub, Linear, Jira, Confluence, and Sentry-style OAuth patterns**, with provider registries and project-source mapping.

11. **Sandbox and execution environment**
   - `app/src/sandbox` is a separate packaged subsystem for isolated repo execution, workspace management, parser runner support, and runtime selection.
   - It has multiple outbound runtimes: local subprocess, Docker, Daytona, and in-memory/file-backed stores.
   - This is the place where code can be analyzed or manipulated without directly coupling agent logic to the host machine.

12. **Async processing and eventing**
   - `app/modules/event_bus` and `app/celery/tasks` coordinate background workflows.
   - The event bus has task handlers and webhook handlers, suggesting a fan-out model for post-processing and external notifications.
   - Parsing, agent jobs, cache cleanup, and context-graph reconciliation all run asynchronously rather than in the request thread.

13. **Supporting product services**
   - Smaller but still relevant modules include:
     - `billing` and `usage` for metering/subscriptions
     - `media` for attachments
     - `search` for query-facing search APIs
     - `tunnel` for workspace/socket connectivity
     - `analytics` and `key_management`
   - These are secondary to the core graph/agent pipeline but feed authentication, billing, collaboration, and workspace connectivity.

14. **Deployment and runtime topology**
   - The repo ships separate deployment artifacts for different services: **API**, **conversation server**, and **Celery**.
   - `compose.yaml`, Dockerfiles, and stage/prod deployment folders show the system is deployed as multiple cooperating processes, not a single monolith process.
   - The architecture diagram should therefore show at least three runtime boxes: API, background workers, and data stores, with the sandbox/parsing and context-engine pipelines as distinct internal subsystems.

Main entry points: [`app/main.py`](https://github.com/potpie-ai/potpie/blob/main/app/main.py) (API server), [`app/src/context-engine`](https://github.com/potpie-ai/potpie/tree/main/app/src/context-engine) (Context engine), [`app/modules/parsing`](https://github.com/potpie-ai/potpie/tree/main/app/modules/parsing) (Parsing pipeline), [`app/modules/intelligence/tools/registry`](https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence/tools/registry) (Tool registry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtimes"]
  node_main_api["API server<br/>FastAPI entrypoint<br/>[main.py]"]
  node_api_router["API router<br/>HTTP routing<br/>[router.py]"]
  node_celery_worker["Celery worker<br/>async worker<br/>[worker.py]"]
  node_context_worker["Context worker<br/>hatchet worker<br/>[worker.py]"]
end

subgraph group_domain["Core domains"]
  node_context_engine["Context engine<br/>hexagonal domain"]
  node_conversation_system["Conversations<br/>chat state"]
  node_intelligence_system["Intelligence<br/>agent orchestration"]
  node_tool_registry["Tool registry<br/>tool boundary"]
  node_code_provider["Code provider<br/>repo access"]
  node_repo_manager["Repo manager<br/>repo lifecycle"]
  node_parsing_pipeline["Parsing pipeline<br/>graph construction"]
  node_rust_parser["Rust parser<br/>indexer"]
  node_sandbox_system["Sandbox<br/>isolated exec"]
  node_integrations_system["Integrations<br/>external adapters"]
end

subgraph group_platform["Platform"]
  node_postgres_db[("Postgres<br/>relational store")]
  node_neo4j_db[("Neo4j<br/>graph store")]
  node_redis[("Redis<br/>broker/cache")]
  node_qdrant[("Qdrant<br/>vector index")]
  node_llm_providers["LLM providers<br/>model backends"]
end

node_main_api -->|"mounts"| node_api_router
node_main_api -->|"serves"| node_conversation_system
node_main_api -->|"serves"| node_intelligence_system
node_main_api -->|"serves"| node_parsing_pipeline
node_main_api -->|"serves"| node_integrations_system
node_main_api -->|"reads/writes"| node_postgres_db
node_main_api -->|"uses"| node_redis
node_celery_worker -->|"consumes"| node_redis
node_celery_worker -->|"runs jobs"| node_parsing_pipeline
node_celery_worker -->|"runs jobs"| node_intelligence_system
node_context_worker -->|"executes"| node_context_engine
node_context_engine -->|"mutates/query"| node_neo4j_db
node_context_engine -->|"persists"| node_postgres_db
node_context_engine -->|"queues"| node_redis
node_context_engine -->|"calls"| node_llm_providers
node_conversation_system -->|"persists"| node_postgres_db
node_conversation_system -->|"streams"| node_redis
node_intelligence_system -->|"invokes"| node_tool_registry
node_tool_registry -->|"queries"| node_code_provider
node_tool_registry -->|"executes"| node_sandbox_system
node_tool_registry -->|"uses"| node_llm_providers
node_code_provider -->|"orchestrates"| node_repo_manager
node_code_provider -->|"shares repos"| node_sandbox_system
node_parsing_pipeline -->|"delegates"| node_rust_parser
node_rust_parser -->|"indexes"| node_qdrant
node_parsing_pipeline -->|"builds graph"| node_neo4j_db
node_parsing_pipeline -->|"indexes"| node_qdrant
node_integrations_system -->|"stores config"| node_postgres_db
node_integrations_system -->|"signals"| node_redis

click node_main_api "https://github.com/potpie-ai/potpie/blob/main/app/main.py"
click node_api_router "https://github.com/potpie-ai/potpie/blob/main/app/api/router.py"
click node_celery_worker "https://github.com/potpie-ai/potpie/blob/main/app/celery/worker.py"
click node_context_worker "https://github.com/potpie-ai/potpie/blob/main/app/src/context-engine/adapters/inbound/hatchet/worker.py"
click node_context_engine "https://github.com/potpie-ai/potpie/tree/main/app/src/context-engine"
click node_conversation_system "https://github.com/potpie-ai/potpie/tree/main/app/modules/conversations"
click node_intelligence_system "https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence"
click node_tool_registry "https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence/tools/registry"
click node_code_provider "https://github.com/potpie-ai/potpie/tree/main/app/modules/code_provider"
click node_repo_manager "https://github.com/potpie-ai/potpie/tree/main/app/modules/repo_manager"
click node_parsing_pipeline "https://github.com/potpie-ai/potpie/tree/main/app/modules/parsing"
click node_rust_parser "https://github.com/potpie-ai/potpie/tree/main/app/src/parsing"
click node_sandbox_system "https://github.com/potpie-ai/potpie/tree/main/app/src/sandbox"
click node_integrations_system "https://github.com/potpie-ai/potpie/tree/main/app/src/integrations"
click node_llm_providers "https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence/provider"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main_api,node_api_router,node_celery_worker,node_context_worker toneBlue
class node_context_engine,node_conversation_system,node_intelligence_system,node_tool_registry,node_code_provider,node_repo_manager,node_parsing_pipeline,node_rust_parser,node_sandbox_system,node_integrations_system toneAmber
class node_postgres_db,node_neo4j_db,node_redis,node_qdrant,node_llm_providers toneMint
```

## Components

### Runtimes

Main processes that serve HTTP, run workers, or host subsystems.

- [API server](https://github.com/potpie-ai/potpie/blob/main/app/main.py): FastAPI entrypoint · Primary HTTP application entrypoint and router assembly. · `app/main.py`
- [API router](https://github.com/potpie-ai/potpie/blob/main/app/api/router.py): HTTP routing · Top-level API routing and module composition. · `app/api/router.py`
- [Celery worker](https://github.com/potpie-ai/potpie/blob/main/app/celery/worker.py): async worker · Background job process for agent and parsing tasks. · `app/celery/worker.py`
- [Context worker](https://github.com/potpie-ai/potpie/blob/main/app/src/context-engine/adapters/inbound/hatchet/worker.py): hatchet worker · Secondary worker path for context-ingestion orchestration. · `app/src/context-engine/adapters/inbound/hatchet/worker.py`

### Core domains

Primary backend subsystems for conversations, agents, parsing, context graph, and repositories.

- [Context engine](https://github.com/potpie-ai/potpie/tree/main/app/src/context-engine): hexagonal domain · Core bounded context for ingestion, reconciliation, temporal search, and context resolution. · `app/src/context-engine`
- [Conversations](https://github.com/potpie-ai/potpie/tree/main/app/modules/conversations): chat state · Conversation, message, session, access, and streaming state management. · `app/modules/conversations`
- [Intelligence](https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence): agent orchestration · Agents, prompts, providers, tool registry, and tracing for LLM-driven workflows. · `app/modules/intelligence`
- [Tool registry](https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence/tools/registry): tool boundary · Curated execution layer that exposes code, web, sandbox, and integration tools to agents. · `app/modules/intelligence/tools/registry`
- [Code provider](https://github.com/potpie-ai/potpie/tree/main/app/modules/code_provider): repo access · Abstraction over GitHub, GitBucket, local repos, and safe repo operations. · `app/modules/code_provider`
- [Repo manager](https://github.com/potpie-ai/potpie/tree/main/app/modules/repo_manager): repo lifecycle · Repo lifecycle helpers and sync coordination for codebase access. · `app/modules/repo_manager`
- [Parsing pipeline](https://github.com/potpie-ai/potpie/tree/main/app/modules/parsing): graph construction · Repository parsing, symbol extraction, and knowledge-graph construction services. · `app/modules/parsing`
- [Rust parser](https://github.com/potpie-ai/potpie/tree/main/app/src/parsing): indexer · Rust-based parser/indexer that drives tree-sitter extraction and code indexing. · `app/src/parsing`
- [Sandbox](https://github.com/potpie-ai/potpie/tree/main/app/src/sandbox): isolated exec · Isolated workspace, runtime selection, and parser runner environment. · `app/src/sandbox`
- [Integrations](https://github.com/potpie-ai/potpie/tree/main/app/src/integrations): external adapters · OAuth, provider registries, and external product integrations. · `app/src/integrations`

### Platform

Shared stores and external infrastructure the backend depends on.

- **Postgres**: relational store · Primary product database for users, projects, conversations, auth, usage, and ledgers.
- **Neo4j**: graph store · Knowledge graph backing code understanding and contextual reasoning.
- **Redis**: broker/cache · Job brokering, coordination, and streaming support.
- **Qdrant**: vector index · Vector-style indexing and retrieval used by parsing and context workflows.
- [LLM providers](https://github.com/potpie-ai/potpie/tree/main/app/modules/intelligence/provider): model backends · Provider abstraction for OpenAI, Anthropic, OpenRouter, Ollama, and similar models. · `app/modules/intelligence/provider`

## Connections

- API server → API router: mounts · API server assembles the top-level router.
- API server → Conversations: serves · HTTP endpoints expose conversation and chat behavior.
- API server → Intelligence: serves · HTTP endpoints expose agent and prompt workflows.
- API server → Parsing pipeline: serves · HTTP endpoints expose parsing and graph construction.
- API server → Integrations: serves · HTTP endpoints manage external integrations and sources.
- API server → Postgres: reads/writes · API persists product state in Postgres.
- API server → Redis: uses · API cooperates with Redis-backed streaming and coordination.
- Celery worker → Redis: consumes · Celery uses Redis for background job transport.
- Celery worker → Parsing pipeline: runs jobs · Background tasks execute parsing and indexing workflows.
- Celery worker → Intelligence: runs jobs · Background tasks execute agent-related work.
- Context worker → Context engine: executes · Worker runs context ingestion and reconciliation use cases.
- Context engine → Neo4j: mutates/query · Context engine reads and updates the knowledge graph.
- Context engine → Postgres: persists · Context engine stores ingestion events, ledgers, and status.
- Context engine → Redis: queues · Context jobs are coordinated through queued execution.
- Context engine → LLM providers: calls · Reconciliation and synthesis can use LLM-backed adapters.
- Conversations → Postgres: persists · Conversations, messages, and access records are stored in Postgres.
- Conversations → Redis: streams · Live response streaming is coordinated via Redis.
- Intelligence → Tool registry: invokes · Agents access capabilities through the tool registry.
- Tool registry → Code provider: queries · Tools retrieve and modify source through repo providers.
- Tool registry → Sandbox: executes · Tools delegate isolated code execution to the sandbox.
- Tool registry → LLM providers: uses · Tool-enabled agents depend on model backends for reasoning.
- Code provider → Repo manager: orchestrates · Repo provider layer coordinates lifecycle and sync helpers.
- Code provider → Sandbox: shares repos · Repo access feeds sandbox workspaces and safe operations.
- Parsing pipeline → Rust parser: delegates · Python parsing services rely on the Rust parser/indexer.
- Rust parser → Qdrant: indexes · Parsed outputs can feed vector-style retrieval/indexing.
- Parsing pipeline → Neo4j: builds graph · Parsing and inference update the knowledge graph.
- Parsing pipeline → Qdrant: indexes · Parsing workflows may also update vector indices.
- Integrations → Postgres: stores config · Integration records and OAuth state are persisted in Postgres.
- Integrations → Redis: signals · Integration workflows can use asynchronous coordination.
