How potpie-ai/potpie works
1. Overall shape
- Potpie is a Python monorepo organized as a modular backend plus a few internal libraries. The main runtime is a FastAPI API server with background workers for async work.
- The product centers on turning repositories into a knowledge graph and then using that graph to drive agent reasoning, code search, and contextual assistance.
2. Primary entrypoints
- app/main.py and app/api/router.py are the main HTTP application entrypoints.
- app/celery/worker.py runs async jobs through Celery + Redis.
- app/src/context-engine/adapters/inbound/hatchet/worker.py and related queue wiring indicate a second worker path for context-ingestion orchestration.
- There are also CLI entrypoints under app/src/context-engine/adapters/inbound/cli and MCP/server-style adapters for local tooling.
3. Core data stores and infrastructure
- PostgreSQL stores product data: users, projects, conversations, messages, integrations, auth providers, usage, and cache/ledger tables.
- Neo4j is the central knowledge graph backing code understanding and agent context.
- Redis is used for job brokering and streaming-style coordination.
- The repo also includes migrations via Alembic, showing the schema is tightly coupled to the backend services.
- Some parsing/indexing flows also reference Qdrant and vector-style indexing services.
4. Repository parsing and knowledge-graph construction
- The parsing pipeline lives in app/modules/parsing and app/src/parsing.
- app/src/parsing is a Rust-based parser/indexer that works with tree-sitter query files to extract symbols and structure from code.
- Python services then take those parse outputs and construct/update the graph via graph_construction, knowledge_graph, and qdrant_indexing_service.
- The flow is: repository source → parser/symbol extraction → graph construction/inference → Neo4j and related indices.
5. Context-engine as the core domain
- app/src/context-engine is the most explicit architecture boundary in the repo.
- It follows a hexagonal style: domain defines ports and entities, application contains use cases/services, and adapters connect to HTTP, GitHub, Postgres, Neo4j, Hatchet, and LLM-backed synthesis/reconciliation.
- This package owns ingestion, reconciliation, temporal search, context resolution, and context-graph jobs.
6. Agent and intelligence subsystem
- app/modules/intelligence is the orchestration layer for agents, tools, prompts, providers, and tracing.
- It contains:
- Prebuilt and custom agents
- Multi-agent execution
- Prompt and model selection
- Tool registry and dispatch
- LLM provider abstraction for OpenAI, Anthropic, OpenRouter, Ollama, etc.
- Agents do not directly “know” the repo; they depend on tools that query the code graph, file contents, sandbox, or external services.
7. Tooling boundaries for agent actions
- Tools are a major architectural boundary under app/modules/intelligence/tools.
- They cover:
- code search and graph queries
- local repo/workspace operations
- sandbox execution
- web tools
- integrations like Jira, Confluence, Linear, and GitHub
- The tool registry/service pattern means agents invoke capabilities through a curated execution layer rather than arbitrary direct access.
8. Conversation and memory layer
- app/modules/conversations manages sessions, conversations, messages, access control, and Redis-based streaming.
- This is the user-facing chat state layer that persists dialogue history in Postgres while coordinating live response streaming.
- It sits between the API and the agent system: user message → conversation/session service → agent execution → streamed response → stored message history.
9. Repository access, code providers, and sync
- app/modules/code_provider abstracts GitHub, GitBucket, local repos, and user-local tunnel access.
- This layer is responsible for cloning, branch operations, PR workflows, and repo-safe operations.
- app/modules/repo_manager and app/modules/sandbox_repos support lifecycle management for codebases and credentials.
- These abstractions are what the parsing and agent tools use to fetch or modify source safely.
10. Integrations and auth
- app/modules/auth supports API keys plus SSO/auth-provider records; the README highlights Firebase Auth in production and a local dummy user in development.
- app/src/integrations is a dedicated bounded context for external product integrations, including OAuth flows and provider-specific adapters.
- Supported integrations in the tree include GitHub, Linear, Jira, Confluence, and Sentry-style OAuth patterns, with provider registries and project-source mapping.
11. Sandbox and execution environment
- app/src/sandbox is a separate packaged subsystem for isolated repo execution, workspace management, parser runner support, and runtime selection.
- It has multiple outbound runtimes: local subprocess, Docker, Daytona, and in-memory/file-backed stores.
- This is the place where code can be analyzed or manipulated without directly coupling agent logic to the host machine.
12. Async processing and eventing
- app/modules/event_bus and app/celery/tasks coordinate background workflows.
- The event bus has task handlers and webhook handlers, suggesting a fan-out model for post-processing and external notifications.
- Parsing, agent jobs, cache cleanup, and context-graph reconciliation all run asynchronously rather than in the request thread.
13. Supporting product services
- Smaller but still relevant modules include:
- billing and usage for metering/subscriptions
- media for attachments
- search for query-facing search APIs
- tunnel for workspace/socket connectivity
- analytics and key_management
- These are secondary to the core graph/agent pipeline but feed authentication, billing, collaboration, and workspace connectivity.
14. Deployment and runtime topology
- The repo ships separate deployment artifacts for different services: API, conversation server, and Celery.
- compose.yaml, Dockerfiles, and stage/prod deployment folders show the system is deployed as multiple cooperating processes, not a single monolith process.
- The architecture diagram should therefore show at least three runtime boxes: API, background workers, and data stores, with the sandbox/parsing and context-engine pipelines as distinct internal subsystems.
Main entry points: main.py API server · context-engine Context engine · parsing Parsing pipeline · registry Tool registry
Components 19
Runtimes
Main processes that serve HTTP, run workers, or host subsystems.
- API server · Primary HTTP application entrypoint and router assembly.
app/main.py - API router · Top-level API routing and module composition.
app/api/router.py - Celery worker · Background job process for agent and parsing tasks.
app/celery/worker.py - Context worker · Secondary worker path for context-ingestion orchestration.
app/src/context-engine/adapters/inbound/hatchet/worker.py
Core domains
Primary backend subsystems for conversations, agents, parsing, context graph, and repositories.
- Context engine · Core bounded context for ingestion, reconciliation, temporal search, and context resolution.
app/src/context-engine - Conversations · Conversation, message, session, access, and streaming state management.
app/modules/conversations - Intelligence · Agents, prompts, providers, tool registry, and tracing for LLM-driven workflows.
app/modules/intelligence - Tool registry · Curated execution layer that exposes code, web, sandbox, and integration tools to agents.
app/modules/intelligence/tools/registry - Code provider · Abstraction over GitHub, GitBucket, local repos, and safe repo operations.
app/modules/code_provider - Repo manager · Repo lifecycle helpers and sync coordination for codebase access.
app/modules/repo_manager - Parsing pipeline · Repository parsing, symbol extraction, and knowledge-graph construction services.
app/modules/parsing - Rust parser · Rust-based parser/indexer that drives tree-sitter extraction and code indexing.
app/src/parsing - Sandbox · Isolated workspace, runtime selection, and parser runner environment.
app/src/sandbox - Integrations · OAuth, provider registries, and external product integrations.
app/src/integrations
Platform
Shared stores and external infrastructure the backend depends on.
- Postgres · Primary product database for users, projects, conversations, auth, usage, and ledgers.
- Neo4j · Knowledge graph backing code understanding and contextual reasoning.
- Redis · Job brokering, coordination, and streaming support.
- Qdrant · Vector-style indexing and retrieval used by parsing and context workflows.
- LLM providers · Provider abstraction for OpenAI, Anthropic, OpenRouter, Ollama, and similar models.
app/modules/intelligence/provider
Connections 29
- API server → API router: mounts · API server assembles the top-level router.
- API server → Conversations: serves · HTTP endpoints expose conversation and chat behavior.
- API server → Intelligence: serves · HTTP endpoints expose agent and prompt workflows.
- API server → Parsing pipeline: serves · HTTP endpoints expose parsing and graph construction.
- API server → Integrations: serves · HTTP endpoints manage external integrations and sources.
- API server → Postgres: reads/writes · API persists product state in Postgres.
- API server → Redis: uses · API cooperates with Redis-backed streaming and coordination.
- Celery worker → Redis: consumes · Celery uses Redis for background job transport.
- Celery worker → Parsing pipeline: runs jobs · Background tasks execute parsing and indexing workflows.
- Celery worker → Intelligence: runs jobs · Background tasks execute agent-related work.
- Context worker → Context engine: executes · Worker runs context ingestion and reconciliation use cases.
- Context engine → Neo4j: mutates/query · Context engine reads and updates the knowledge graph.
- Context engine → Postgres: persists · Context engine stores ingestion events, ledgers, and status.
- Context engine → Redis: queues · Context jobs are coordinated through queued execution.
- Context engine → LLM providers: calls · Reconciliation and synthesis can use LLM-backed adapters.
- Conversations → Postgres: persists · Conversations, messages, and access records are stored in Postgres.
- Conversations → Redis: streams · Live response streaming is coordinated via Redis.
- Intelligence → Tool registry: invokes · Agents access capabilities through the tool registry.
- Tool registry → Code provider: queries · Tools retrieve and modify source through repo providers.
- Tool registry → Sandbox: executes · Tools delegate isolated code execution to the sandbox.
- Tool registry → LLM providers: uses · Tool-enabled agents depend on model backends for reasoning.
- Code provider → Repo manager: orchestrates · Repo provider layer coordinates lifecycle and sync helpers.
- Code provider → Sandbox: shares repos · Repo access feeds sandbox workspaces and safe operations.
- Parsing pipeline → Rust parser: delegates · Python parsing services rely on the Rust parser/indexer.
- Rust parser → Qdrant: indexes · Parsed outputs can feed vector-style retrieval/indexing.
- Parsing pipeline → Neo4j: builds graph · Parsing and inference update the knowledge graph.
- Parsing pipeline → Qdrant: indexes · Parsing workflows may also update vector indices.
- Integrations → Postgres: stores config · Integration records and OAuth state are persisted in Postgres.
- Integrations → Redis: signals · Integration workflows can use asynchronous coordination.