How lucasdoreac/sober works
1. Repository purpose
- This is a self-hosted “control plane” for a personal AI system.
- The architecture is centered on two ingress paths: a Streamlit web UI for dashboards/chat and a WhatsApp ingress through WAHA.
- Both paths terminate in the same FastAPI backend, which is the orchestration boundary.
2. Primary runtime split
- frontend/ runs the Streamlit UI.
- app/ runs the API layer, webhook handlers, dependency wiring, logging, and orchestration entrypoints.
- The stack is designed for bare-metal or Docker Compose deployment, not Kubernetes.
3. Ingress and request boundary
- The main backend boundary is split between regular API chat traffic and WAHA webhook traffic.
- app/router.py and app/router_waha.py appear to define those separate ingress routes.
- The WhatsApp path has an explicit authorization gate (“whitelist/soberania”) before any agent execution.
4. Idempotency vs conversation state
- The README makes a key architectural distinction:
- request_id = transport idempotency, preventing duplicate processing.
- thread_id = conversational identity, used for LangGraph checkpoints and memory.
- The app/repositories/dedup.py module and the initdb/001_request_dedup.sql schema support the dedup boundary in PostgreSQL.
5. Orchestration core
- The system’s reasoning/execution layer is built around LangGraph-style orchestration.
- workflows/orchestrator.py and app/graph.py are the likely composition points for the conversation graph.
- The README indicates the graph runs asynchronously in background tasks so webhook responses return quickly.
6. Agent architecture
- agents/ contains a multi-role agent set: commercial, concierge, executor, financeiro, fiscal, pm, qa, supervisor, sustentacao.
- agents/base.py suggests a common agent contract, with the specialized files implementing role-specific behavior.
- prompts/*.md holds the role prompts, so behavior is split between code and prompt assets.
7. Tool and integration layer
- integrations/ is the external-system boundary for business systems: CRM, billing, fiscal, and WhatsApp.
- mcp_servers/ exposes at least CRM and WhatsApp capability through MCP-style servers, which is a separate tool-access layer for agents.
- app/mcp_client.py likely connects the graph/agents to those servers.
8. WhatsApp egress/inbound bridge
- The WhatsApp engine is WAHA (self-hosted WhatsApp API).
- app/services/waha_client.py is the outbound client used by the backend to send WhatsApp messages back to the user.
- app/router_waha.py is the inbound webhook endpoint from WAHA to the API.
9. Persistence and state
- PostgreSQL is the main transactional store for dedup, LangGraph checkpoints, and audit-related state.
- initdb/ contains schema initialization/migrations for agency/request dedup/thread interventions.
- Qdrant is part of the architecture as the vector store for RAG and long-term memory, per the README.
- The repo also mentions Ollama and Groq as inference backends, implying model execution is abstracted behind the orchestration layer.
10. Frontend shape
- frontend/app.py is the Streamlit entrypoint.
- frontend/pages/ is organized as operational dashboards rather than a single chat screen: fleets, approvals, projects, agents, contacts, payments, incidents.
- frontend/utils/api.py suggests the UI is a thin client over the FastAPI backend, not a separate business logic tier.
11. Cross-cutting concerns
- app/logging_async.py and app/watchdog.py indicate operational support for async logging and health/watchdog behavior.
- app/dependencies.py and app/config.py centralize runtime wiring and environment-driven configuration.
- The architecture favors explicit dependency injection and boundary checks over hidden framework magic.
12. Deployment and operational model
- docker-compose.yml, Dockerfile, and Dockerfile.frontend define a multi-service deployment.
- The README’s volume layout shows persistent state for Postgres, Qdrant, Ollama, and WAHA.
- This is an infra-heavy application where data durability and service isolation are first-class concerns.
13. Quality and verification
- tests/ covers dependencies, routing, sustentacao behavior, and watchdog logic.
- Utility scripts like audit_deps.py, check_routes.py, and diagnostico_sober.py suggest the repo includes built-in architecture/health checks.
14. What an architecture diagram should emphasize
- Draw the system as two user-facing ingress channels into one FastAPI control plane.
- Show the control plane splitting into: authorization/dedup, async orchestration, agent/tool execution, and external persistence/model services.
- Make the request_id and thread_id separation explicit, because it is central to correctness here.
Main entry points: graph.py Orchestrator · main.py API app · docker-compose.yml Compose deploy · router.py API routes
Components 25
Frontend
Streamlit dashboards and thin API client.
- Streamlit UI · Main Streamlit entrypoint for dashboards and chat.
frontend/app.py - Dashboards · Operational pages for fleets, approvals, projects, agents, contacts, payments, and incidents.
frontend/pages - API client · Thin client used by the UI to call the FastAPI backend.
frontend/utils/api.py
FastAPI control plane
Ingress, dependency wiring, auth, and orchestration boundary.
- API app · Backend application bootstrap and runtime entrypoint.
app/main.py - Config · Environment-driven configuration and settings.
app/config.py - Dependencies · Dependency injection and service wiring for the backend.
app/dependencies.py - API routes · Regular API ingress for chat and control traffic.
app/router.py - WAHA webhook · Inbound webhook boundary from WAHA into the backend.
app/router_waha.py
Orchestration
Conversation graph, background execution, and agent coordination.
- Orchestrator · Conversation graph composition and orchestration boundary.
app/graph.py - Workflow engine · Runs graph execution and background task coordination.
workflows/orchestrator.py
Agents
Role-specific agents plus shared agent contract and prompts.
- Agent base · Shared contract used by specialized role agents.
agents/base.py - Role agents · Commercial, concierge, executor, financeiro, fiscal, pm, qa, supervisor, and sustentacao roles.
agents - Prompts · Role prompt definitions that shape agent behavior.
prompts
Integrations
External business systems and MCP tool servers.
- MCP client · Backend client connecting orchestration to MCP servers.
app/mcp_client.py - MCP servers · CRM and WhatsApp capability exposed through MCP-style servers.
mcp_servers - Business systems · CRM, billing, fiscal, and WhatsApp integration layer.
integrations - WAHA client · Outbound WhatsApp client used to send messages back to users.
app/services/waha_client.py
State
Database, deduplication, checkpoints, and vector memory.
- Request dedup · Transport-level duplicate suppression keyed by request_id.
app/repositories/dedup.py - Thread state · Conversation identity and checkpoint state keyed by thread_id.
initdb/005_thread_interventions.sql - PostgreSQL · Persistent store for dedup, agency schema, checkpoints, and audit-related state.
initdb - Qdrant · Vector store for RAG and long-term memory.
- Model backends · Ollama and Groq inference backends used behind the orchestration layer.
Ops
Logging, watchdogs, deployment, and verification utilities.
- Ops hooks · Async logging and watchdog health behavior.
app/logging_async.py - Compose deploy · Docker Compose and container build setup for the self-hosted stack.
docker-compose.yml - Checks · Dependency, route, sustention, and watchdog verification scripts/tests.
tests
Connections 29
- Streamlit UI → Dashboards: renders · The Streamlit entrypoint hosts the dashboard pages.
- Streamlit UI → API client: calls · UI actions go through the API client.
- API client → API routes: requests · Thin client sends user traffic to the backend ingress.
- API app → API routes: mounts · Primary API app exposes the regular routes.
- API app → WAHA webhook: mounts · Primary API app exposes the WAHA webhook routes.
- API app → Config: loads · Runtime configuration is centralized here.
- API app → Dependencies: wires · Dependency injection and services are assembled here.
- API routes → Orchestrator: enters · Regular chat/control requests hand off to orchestration.
- WAHA webhook → Request dedup: checks · Webhook traffic is deduplicated by request_id.
- WAHA webhook → Orchestrator: dispatches · Authorized WAHA events are forwarded into the orchestration graph.
- Request dedup → PostgreSQL: persists · Dedup state is stored in PostgreSQL.
- Orchestrator → Workflow engine: runs · The graph is executed asynchronously in background tasks.
- Orchestrator → Thread state: tracks · Conversation identity and checkpoints follow thread_id.
- Workflow engine → Agent base: invokes · Orchestration executes agent contracts.
- Agent base → Role agents: specializes · Specialized role agents implement the shared base.
- Role agents → Prompts: uses · Behavior is partly defined by prompt assets.
- Role agents → MCP client: tools · Agents access external capabilities through the MCP bridge.
- MCP client → MCP servers: connects · Client reaches the CRM and WhatsApp tool servers.
- MCP servers → Business systems: exposes · Tool servers wrap business system integrations.
- Orchestrator → WAHA client: responds · Outbound WhatsApp messages are sent back through the WAHA client.
- Orchestrator → Model backends: queries · LLM execution is abstracted behind the orchestration layer.
- Orchestrator → Qdrant: retrieves · Vector memory and RAG lookups use Qdrant.
- Ops hooks → API app: supports · Operational hooks cover async logging and health monitoring.
- Compose deploy → API app: runs · Compose and container files define the backend service.
- Compose deploy → Streamlit UI: runs · Compose and container files define the frontend service.
- Compose deploy → PostgreSQL: persists · Persistent database volumes back the stack.
- Compose deploy → Qdrant: persists · Persistent vector-store volumes back the stack.
- Checks → API routes: verifies · Repository checks validate routing behavior.
- Checks → Ops hooks: verifies · Repository checks validate watchdog behavior.