How heygaia/gaia works
Project Type and Purpose
• This is a full-stack, monorepo-style application (GAIA) that provides a proactive AI assistant via a Next.js-based frontend and a FastAPI-based backend, supported by background workers, multiple databases, and external integrations.
• The frontend serves both landing pages and the authenticated “main” UI; the backend exposes a REST/WebSocket API, schedules async tasks, orchestrates AI calls, and persists data.
File Structure Analysis
• Top level:
• frontend/ – Next.js (React + TypeScript + Tailwind CSS) app
• backend/ – FastAPI Python service, ARQ worker, Celery‐style tasks
• docker-compose*.yml, Dockerfiles – container orchestration
• docs/, SYSTEM_OVERVIEW.md, PUBSUB_DETAILED.md – architectural docs
• backend/app:
• api/v1/router – API endpoints organized by resource (chat, calendar, mail, etc.)
• db – data layer wrappers for MongoDB, PostgreSQL, Redis, RabbitMQ, ChromaDB
• langchain – LLM orchestration layer, prompts, tools, graph building
• services, utils, tasks, worker_node – business logic, helpers, async workers
• frontend/src:
• app/(landing) + app/(main) – Next.js routing groups; public vs. authenticated
• components, features, hooks, services, redux – UI, domain logic, state management
• Deployment: Uses Docker Compose for local/dev and a separate prod compose file.
a. Frontend Client
• Next.js server + React SSR/CSR
• Public landing pages and authenticated SPA under same domain
• WebSocket client for real-time notifications
b. API Layer
• FastAPI service (HTTP REST + WebSocket)
• Authentication, rate limiting, dependency injection
c. Background Workers
• ARQ worker (Redis-backed Python jobs)
• Worker_node processes (email workflows, custom workflows)
d. Databases & Caches
• MongoDB: primary NoSQL store (collections, indices)
• PostgreSQL: relational data (e.g., user subscriptions, analytics)
• Redis: cache, rate-limit store, ARQ queue
• RabbitMQ: pub/sub for inter-service events (e.g., notifications)
• ChromaDB: vector store for embeddings/memory
e. External Services
• OAuth providers (Google, Microsoft)
• Cloudinary (image hosting)
• Email provider & webhooks
• Payment gateway (Razorpay)
• OpenAI / other LLM endpoints via LangChain
Relationships & Data Flow
• User → Next.js UI → FastAPI HTTP endpoints (login, fetch data, send messages)
• UI WebSocket ↔ FastAPI for live notifications
• FastAPI → publishes tasks to Redis (ARQ) or RabbitMQ → processed by worker_node
• Workers call LangChain core → LLMs; store embeddings in Chroma; persist results
• FastAPI reads/writes to MongoDB/Postgres/Redis/Chroma as needed
• FastAPI integrates with OAuth, Cloudinary, payment, email providers via service clients
Architectural Patterns & Principles
• Separation of concerns: UI / API / Async workers / Data stores
• Modular routers & service layer (thin controllers, fat services)
• Dependency injection in FastAPI for config, auth, rate limits
• Event-driven design: pub/sub for cross-component notifications
• Hexagonal style for LangChain tools (plugins)