# heygaia/gaia architecture

> Architecture diagram of the GitHub repository heygaia/gaia, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/heygaia/gaia
- Repository: https://github.com/heygaia/gaia

## Overview

##### Project Type and Purpose
- This is a full-stack, monorepo-style application (GAIA) that provides a proactive AI assistant via a Next.js-based frontend and a FastAPI-based backend, supported by background workers, multiple databases, and external integrations.
- The frontend serves both landing pages and the authenticated “main” UI; the backend exposes a REST/WebSocket API, schedules async tasks, orchestrates AI calls, and persists data.

##### File Structure Analysis
- Top level:
- frontend/ – Next.js (React + TypeScript + Tailwind CSS) app
- backend/ – FastAPI Python service, ARQ worker, Celery‐style tasks
- docker-compose*.yml, Dockerfiles – container orchestration
- docs/, SYSTEM_OVERVIEW.md, PUBSUB_DETAILED.md – architectural docs
- backend/app:
- api/v1/router – API endpoints organized by resource (chat, calendar, mail, etc.)
- db – data layer wrappers for MongoDB, PostgreSQL, Redis, RabbitMQ, ChromaDB
- langchain – LLM orchestration layer, prompts, tools, graph building
- services, utils, tasks, worker_node – business logic, helpers, async workers
- frontend/src:
- app/(landing) + app/(main) – Next.js routing groups; public vs. authenticated
- components, features, hooks, services, redux – UI, domain logic, state management
- Deployment: Uses Docker Compose for local/dev and a separate prod compose file.

a. Frontend Client
- Next.js server + React SSR/CSR
- Public landing pages and authenticated SPA under same domain
- WebSocket client for real-time notifications
b. API Layer
- FastAPI service (HTTP REST + WebSocket)
- Authentication, rate limiting, dependency injection
c. Background Workers
- ARQ worker (Redis-backed Python jobs)
- Worker_node processes (email workflows, custom workflows)
d. Databases & Caches
- MongoDB: primary NoSQL store (collections, indices)
- PostgreSQL: relational data (e.g., user subscriptions, analytics)
- Redis: cache, rate-limit store, ARQ queue
- RabbitMQ: pub/sub for inter-service events (e.g., notifications)
- ChromaDB: vector store for embeddings/memory
e. External Services
- OAuth providers (Google, Microsoft)
- Cloudinary (image hosting)
- Email provider & webhooks
- Payment gateway (Razorpay)
- OpenAI / other LLM endpoints via LangChain

##### Relationships & Data Flow
- User → Next.js UI → FastAPI HTTP endpoints (login, fetch data, send messages)
- UI WebSocket ↔ FastAPI for live notifications
- FastAPI → publishes tasks to Redis (ARQ) or RabbitMQ → processed by worker_node
- Workers call LangChain core → LLMs; store embeddings in Chroma; persist results
- FastAPI reads/writes to MongoDB/Postgres/Redis/Chroma as needed
- FastAPI integrates with OAuth, Cloudinary, payment, email providers via service clients

##### Architectural Patterns & Principles
- Separation of concerns: UI / API / Async workers / Data stores
- Modular routers & service layer (thin controllers, fat services)
- Dependency injection in FastAPI for config, auth, rate limits
- Event-driven design: pub/sub for cross-component notifications
- Hexagonal style for LangChain tools (plugins)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Client (Browser)"
        NextJS["Next.js SSR/CSR"]:::frontend
        WSClient["WebSocket Client"]:::frontend
    end

    subgraph "API Server"
        FastAPI["FastAPI REST + WS"]:::api
        subgraph "Routers"
            Router["Routes (chat, mail, calendar, etc.)"]:::api
            WSRoute["WebSocket Router"]:::api
        end
        subgraph "Auth & DI"
            OAuthDep["OAuth Dependencies"]:::api
            RateLimit["Rate Limiting"]:::api
            AuthMW["Auth Middleware"]:::api
        end
    end

    subgraph "Background Workers"
        ARQ["ARQ Worker"]:::async
        WorkerNode["Worker Node"]:::async
        subgraph "Task Definitions"
            MailTasks["mail_tasks"]:::async
            RemTasks["reminder_tasks"]:::async
        end
    end

    subgraph "LangChain Orchestration"
        Agent["Agent Core"]:::async
        GraphBuilder["Graph Builder"]:::async
        Prompts["Prompts"]:::async
        Tools["Tools"]:::async
    end

    subgraph "Business Services"
        ChatSvc["Chat Service"]:::api
        MailSvc["Mail Service"]:::api
        CalSvc["Calendar Service"]:::api
        NotifSvc["Notification Service"]:::api
        Payments["Payments Service"]:::api
        OAuthSvc["OAuth Service"]:::api
    end

    subgraph "Utilities"
        AuthUtils["Auth Utils"]:::api
        Orchestrator["Notification Orchestrator"]:::api
        EmbeddingUtils["Embedding Utils"]:::api
        CalUtils["Calendar Utils"]:::api
    end

    subgraph "Data Stores"
        MongoDB["MongoDB"]:::db
        Postgres["PostgreSQL"]:::db
        Redis["Redis"]:::db
        RabbitMQ["RabbitMQ"]:::db
        Chroma["ChromaDB"]:::db
    end

    subgraph "External Services"
        OAuthExt["OAuth Providers"]:::external
        Cloudinary["Cloudinary"]:::external
        EmailExt["Email Provider & Webhooks"]:::external
        PaymentExt["Payment Gateway"]:::external
        LLM["OpenAI/Other LLMs"]:::external
    end

    %% Connections
    NextJS -->|HTTP| FastAPI
    NextJS -->|WebSocket| FastAPI
    WSClient -->|WebSocket| FastAPI

    FastAPI --> Router
    FastAPI --> WSRoute
    FastAPI --> OAuthDep
    FastAPI --> RateLimit
    FastAPI --> AuthMW

    FastAPI -->|enqueue task| Redis
    Redis --> ARQ
    ARQ --> MailTasks
    ARQ --> RemTasks

    FastAPI -->|publish event| RabbitMQ
    RabbitMQ --> WorkerNode

    WorkerNode --> Agent
    WorkerNode --> GraphBuilder
    WorkerNode --> Prompts
    WorkerNode --> Tools

    Agent --> Chroma
    GraphBuilder --> Chroma

    FastAPI --> MongoDB
    FastAPI --> Postgres
    FastAPI --> Redis
    FastAPI --> Chroma

    FastAPI --> OAuthExt
    FastAPI --> Cloudinary
    FastAPI --> EmailExt
    FastAPI --> PaymentExt
    FastAPI --> LLM

    Router --> ChatSvc
    Router --> MailSvc
    Router --> CalSvc
    Router --> NotifSvc
    Router --> Payments
    Router --> OAuthSvc

    AuthMW --> AuthUtils
    NotifSvc --> Orchestrator
    Agent --> EmbeddingUtils
    CalSvc --> CalUtils

    %% Click Events
    click NextJS "https://github.com/heygaia/gaia/tree/master/frontend/src/app/(landing)"
    click NextJS "https://github.com/heygaia/gaia/tree/master/frontend/src/app/(main)"
    click NextJS "https://github.com/heygaia/gaia/blob/master/frontend/next.config.mjs"
    click NextJS "https://github.com/heygaia/gaia/blob/master/frontend/src/layouts/ProvidersLayout.tsx"

    click WSClient "https://github.com/heygaia/gaia/blob/master/frontend/src/features/notification/hooks/useNotificationWebSocket.ts"
    click WSClient "https://github.com/heygaia/gaia/blob/master/frontend/src/services/api/notifications.ts"

    click FastAPI "https://github.com/heygaia/gaia/blob/master/backend/app/main.py"
    click Router "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/routes.py"
    click WSRoute "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/router/websocket.py"
    click Router "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/router/chat.py"
    click Router "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/router/mail.py"
    click Router "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/router/calendar.py"

    click OAuthDep "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/dependencies/oauth_dependencies.py"
    click RateLimit "https://github.com/heygaia/gaia/blob/master/backend/app/config/rate_limits.py"
    click AuthMW "https://github.com/heygaia/gaia/blob/master/backend/app/middleware/auth_middleware.py"

    click ARQ "https://github.com/heygaia/gaia/blob/master/backend/app/arq_worker.py"
    click WorkerNode "https://github.com/heygaia/gaia/blob/master/backend/app/worker.py"
    click WorkerNode "https://github.com/heygaia/gaia/blob/master/backend/app/worker_node/process_email.py"
    click WorkerNode "https://github.com/heygaia/gaia/blob/master/backend/app/worker_node/process_workflow.py"

    click MailTasks "https://github.com/heygaia/gaia/blob/master/backend/app/tasks/mail_tasks.py"
    click RemTasks "https://github.com/heygaia/gaia/blob/master/backend/app/tasks/reminder_tasks.py"

    click MongoDB "https://github.com/heygaia/gaia/blob/master/backend/app/db/mongodb/mongodb.py"
    click MongoDB "https://github.com/heygaia/gaia/blob/master/backend/app/db/mongodb/collections.py"
    click Postgres "https://github.com/heygaia/gaia/blob/master/backend/app/db/postgresql.py"
    click Redis "https://github.com/heygaia/gaia/blob/master/backend/app/db/redis.py"
    click RabbitMQ "https://github.com/heygaia/gaia/blob/master/backend/app/db/rabbitmq.py"
    click Chroma "https://github.com/heygaia/gaia/blob/master/backend/app/db/chromadb.py"

    click Agent "https://github.com/heygaia/gaia/blob/master/backend/app/langchain/core/agent.py"
    click GraphBuilder "https://github.com/heygaia/gaia/blob/master/backend/app/langchain/core/graph_builder/build_graph.py"
    click Prompts "https://github.com/heygaia/gaia/blob/master/backend/app/langchain/prompts/*.py"
    click Tools "https://github.com/heygaia/gaia/blob/master/backend/app/langchain/tools/*.py"

    click ChatSvc "https://github.com/heygaia/gaia/blob/master/backend/app/services/chat_service.py"
    click MailSvc "https://github.com/heygaia/gaia/blob/master/backend/app/services/mail_service.py"
    click CalSvc "https://github.com/heygaia/gaia/blob/master/backend/app/services/calendar_service.py"
    click NotifSvc "https://github.com/heygaia/gaia/blob/master/backend/app/services/notification_service.py"
    click Payments "https://github.com/heygaia/gaia/blob/master/backend/app/services/payments/client.py"
    click Payments "https://github.com/heygaia/gaia/blob/master/backend/app/services/payments/core.py"
    click Payments "https://github.com/heygaia/gaia/blob/master/backend/app/services/payments/webhooks.py"
    click OAuthSvc "https://github.com/heygaia/gaia/blob/master/backend/app/services/oauth_service.py"

    click AuthUtils "https://github.com/heygaia/gaia/blob/master/backend/app/utils/auth_utils.py"
    click Orchestrator "https://github.com/heygaia/gaia/blob/master/backend/app/utils/notification/orchestrator.py"
    click EmbeddingUtils "https://github.com/heygaia/gaia/blob/master/backend/app/utils/embedding_utils.py"
    click CalUtils "https://github.com/heygaia/gaia/blob/master/backend/app/utils/calendar_utils.py"

    click OAuthExt "https://github.com/heygaia/gaia/blob/master/backend/app/config/oauth_config.py"
    click OAuthSvc "https://github.com/heygaia/gaia/blob/master/backend/app/services/oauth_service.py"
    click Cloudinary "https://github.com/heygaia/gaia/blob/master/backend/app/config/cloudinary.py"
    click EmailExt "https://github.com/heygaia/gaia/blob/master/backend/app/api/v1/router/mail_webhook.py"
    click EmailExt "https://github.com/heygaia/gaia/blob/master/backend/app/services/mail_webhook_service.py"
    click PaymentExt "https://github.com/heygaia/gaia/blob/master/backend/app/services/payments/client.py"
    click PaymentExt "https://github.com/heygaia/gaia/blob/master/backend/app/services/payments/webhooks.py"
    click LLM "https://github.com/heygaia/gaia/blob/master/backend/app/langchain/llm/client.py"
    click LLM "https://github.com/heygaia/gaia/blob/master/backend/app/langchain/llm/chatbot.py"

    %% Styles
    classDef frontend fill:#cce5ff,stroke:#0066cc,stroke-width:1px
    classDef api fill:#d4edda,stroke:#28a745,stroke-width:1px
    classDef async fill:#fff3cd,stroke:#ffca28,stroke-width:1px
    classDef db fill:#e2d5f7,stroke:#8a2be2,stroke-width:1px
    classDef external fill:#ebebeb,stroke:#6c757d,stroke-width:1px
```
