# datarohit/agentsphere architecture

> Architecture diagram of the GitHub repository datarohit/agentsphere, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datarohit/agentsphere
- Repository: https://github.com/datarohit/agentsphere

## Overview

##### Project Type and Purpose
- Type: Full-stack web application
- Purpose: A dynamic AI platform where users manage AI agents (single & group), run conversations in real time via WebSockets, integrate external tools (MCP servers), and collaborate in organizations.
- Tech Highlights:
– Frontend: Next.js + React + Tailwind + Redux Toolkit
– Backend: Django 5 + Django REST Framework + Channels (WebSockets)
– Infrastructure: Docker Compose, Nginx, PostgreSQL, Redis, Celery, Vault, MinIO

##### File Structure
- Top-level dirs:
– backend/: Django “monolithic” code broken into Django apps (agents, chats, conversation, llms, organization, tools, users, common, contrib)
– frontend/: Next.js App Router pages under /app plus reusable UI components; Redux slices in store/
– compose/: Docker-compose fragments for each service (django, nextjs, nginx, postgres)
– docker-compose.yml: Orchestrates all containers
– makefile: Dev commands
- Patterns & config:
– backend/config/: Django settings, ASGI (for WebSockets), Celery, routing, OpenAPI
– Celery tasks (in apps/chats/tasks, conversation/tasks, organization/tasks)
– WebSockets via Django Channels (apps/conversation/consumers + routing)
– Vault and MinIO clients under apps/common/utils
– Frontend state management via Redux Toolkit (store), fetch calls to REST API, WebSocket clients for live chat

##### README Insights
- Architecture hints:
– Docker Compose services: Django app, Next.js app, Nginx reverse proxy, PostgreSQL, Redis, Vault, MinIO, Celery worker & beat & flower, Mailpit, PgAdmin
– Security: JWT auth, Vault for API keys, CSRF protection
– Real-time: WebSocket via Django Channels
- External dependencies: LLM providers (Gemini), Email (Mailpit), Monitoring (Sentry, Silk)

##### Main Components & Interactions
a. Components
1. Client / Browser
– Next.js frontend (React pages, UI components, Redux store, WebSocket client)
2. Reverse Proxy
– Nginx (routes HTTP to Next.js or Django, handles SSL if any)
3. Frontend Service
– Next.js server (Docker container)
4. Backend Service
– Django ASGI app (Docker container) exposing:
- REST API (Django REST Framework) on /api/v1/
- WebSocket endpoint (/ws/conversation/)
5. Task Queue
– Celery worker + Celery beat + Flower monitor
6. Datastore
– PostgreSQL (persistent data: users, agents, chats, orgs, tools)
7. Cache / Broker
– Redis (cache, Celery broker, Channels layer)
8. Secret Manager
– Vault (stores LLM API keys, other secrets)
9. Object Storage
– MinIO (S3-compatible for file uploads, avatars)
10. Auxiliary Services
– Mailpit (email dev server)
– PgAdmin (DB admin)
11. External Services
– LLM Provider (Gemini, others)
– MCP Tool Endpoints

b. Relationships & Data Flows
– Browser ↔ Nginx ↔ Next.js (HTTP)
– Browser WebSocket ↔ Nginx ↔ Django Channels
– Next.js ↔ Django REST (HTTP) for API calls
– Django ↔ PostgreSQL (ORM read/write)
– Django ↔ Redis (cache, Channels Pub/Sub, Celery broker)
– Django/Celery ↔ Vault (secrets lookup)
– Django ↔ MinIO (object storage)
– Celery workers consume tasks enqueued by Django for background processing (message deletion, chat summary generation, org cleanup)
– Django ↔ LLM Provider (external HTTP) to generate AI responses
– Mailpit captures outbound emails for activation, password reset
– PgAdmin connects to PostgreSQL for DB maintenance

c. Architectural Patterns & Principles
– Modular Monolith: Django apps isolate domain logic
– REST + WebSockets: Separate synchronous and asynchronous communication
– Event-driven: Celery tasks for offloaded workloads
– 12-factor: Configuration via env files, Dockerized services
– Secure by design: Vault-backed secrets, JWT, CSRF
– Component reuse: shadcn/ui + Radix UI for consistent frontend

d. Key Technologies & Frameworks
– Django, DRF, Channels, Celery, PostgreSQL, Redis, Vault, MinIO, Nginx, Next.js, React, Redux Toolkit, Tailwind CSS

##### Detailed Notes for the Engineer
– Show inter-service ports (e.g., Redis:6379, PostgreSQL:5432, API:8080, Next.js:3000)
– Illustrate Celery beat → task schedule → queue → worker
– Highlight that Channels uses Redis as channel layer for WebSocket group messaging
– Capture that Vault is read at app start or on demand for LLM API keys
– Indicate where static assets (CSS, JS) and media files are served (Next.js public folder, MinIO)
– Label environment config mounts (e.g., .env files injected into containers)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Presentation Layer
    subgraph "Presentation Layer" 
        direction TB
        Browser["Client Browser"]:::frontend
        subgraph "Next.js (Frontend Service)" 
            direction TB
            NextJS["Next.js App Server"]:::frontend
            UIComponents["UI Components"]:::frontend
            ReduxStore["Redux Store"]:::frontend
        end
    end

    %% API / Business Logic Layer
    subgraph "API / Business Logic Layer"
        direction TB
        DjangoASGI["Django ASGI App (REST + Channels)"]:::backend
        subgraph "Django Configuration" 
            direction TB
            ASGI["asgi.py"]:::backend
            Settings["settings.py"]:::backend
            Routing["routing.py"]:::backend
            WebsocketCfg["websocket.py"]:::backend
            OpenAPI["openapi.py"]:::backend
            URLs["urls.py"]:::backend
            CeleryApp["celery_app.py"]:::backend
        end
        subgraph "Django Apps" 
            direction TB
            Agents["agents"]:::backend
            Chats["chats"]:::backend
            ConversationApp["conversation"]:::backend
            LLMS["llms"]:::backend
            Org["organization"]:::backend
            Tools["tools"]:::backend
            Users["users"]:::backend
        end
        subgraph "WebSocket Routing"
            direction TB
            SessionConsumer["session_consumer.py"]:::backend
            ChannelsRouting["routing.py"]:::backend
        end
        subgraph "Background Tasks"
            direction TB
            TasksChats["chats/tasks"]:::async
            TasksConv["conversation/tasks"]:::async
            TasksOrg["organization/tasks"]:::async
        end
        VaultUtil["vault.py"]:::infra
        EmailUtil["email.py"]:::infra
    end

    %% Data / Async Layer
    subgraph "Data / Async Layer"
        direction TB
        Postgres["PostgreSQL"]:::data
        Redis["Redis"]:::data
        subgraph "Celery Cluster" 
            direction TB
            CeleryBeat["Celery Beat"]:::async
            CeleryWorker["Celery Worker"]:::async
            Flower["Flower Monitor"]:::infra
        end
        MinIO["MinIO (Object Storage)"]:::infra
    end

    %% Infrastructure & External Services
    subgraph "Infrastructure / External"
        direction TB
        Nginx["Nginx Reverse Proxy"]:::infra
        Vault["Vault (Secret Manager)"]:::infra
        Mailpit["Mailpit"]:::infra
        PgAdmin["PgAdmin"]:::infra
        LLMProvider["LLM Provider (Gemini)"]:::external
        MCPTools["MCP Tool Endpoints"]:::external
        DockerCompose["docker-compose.yml"]:::infra
        ComposeFragments["compose/*"]:::infra
    end

    %% Connections - HTTP
    Browser -->|"HTTP"| Nginx
    Nginx -->|"HTTP / Next.js:3000"| NextJS
    NextJS -->|"REST API /api/v1"| DjangoASGI
    DjangoASGI -->|"ORM / JDBC"| Postgres
    DjangoASGI -->|"Cache / Channel Layer"| Redis
    DjangoASGI -->|"Object Storage"| MinIO
    DjangoASGI -->|"Secrets"| Vault
    DjangoASGI -->|"Email"| Mailpit
    DjangoASGI -->|"LLM HTTP"| LLMProvider
    DjangoASGI -->|"MCP HTTP"| MCPTools

    %% WebSocket (dashed)
    Browser -.->|"WebSocket /ws/conversation"| Nginx
    Nginx -.->|"WS"| DjangoASGI
    DjangoASGI -.->|"Channels Pub/Sub"| Redis
    DjangoASGI --> SessionConsumer
    SessionConsumer --> ChannelsRouting

    %% Celery Tasks (dotted)
    DjangoASGI -.->|"Enqueue Task"| CeleryBeat
    DjangoASGI -.->|"Enqueue Task"| CeleryWorker
    CeleryBeat --> CeleryWorker
    CeleryWorker --> TasksChats
    CeleryWorker --> TasksConv
    CeleryWorker --> TasksOrg
    CeleryWorker -->|"DB / Cache"| Postgres
    CeleryWorker -->|"Cache / Broker"| Redis

    %% Orchestration
    DockerCompose --> NextJS
    DockerCompose --> DjangoASGI
    DockerCompose --> Nginx
    DockerCompose --> Postgres
    DockerCompose --> Redis
    DockerCompose --> Vault
    DockerCompose --> MinIO
    DockerCompose --> CeleryWorker
    DockerCompose --> CeleryBeat
    DockerCompose --> Flower
    DockerCompose --> Mailpit
    DockerCompose --> PgAdmin
    DockerCompose --> ComposeFragments

    %% Click Events
    click NextJS "https://github.com/datarohit/agentsphere/tree/master/frontend/app"
    click UIComponents "https://github.com/datarohit/agentsphere/tree/master/frontend/components"
    click ReduxStore "https://github.com/datarohit/agentsphere/blob/master/frontend/app/store/index.ts"
    click Nginx "https://github.com/datarohit/agentsphere/blob/master/compose/nginx/default.conf"
    click Nginx "https://github.com/datarohit/agentsphere/tree/master/compose/nginx/dockerfile"
    click ASGI "https://github.com/datarohit/agentsphere/blob/master/backend/config/asgi.py"
    click Settings "https://github.com/datarohit/agentsphere/blob/master/backend/config/settings.py"
    click Routing "https://github.com/datarohit/agentsphere/blob/master/backend/config/routing.py"
    click WebsocketCfg "https://github.com/datarohit/agentsphere/blob/master/backend/config/websocket.py"
    click OpenAPI "https://github.com/datarohit/agentsphere/blob/master/backend/config/openapi.py"
    click URLs "https://github.com/datarohit/agentsphere/blob/master/backend/config/urls.py"
    click Agents "https://github.com/datarohit/agentsphere/tree/master/backend/apps/agents"
    click Chats "https://github.com/datarohit/agentsphere/tree/master/backend/apps/chats"
    click ConversationApp "https://github.com/datarohit/agentsphere/tree/master/backend/apps/conversation"
    click LLMS "https://github.com/datarohit/agentsphere/tree/master/backend/apps/llms"
    click Org "https://github.com/datarohit/agentsphere/tree/master/backend/apps/organization"
    click Tools "https://github.com/datarohit/agentsphere/tree/master/backend/apps/tools"
    click Users "https://github.com/datarohit/agentsphere/tree/master/backend/apps/users"
    click SessionConsumer "https://github.com/datarohit/agentsphere/blob/master/backend/apps/conversation/consumers/session_consumer.py"
    click ChannelsRouting "https://github.com/datarohit/agentsphere/blob/master/backend/apps/conversation/routing.py"
    click CeleryApp "https://github.com/datarohit/agentsphere/blob/master/backend/config/celery_app.py"
    click TasksChats "https://github.com/datarohit/agentsphere/tree/master/backend/apps/chats/tasks"
    click TasksConv "https://github.com/datarohit/agentsphere/tree/master/backend/apps/conversation/tasks"
    click TasksOrg "https://github.com/datarohit/agentsphere/tree/master/backend/apps/organization/tasks"
    click VaultUtil "https://github.com/datarohit/agentsphere/blob/master/backend/apps/common/utils/vault.py"
    click EmailUtil "https://github.com/datarohit/agentsphere/blob/master/backend/apps/common/utils/email.py"
    click DockerCompose "https://github.com/datarohit/agentsphere/blob/master/docker-compose.yml"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose/django"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose/nextjs"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose/postgres"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose/redis"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose/nginx"
    click ComposeFragments "https://github.com/datarohit/agentsphere/tree/master/compose/postgres/maintenance"

    %% Styles
    classDef frontend fill:#E0F7FA,stroke:#006064,color:#006064
    classDef backend fill:#E8F5E9,stroke:#2E7D32,color:#2E7D32
    classDef data fill:#FFF3E0,stroke:#EF6C00,color:#EF6C00
    classDef async fill:#FFE0B2,stroke:#FB8C00,color:#BF360C
    classDef infra fill:#ECEFF1,stroke:#546E7A,color:#37474F
    classDef external fill:#F3E5F5,stroke:#8E24AA,color:#6A1B9A
```
