# rockie/gurubase architecture

> Architecture diagram of the GitHub repository rockie/gurubase, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rockie/gurubase
- Repository: https://github.com/rockie/gurubase

## Overview

##### Project type and purpose
- Gurubase is a full-stack, self-hostable, Retrieval-Augmented Generation (RAG) microservices system that lets you create AI-powered Q&A assistants (“Gurus”) on any topic.
- It consists of a Next.js 14 + TailwindCSS frontend, a Django REST backend with Celery workers, and several supporting services (PostgreSQL, Redis, RabbitMQ, Milvus).

##### File structure
- Top-level directories:
- .dev – local and self-hosted Docker Compose configs and Nginx setup
- .github – CI/CD workflows for building and publishing Docker images and running linters/tests
- src/gurubase-frontend – Next.js application with components, pages, Redux, hooks, styling, and build configs
- src/gurubase-backend – Django project with apps (accounts, analytics, core), Celery tasks, management scripts, migrations, Dockerfile, and ARCHITECTURE.md
- Root docker-compose.yml and start scripts orchestrate the microservices locally or in staging/self-hosted environments.

##### README for architectural insights
- It explicitly calls out a microservices architecture deployed via Docker Compose.
- Key services:
- Frontend: Next.js (React SSR/SSG)
- Backend API: Django REST Framework, ASGI for async tasks
- Vector store: Milvus for similarity search
- Message queue: RabbitMQ for Celery tasks
- Cache: Redis for rate-limiting, caching embedding lookups, session data
- Database: PostgreSQL for relational data (users, settings, crawl state)
- External services: LLM API providers, GitHub / YouTube / Web crawlers, OAuth providers, Sentry

## Diagram (Mermaid)

```mermaid
graph TD
  %% Presentation Layer
  subgraph "Presentation Layer"
    Browser["User Browser"]:::external
    Frontend["Next.js Frontend"]:::frontend
  end

  %% API Layer
  subgraph "API Layer"
    Ingress["Nginx Ingress"]:::infra
    APIApp["Django REST API"]:::backend
    ASGI["ASGI Entry (asgi.py)"]:::backend
    WSGI["WSGI Entry (wsgi.py)"]:::backend
    Settings["Settings (settings.py)"]:::backend
    URLs["URL Routing (urls.py)"]:::backend
    Accounts["Accounts App"]:::backend
    AnalyticsApp["Analytics App"]:::backend
    CoreApp["Core App"]:::backend
    CeleryConfig["Celery Config (celery.py)"]:::backend
  end

  %% Worker Layer
  subgraph "Worker Layer"
    CeleryWorker["Celery Worker"]:::worker
    CeleryBeat["Celery Beat"]:::worker
    Tasks["RAG & Indexing Tasks"]:::worker
    PeriodicCmds["Periodic Commands"]:::worker
    DiscordListener["Discord Listener"]:::worker
    GitHubHandler["GitHub Crawler"]:::worker
    MilvusUtils["Milvus Utils"]:::worker
  end

  %% Storage Layer
  subgraph "Storage Layer"
    Postgres["PostgreSQL"]:::storage
    Redis["Redis"]:::storage
    Milvus["Milvus Vector Store"]:::storage
  end

  %% Infrastructure & Orchestration
  subgraph "Infrastructure"
    RabbitMQ["RabbitMQ"]:::infra
    DockerCompose["Docker Compose"]:::infra
    Workflows["GitHub Actions"]:::infra
  end

  %% External Integrations
  subgraph "External Services"
    OpenAI["LLM API Providers"]:::external
    GitHubAPI["GitHub API"]:::external
    YouTubeAPI["YouTube API"]:::external
    OAuth["OAuth Providers"]:::external
    Sentry["Sentry"]:::external
  end

  %% Connections
  Browser --> Frontend
  Frontend --> Ingress
  Ingress --> APIApp
  APIApp --> ASGI
  APIApp --> WSGI
  APIApp --> Settings
  APIApp --> URLs
  APIApp --> Accounts
  APIApp --> AnalyticsApp
  APIApp --> CoreApp
  APIApp --> Redis
  APIApp --> Postgres
  CoreApp --> Milvus
  CoreApp --> RabbitMQ
  RabbitMQ --> CeleryWorker
  CeleryWorker --> Tasks
  CeleryWorker --> GitHubHandler
  CeleryWorker --> DiscordListener
  CeleryBeat --> PeriodicCmds
  PeriodicCmds --> Milvus
  PeriodicCmds --> Redis
  PeriodicCmds --> Postgres
  CeleryWorker --> OpenAI
  AnalyticsApp --> Postgres
  AnalyticsApp --> Redis
  DockerCompose --> Ingress
  DockerCompose --> APIApp
  DockerCompose --> CeleryWorker
  DockerCompose --> CeleryBeat
  DockerCompose --> Postgres
  DockerCompose --> Redis
  DockerCompose --> Milvus
  DockerCompose --> RabbitMQ
  Workflows --> DockerCompose

  %% Click Events
  click Frontend "https://github.com/rockie/gurubase/tree/develop/src/gurubase-frontend"
  click APIApp "https://github.com/rockie/gurubase/tree/develop/src/gurubase-backend/backend/backend"
  click ASGI "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/backend/asgi.py"
  click WSGI "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/backend/wsgi.py"
  click Settings "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/backend/settings.py"
  click URLs "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/backend/urls.py"
  click Accounts "https://github.com/rockie/gurubase/tree/develop/src/gurubase-backend/backend/accounts"
  click AnalyticsApp "https://github.com/rockie/gurubase/tree/develop/src/gurubase-backend/backend/analytics"
  click CoreApp "https://github.com/rockie/gurubase/tree/develop/src/gurubase-backend/backend/core"
  click CeleryConfig "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/backend/celery.py"
  click Tasks "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/core/tasks.py"
  click PeriodicCmds "https://github.com/rockie/gurubase/tree/develop/src/gurubase-backend/backend/core/management/commands"
  click DiscordListener "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/core/management/commands/discordListener.py"
  click GitHubHandler "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/core/github_handler.py"
  click MilvusUtils "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/backend/core/milvus_utils.py"
  click CeleryWorker "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/start_scripts/start_celery_worker.sh"
  click CeleryBeat "https://github.com/rockie/gurubase/blob/develop/src/gurubase-backend/start_scripts/start_celery_beat.sh"
  click Ingress "https://github.com/rockie/gurubase/blob/develop/.dev/nginx/nginx.conf"
  click DockerCompose "https://github.com/rockie/gurubase/blob/develop/docker-compose.yml"
  click Workflows "https://github.com/rockie/gurubase/tree/develop/.github/workflows"

  %% Styles
  classDef frontend fill:#ffdddd,stroke:#ff5555;
  classDef backend fill:#ddffdd,stroke:#55aa55;
  classDef worker fill:#ddddff,stroke:#5555ff;
  classDef storage fill:#ffddff,stroke:#aa55aa;
  classDef infra fill:#ffffdd,stroke:#aaaa55;
  classDef external fill:#ddffff,stroke:#55aaaa;
```
