# salam59/django-slack-rag-bot architecture

> Architecture diagram of the GitHub repository salam59/django-slack-rag-bot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/salam59/django-slack-rag-bot
- Repository: https://github.com/salam59/django-slack-rag-bot

## Overview

##### Project type and purpose
- This is a monolithic Django-based backend service exposing a Slack Bot interface. It integrates a Retrieval-Augmented Generation (RAG) pipeline to answer questions about system design by extracting images and text from a PDF, embedding that content, indexing it in MongoDB, retrieving relevant chunks, and composing answers with an LLM.

##### File structure
- Top-level:
- README.md, requirements.txt, docker-compose.yaml, manage.py, db.sqlite3 (local fallback)
- src/app: the Django project (“settings.py”, “urls.py”, ASGI/WSGI entrypoints, Celery setup)
- src/bot: a Django app housing models, views (incoming Slack event endpoint), tasks (PDF parsing, indexing, RAG retrieval), utils, admin, tests, migrations
- src/slacky: Slack messaging helper library
- src/helpers: environment loader
- src/data: static assets & PDF sources, including the system-design PDF and images used in answers
- src/notebooks: exploratory Jupyter workflows (pdf mining, setup)

##### README for additional insights
- Tech stack: Django, Cloudflare Tunnels, Celery (broker + result backend via env vars), pdfminer.six for PDF extraction, jina.ai for embeddings, Mistral AI for generation, MongoDB Atlas with vector search, Slack API

##### Diagramming the architecture
a. Main components
- Slack platform (user encounters)
- Cloudflare Tunnel (tunnels external traffic to local/hosted Django)
- Django Web Server (ASGI/WSGI)
- Celery Worker(s)
- Celery Broker & Result Backend (e.g., Redis/RabbitMQ as per CELERY_* vars)
- PDF Processor (pdfminer.six)
- Embedding Service (jina.ai embedding API)
- Vector Store (MongoDB Atlas with vector search index)
- LLM Generation Service (Mistral AI API)
- Data Storage (src/data folder assets, SQLite fallback)
- Slacky Helper (utility in code to abstract Slack message formatting)

b. Relationships and interactions
1. User sends a message in Slack → Slack Events API → Cloudflare Tunnel → Django “/slack/events/” endpoint (views.py)
2. Django view enqueues a Celery task (bot/tasks.py) for the RAG pipeline
3. Celery Worker pulls task:
i. If first-time or updated, parse PDF via pdfminer.six → produce text & images → send embedding requests to jina.ai → index vectors into MongoDB
ii. For a query: embed query text via jina.ai → perform vector search in MongoDB → retrieve top-k chunks (text + images)
iii. Send retrieved context + user query to Mistral AI → receive generated answer
4. Celery Worker uses Slacky.messages to post response (text + images) back to Slack via Slack Web API
5. Django responds to Slack Events API call with HTTP 200 to acknowledge

c. Architectural patterns & design principles
- Separation of concerns: Web/API layer vs. background processing (Celery)
- RAG pipeline broken into modular tasks: PDF parsing, embedding, indexing, retrieval, generation
- Externalized services: LLM, embedding service, vector DB, Slack API — keep core logic thin
- Use of Django apps to isolate bot logic from project settings

d. Key technologies & frameworks
- Django (REST endpoint, settings management, URL routing)
- Celery (task queue, broker & result backend via environment)
- pdfminer.six (PDF text + image extraction)
- jina.ai (text embedding service)
- MongoDB Atlas (vector search index)
- Mistral AI (LLM completion API)
- Cloudflare Tunnel (expose local dev server)
- Slack API / Slacky.messages (message and file upload flows)

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Request-Response Layer"
        extSlack["Slack Platform"]:::external
        extTunnel["Cloudflare Tunnel"]:::external
        DjangoServer["Django Web Server"]:::internal
        SlackEndpoint["Slack Events Endpoint"]:::internal
    end

    extSlack --> extTunnel --> DjangoServer --> SlackEndpoint
    SlackEndpoint -->|"enqueue task"| CeleryBroker

    subgraph "Async Processing Layer"
        CeleryBroker["Celery Broker & Backend"]:::external
        CeleryWorker["Celery Worker"]:::internal
    end

    CeleryBroker --> CeleryWorker

    subgraph "RAG Pipeline"
        PDFProcessor["PDF Processor"]:::internal
        Models["Data Models & Metadata"]:::internal
        EmbeddingService["Embedding Service (jina.ai)"]:::external
        VectorStore["Vector Store (MongoDB Atlas)"]:::datastore
        RAGTasks["RAG Pipeline Tasks"]:::internal
        LLMService["LLM Service (Mistral AI)"]:::external
        Slacky["Slacky Helper"]:::internal
        SlackAPI["Slack Web API"]:::external
        DataStorage["Static Data Storage"]:::datastore
    end

    CeleryWorker --> PDFProcessor
    PDFProcessor --> Models
    Models --> VectorStore
    PDFProcessor --> DataStorage
    CeleryWorker --> RAGTasks
    RAGTasks --> EmbeddingService
    RAGTasks --> VectorStore
    RAGTasks --> LLMService
    LLMService --> Slacky
    Slacky --> SlackAPI

    Env["Environment Loader"]:::internal
    CeleryConfig["Celery Configuration"]:::internal
    Env --> DjangoServer
    Env --> CeleryWorker
    CeleryConfig --> CeleryWorker

    Notebooks["Exploratory Notebooks"]:::external
    Notebooks -.-> PDFProcessor

    click DjangoServer "https://github.com/salam59/django-slack-rag-bot/tree/main/src/app/"
    click SlackEndpoint "https://github.com/salam59/django-slack-rag-bot/blob/main/src/bot/views.py"
    click CeleryConfig "https://github.com/salam59/django-slack-rag-bot/blob/main/src/app/celery.py"
    click RAGTasks "https://github.com/salam59/django-slack-rag-bot/blob/main/src/bot/tasks.py"
    click PDFProcessor "https://github.com/salam59/django-slack-rag-bot/blob/main/src/bot/utils.py"
    click Models "https://github.com/salam59/django-slack-rag-bot/blob/main/src/bot/models.py"
    click BotApp "https://github.com/salam59/django-slack-rag-bot/blob/main/src/bot/apps.py"
    click BotApp "https://github.com/salam59/django-slack-rag-bot/blob/main/src/bot/admin.py"
    click Slacky "https://github.com/salam59/django-slack-rag-bot/blob/main/src/slacky/messages.py"
    click Env "https://github.com/salam59/django-slack-rag-bot/blob/main/src/helpers/env.py"
    click DataStorage "https://github.com/salam59/django-slack-rag-bot/blob/main/src/data/system-design.pdf"
    click DataStorage "https://github.com/salam59/django-slack-rag-bot/tree/main/src/data/readme_images/"
    click DataStorage "https://github.com/salam59/django-slack-rag-bot/tree/main/src/data/other_images/"
    click Notebooks "https://github.com/salam59/django-slack-rag-bot/tree/main/src/notebooks/"

    classDef internal fill:#D6EAF8,stroke:#5499C7
    classDef external fill:#FCF3CF,stroke:#F4D03F
    classDef datastore fill:#E8DAEF,stroke:#BB8FCE
```
