# deep-diver/llm-as-chatbot architecture

> Architecture diagram of the GitHub repository deep-diver/llm-as-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deep-diver/llm-as-chatbot
- Repository: https://github.com/deep-diver/llm-as-chatbot

## Overview

This repository serves instruction-tuned language models as chatbots through two front ends: a Gradio application and a Discord bot. Both rely on model selection/loading, model-specific conversation formatting, and text generation; search can augment answers. The graph preserves the documented model zoo and both user-facing routes. Some subsystem links are based on the SOURCE INDEX or README rather than complete excerpts, so uncertain implementation details are omitted.

Main entry points: [`chats`](https://github.com/deep-diver/llm-as-chatbot/tree/main/chats) (Model chat adapters), [`discord_app.py`](https://github.com/deep-diver/llm-as-chatbot/blob/main/discord_app.py) (Discord bot), [`global_vars.py`](https://github.com/deep-diver/llm-as-chatbot/blob/main/global_vars.py) (Model runtime), [`discordbot/req.py`](https://github.com/deep-diver/llm-as-chatbot/blob/main/discordbot/req.py) (Discord requests)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontends["Chat front ends"]
  node_gradio["Gradio UI<br/>[app.py]"]
  node_discord["Discord bot<br/>[discord_app.py]"]
  node_discord_requests["Discord requests<br/>[req.py]"]
  node_discord_flags["Request flags<br/>[flags.py]"]
end

subgraph group_conversation["Conversation handling"]
  node_chat_router["Model chat router<br/>[central.py]"]
  node_chat_adapters["Model chat adapters"]
  node_context["Prompt and context<br/>[utils.py]"]
end

subgraph group_inference["Models and inference"]
  node_prompt_inputs["Generation inputs<br/>[pre.py]"]
  node_stream_output["Stream formatting<br/>[post.py]"]
  node_batch_generation["Batch generation<br/>[batch_gen.py]"]
  node_model_runtime["Model runtime<br/>[global_vars.py]"]
  node_model_zoo["Model definitions"]
end

subgraph group_integrations["External services"]
  node_serper["Serper search"]
  node_discord_service["Discord service"]
  node_huggingface["Hugging Face Hub"]
end

node_user(("Chat user"))

node_user -->|"chats through"| node_gradio
node_user -->|"mentions bot"| node_discord
node_gradio -->|"routes chat"| node_chat_router
node_discord -->|"handles requests"| node_discord_requests
node_discord -->|"initializes model"| node_model_runtime
node_discord_requests -->|"parses options"| node_discord_flags
node_discord_requests -.->|"builds prompt"| node_context
node_discord_requests -->|"generates response"| node_model_runtime
node_chat_router -->|"dispatches model"| node_chat_adapters
node_chat_adapters -->|"builds prompts"| node_context
node_chat_adapters -->|"prepares generation"| node_prompt_inputs
node_chat_adapters -->|"strips response"| node_stream_output
node_chat_adapters -->|"calls batch generation"| node_batch_generation
node_prompt_inputs -->|"invokes generation"| node_model_runtime
node_model_runtime -->|"loads model implementation"| node_model_zoo
node_model_runtime -.->|"loads model weights"| node_huggingface
node_chat_adapters -->|"searches when enabled"| node_serper
node_discord -.->|"searches when enabled"| node_serper
node_discord -->|"connects bot"| node_discord_service

click node_gradio "https://github.com/deep-diver/llm-as-chatbot/blob/main/app.py"
click node_discord "https://github.com/deep-diver/llm-as-chatbot/blob/main/discord_app.py"
click node_discord_requests "https://github.com/deep-diver/llm-as-chatbot/blob/main/discordbot/req.py"
click node_discord_flags "https://github.com/deep-diver/llm-as-chatbot/blob/main/discordbot/flags.py"
click node_chat_router "https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/central.py"
click node_chat_adapters "https://github.com/deep-diver/llm-as-chatbot/tree/main/chats"
click node_context "https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/utils.py"
click node_prompt_inputs "https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/pre.py"
click node_stream_output "https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/post.py"
click node_batch_generation "https://github.com/deep-diver/llm-as-chatbot/blob/main/gens/batch_gen.py"
click node_model_runtime "https://github.com/deep-diver/llm-as-chatbot/blob/main/global_vars.py"
click node_model_zoo "https://github.com/deep-diver/llm-as-chatbot/tree/main/models"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gradio,node_discord,node_discord_requests,node_discord_flags,node_user toneBlue
class node_chat_router,node_chat_adapters,node_context toneAmber
class node_prompt_inputs,node_stream_output,node_batch_generation,node_model_runtime,node_model_zoo toneMint
class node_serper,node_discord_service,node_huggingface toneRose
```

## Components

### Chat front ends

- [Gradio UI](https://github.com/deep-diver/llm-as-chatbot/blob/main/app.py): `app.py`
- [Discord bot](https://github.com/deep-diver/llm-as-chatbot/blob/main/discord_app.py): `discord_app.py`
- [Discord requests](https://github.com/deep-diver/llm-as-chatbot/blob/main/discordbot/req.py): `discordbot/req.py`
- [Request flags](https://github.com/deep-diver/llm-as-chatbot/blob/main/discordbot/flags.py): `discordbot/flags.py`

### Conversation handling

- [Model chat router](https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/central.py): `chats/central.py`
- [Model chat adapters](https://github.com/deep-diver/llm-as-chatbot/tree/main/chats): `chats`
- [Prompt and context](https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/utils.py): `chats/utils.py`

### Models and inference

- [Generation inputs](https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/pre.py): `chats/pre.py`
- [Stream formatting](https://github.com/deep-diver/llm-as-chatbot/blob/main/chats/post.py): `chats/post.py`
- [Batch generation](https://github.com/deep-diver/llm-as-chatbot/blob/main/gens/batch_gen.py): `gens/batch_gen.py`
- [Model runtime](https://github.com/deep-diver/llm-as-chatbot/blob/main/global_vars.py): `global_vars.py`
- [Model definitions](https://github.com/deep-diver/llm-as-chatbot/tree/main/models): `models`

### External services

- **Serper search**
- **Discord service**
- **Hugging Face Hub**

### Other components

- **Chat user**

## Connections

- Chat user → Gradio UI: chats through
- Chat user → Discord bot: mentions bot
- Gradio UI → Model chat router: routes chat (evidence: `app.py`)
- Discord bot → Discord requests: handles requests (evidence: `discord_app.py`)
- Discord bot → Model runtime: initializes model (evidence: `discord_app.py`)
- Discord requests → Request flags: parses options (evidence: `discordbot/req.py`)
- Discord requests → Prompt and context: builds prompt
- Discord requests → Model runtime: generates response (evidence: `discordbot/req.py`)
- Model chat router → Model chat adapters: dispatches model (evidence: `chats/central.py`)
- Model chat adapters → Prompt and context: builds prompts (evidence: `chats/xgen.py`)
- Model chat adapters → Generation inputs: prepares generation (evidence: `chats/xgen.py`)
- Model chat adapters → Stream formatting: strips response (evidence: `chats/xgen.py`)
- Model chat adapters → Batch generation: calls batch generation (evidence: `chats/alpaca.py`)
- Generation inputs → Model runtime: invokes generation (evidence: `chats/pre.py`)
- Model runtime → Model definitions: loads model implementation (evidence: `global_vars.py`)
- Model runtime → Hugging Face Hub: loads model weights
- Model chat adapters → Serper search: searches when enabled (evidence: `chats/utils.py`)
- Discord bot → Serper search: searches when enabled
- Discord bot → Discord service: connects bot