# kimhungbui/khoj architecture

> Architecture diagram of the GitHub repository kimhungbui/khoj, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kimhungbui/khoj
- Repository: https://github.com/kimhungbui/khoj

## Overview

##### Main Components
- Client Interfaces:
- Multiple “interfaces” directory under src/interface that contains clients for Android, desktop, Emacs, Obsidian, and web.
- Each of these clients has its own tech stack (for example, Android uses Gradle/Java/Kotlin while the web interface uses Next.js with TypeScript/React).

- Backend Server & API Layer:
- The core resides in src/khoj, which follows a modular monolith pattern. It includes typical server files (manage.py, settings.py, asgi.py) and domain directories like “app”, “database”, “routers”, “processor”, “utils”, etc.
- The “routers” directory hints at REST-like API endpoints for chat, agents, subscriptions, authentication, and more.

- Processors & Content Handlers:
- The processor directory contains submodules to convert various document formats (docx, pdf, markdown, plaintext, etc.) into a standard “entry” format.
- There are dedicated modules for online and offline conversation processing (including integration with providers such as OpenAI, Anthropic, Google, etc.).

- Database:
- The database directory (with migrations, models, management commands) represents the data layer where user data, conversation logs, configurations, and more are persisted.

- Telemetry & Analytics:
- A separate component exists in src/telemetry to capture usage metrics or diagnostics, likely playing a role in monitoring or analytics.

- External Integrations:
- Additional integrations include email interfaces (located within src/khoj/interface/email), search filters, and tool integrations (e.g., online search, code execution, voice chat).
- The project also has CI/CD pipelines and Docker-related files (Dockerfile, docker-compose.yml) to facilitate deployment and containerization.

##### Relationships & Data Flows
- Client Interfaces ↔ API Gateway:

- Backend Service Components:
- Within the backend, illustrate a modular division:
- “Routers” for exposing APIs to client interfaces.
- “Database” for data persistence.
- “Utilities/Helpers” supporting common application logic.

- Telemetry:

- External Services:

##### Architectural Patterns & Key Design Principles
- Modular Monolith:
- The backend is organized into modules (routers, processors, database, utils) that communicate internally even though they operate as one unified system.

- Multi-Interface Client Architecture:
- Separation of multiple client apps, each built with its own technology stack, yet unified through common API endpoints.

- Containerization & Deployment:

- CI/CD Pipeline & Automated Testing:
- While not part of runtime architecture, the existence of extensive GitHub workflows (CI, Docker builds, tests, etc.) can be noted for how the project ensures continuous integration and reliability.

##### Technologies, Frameworks & Libraries
- Backend: Python (likely Django given the presence of manage.py, settings.py, migrations), using ASGI for scaling.
- Frontend:
- Android interfaces using Gradle & Java/Kotlin,
- Desktop interfaces likely built with Electron-like stacks,
- Web interfaces using Next.js with TypeScript/React.
- Deployment and Containerization: Docker, docker-compose, and GitHub Actions for CI/CD.
- Content Processing: Specialized Python scripts/modules for processing docx, PDF, markdown, etc.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Interfaces Cluster
    subgraph "Client Interfaces"
        A["Android App"]
        B["Desktop App"]
        C["Emacs Client"]
        D["Obsidian Plugin"]
        E["Web App"]
    end

    %% Backend Cluster
    subgraph "Backend Cluster"
        F["Backend Server"]
        G["API Routers"]
        H["Content Processors"]
        I[(Database)]
        J["Email Integration"]
    end

    %% Telemetry & Deployment Cluster
    subgraph "Telemetry & Deployment"
        K["Telemetry Service"]
        L["Docker & CI/CD"]
    end

    %% External Services Cluster
    subgraph "External Services"
        M["External LLMs & Search"]
    end

    %% Client Interfaces to API Routers
    A -->|"API"| G
    B -->|"API"| G
    C -->|"API"| G
    D -->|"API"| G
    E -->|"API"| G

    %% Backend internal communication
    G -->|"Request"| F
    G -->|"Process"| H
    F -->|"DataAccess"| I
    H -->|"Store"| I
    H -->|"Integrate"| M
    F -->|"Email"| J

    %% Telemetry integration
    G -->|"Metrics"| K
    F -->|"Metrics"| K

    %% Deployment links
    F -->|"Deploy"| L
    K -->|"Deploy"| L

    %% Styling Classes
    class A,B,C,D,E client
    class F,G,H,I,J backend
    class K telemetry
    class L deploy
    class M external

    %% Class Definitions with Colors
    classDef client fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef backend fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef telemetry fill:#FFD700,stroke:#000,stroke-width:1px;
    classDef deploy fill:#D3D3D3,stroke:#000,stroke-width:1px;
    classDef external fill:#FFB6C1,stroke:#000,stroke-width:1px;

    %% Click Events for Component Mappings
    click A "https://github.com/kimhungbui/khoj/tree/master/src/interface/android"
    click B "https://github.com/kimhungbui/khoj/tree/master/src/interface/desktop"
    click C "https://github.com/kimhungbui/khoj/tree/master/src/interface/emacs"
    click D "https://github.com/kimhungbui/khoj/tree/master/src/interface/obsidian"
    click E "https://github.com/kimhungbui/khoj/tree/master/src/interface/web"
    click F "https://github.com/kimhungbui/khoj/tree/master/src/khoj"
    click G "https://github.com/kimhungbui/khoj/tree/master/src/khoj/routers"
    click H "https://github.com/kimhungbui/khoj/tree/master/src/khoj/processor"
    click I "https://github.com/kimhungbui/khoj/tree/master/src/khoj/database"
    click J "https://github.com/kimhungbui/khoj/tree/master/src/khoj/interface/email"
    click K "https://github.com/kimhungbui/khoj/tree/master/src/telemetry"
```
