# jupyterlab/jupyter-ai architecture

> Architecture diagram of the GitHub repository jupyterlab/jupyter-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jupyterlab/jupyter-ai
- Repository: https://github.com/jupyterlab/jupyter-ai

## Overview

##### Project Type and Purpose
- Jupyter AI is a polyrepo/monorepo containing multiple packages.
- It delivers:
- A Python library (`jupyter-ai`) that works as a Jupyter server extension and provides AI “chat” and “completion” endpoints.
- A Python package (`jupyter-ai-magics`) that registers the `%%ai` IPython magic.
- A TypeScript/React JupyterLab extension (inside `packages/jupyter-ai/src`) providing a native chat UI and settings widgets.
- Auxiliary packages for cookiecutter templates, tests, docs, examples, and CI pipelines.
- Purpose: let notebook users run generative AI prompts via `%load_ext jupyter_ai` or via a chat sidebar in JupyterLab, abstracting over many model providers.

##### File Structure
- Top-level orchestrator: `lerna.json`, `nx.json` drive builds and releases across packages.
- `packages/jupyter-ai-magics`: Python code for magics, provider plugins, tests.
- `packages/jupyter-ai`:
- Python server extension (`jupyter_ai/…`)
- TypeScript/React frontend (`src/…`)
- Jupyter configurators (`jupyter-config`)
- UI tests (`ui-tests/…`)
- CI workflows under `.github/workflows` automate lint, tests, releases.
- Docs and examples live in `docs/` and `examples/` for user/developer reference.

##### Extract Architectural Insights from README
- Two main user interfaces: `%%ai` magic (Python kernel side) and JupyterLab chat UI (JS extension).
- Providers are pluggable via Python “partner_providers” modules and selected at runtime.
- Supports streaming responses and context providers (e.g., loading local documents).

##### Main Components and Their Interactions
a. User Interfaces
- Notebook cell running `%%ai` → IPython kernel → `jupyter-ai-magics` → Python provider → stream result → notebook output.
- JupyterLab sidecar chat UI → TS extension → Jupyter server extension RPC endpoint → backend logic → provider → stream back → UI renders.
b. Backend Logic (Python)
- Chat handlers (`chat_handlers/…`) & completion handlers (`completions/…`)
- Context providers & document loaders (to inject additional context)
- Provider abstractions & partner implementations (OpenAI, Anthropic, local models, etc.)
c. Frontend (TypeScript/React)
- React components (chat window, settings widget)
- Telemetry & configuration contexts
- API client calling Jupyter server REST/WebSocket endpoints
d. External Services
- Cloud model APIs (OpenAI, AWS, Cohere, etc.)
- Local model runtimes (GPT4All, Ollama)

##### Architectural Patterns and Design Principles
- Plugin architecture: providers loaded dynamically via Python entry points.
- Separation of concerns: clear split between UI (TS) and business logic (Python).
- Streaming and event-driven flows (WebSocket or HTTP streaming).
- Monorepo orchestrator (Lerna/Nx) for unified versioning and cross-package testing.

- Component labels (UI, magics, server extension, handlers, providers)
- Optional: annotate streaming vs. request/response semantics

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Monorepo Tooling
    subgraph "Monorepo Orchestrator" 
        MonorepoOrch["Lerna/Nx Orchestrator"]:::infra
    end
    click MonorepoOrch "https://github.com/jupyterlab/jupyter-ai/blob/main/lerna.json"

    %% CI, Docs, Examples
    subgraph "Infra & Resources"
        CI["CI Workflows"]:::infra
        Docs["Documentation Site"]:::infra
        Examples["Example Notebooks"]:::infra
    end
    click CI "https://github.com/jupyterlab/jupyter-ai/tree/main/.github/workflows/"
    click Docs "https://github.com/jupyterlab/jupyter-ai/tree/main/docs/"
    click Examples "https://github.com/jupyterlab/jupyter-ai/tree/main/examples/"

    %% User Interfaces
    subgraph "User Interfaces" 
        NotebookUI(["Notebook Cell with %%ai Magic"]):::frontend
        LabUI(["JupyterLab Chat Sidebar"]):::frontend
    end

    %% Python Backend
    subgraph "Python Kernel & Server Extension"
        Magics["jupyter-ai-magics (Python Kernel Magic)"]:::backend
        MagicsPP["Partner Providers"]:::backend
        Utils["Completion Utils & Models"]:::backend
        ServerExt["Jupyter Server Extension"]:::backend
        ChatHandlers["Chat Handlers"]:::backend
        CompletionHandlers["Completion Handlers"]:::backend
        ContextProviders["Context Providers"]:::backend
        DocLoaders["Document Loaders"]:::backend
        ConfigMgr["Config Manager & Schema"]:::backend
    end
    click Magics "https://github.com/jupyterlab/jupyter-ai/blob/main/packages/jupyter-ai-magics/jupyter_ai_magics/magics.py"
    click MagicsPP "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai-magics/jupyter_ai_magics/partner_providers/"
    click Utils "https://github.com/jupyterlab/jupyter-ai/blob/main/packages/jupyter-ai-magics/jupyter_ai_magics/completion_utils.py"
    click ServerExt "https://github.com/jupyterlab/jupyter-ai/blob/main/packages/jupyter-ai/jupyter_ai/extension.py"
    click ChatHandlers "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/jupyter_ai/chat_handlers/"
    click CompletionHandlers "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/jupyter_ai/completions/handlers/"
    click ContextProviders "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/jupyter_ai/context_providers/"
    click DocLoaders "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/jupyter_ai/document_loaders/"
    click ConfigMgr "https://github.com/jupyterlab/jupyter-ai/blob/main/packages/jupyter-ai/jupyter_ai/config_manager.py"

    %% TypeScript Frontend
    subgraph "TypeScript/React Frontend"
        FEroot["Frontend Extension Root"]:::frontend
        Components["React Components (Chat UI & Settings)"]:::frontend
        Contexts["Telemetry & Config Contexts"]:::frontend
        FEhandler["API Handler & Plugin Reg."]:::frontend
        UItests["UI End-to-End Tests"]:::infra
    end
    click FEroot "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/src/"
    click Components "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/src/components/"
    click Contexts "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/src/contexts/"
    click FEhandler "https://github.com/jupyterlab/jupyter-ai/blob/main/packages/jupyter-ai/src/handler.ts"
    click UItests "https://github.com/jupyterlab/jupyter-ai/tree/main/packages/jupyter-ai/ui-tests/"

    %% External Services
    subgraph "External Model Providers"
        ProviderAbstr["Provider Abstraction"]:::backend
        OpenAI["OpenAI / AWS / Cohere / Local Models"]:::external
    end

    %% Flows: Magic Path
    NotebookUI -->|"%%ai →"| Magics
    Magics -->|"invoke"| Utils
    Utils -->|"calls"| MagicsPP
    MagicsPP -->|"stream chunks ←"| Magics
    Magics -->|"response ←"| NotebookUI

    %% Flows: Chat UI Path
    LabUI -->|"UI interaction"| FEhandler
    FEhandler -->|"REST/WebSocket →"| ServerExt
    ServerExt -->|"route →"| ChatHandlers
    ServerExt -->|"route →"| CompletionHandlers
    ChatHandlers -->|"use"| ContextProviders
    ChatHandlers -->|"use"| DocLoaders
    ChatHandlers -->|"delegate →"| ProviderAbstr
    CompletionHandlers -->|"delegate →"| ProviderAbstr
    ProviderAbstr -->|"API calls →"| OpenAI
    OpenAI -->|"stream ←"| ProviderAbstr
    ProviderAbstr -->|"stream ←"| ChatHandlers
    ChatHandlers -->|"response ←"| FEhandler
    FEhandler -->|"render ←"| LabUI

    %% Interconnections
    ConfigMgr -.->|provides config| ServerExt

    %% Styles
    classDef frontend fill:#E8F0FE,stroke:#1A73E8;
    classDef backend fill:#F1F3F4,stroke:#5F6368;
    classDef external fill:#FFF4E5,stroke:#F29900,stroke-dasharray: 5 5;
    classDef infra fill:#E6F4EA,stroke:#34A853;
```
