# mysterionrise/agentbricks-experiments architecture

> Architecture diagram of the GitHub repository mysterionrise/agentbricks-experiments, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mysterionrise/agentbricks-experiments
- Repository: https://github.com/mysterionrise/agentbricks-experiments

## Overview

##### Project type & purpose (what you’re diagramming)
This repo is not a “traditional app” with services and source packages; it’s a **minimal Databricks workspace demo** to stand up **two Agent Bricks workflows**:

1. **Knowledge Assistant (KA)** backed by **Unity Catalog (UC) Volume files** generated by a notebook.
2. **Custom LLM** backed by a **toy labeled UC table** (`request`, `response`) generated by a notebook.

##### File structure → architectural components
From the tree:

- `notebooks/00_check_prereqs.py`
- Validates workspace/runtime prerequisites (notably embedding endpoint availability: `databricks-gte-large-en`).
- `notebooks/01_setup_catalog_schema_volume.py`
- Creates/ensures **UC catalog + schema + volume** (default `main.ab_demo.ka_data`).
- `notebooks/02_load_sample_docs_to_volume.py`
- Writes **markdown docs** into the UC Volume path:
`/Volumes/<catalog>/<schema>/<volume>/knowledge`
- `notebooks/03_build_custom_llm_dataset.py`
- Creates UC table `main.ab_demo.custom_llm_tiny` with `(request,response)` examples.
- `notebooks/04_query_knowledge_assistant_example.py`
- Shows how to **query the deployed agent endpoint** via **MLflow Deployments**.

Other files:

Architecturally, the notebooks are “infrastructure + data bootstrapping scripts,” not a runtime service.

##### README → key platform interactions you must capture
Important constraints and flows called out in the README:

- Requires **Unity Catalog**, **Serverless compute**, and access to foundation models via `system.ai`.
- Knowledge Assistant knowledge source must be:
- **UC Volume files** OR **Vector Search index** (not used in this demo, but mention as an optional branch).
- Knowledge Assistant depends on an **embedding model endpoint**: `databricks-gte-large-en`.
- Creation/deployment is done through **Databricks UI** (Agents → Knowledge Assistant / Custom LLM), after data is prepared by notebooks.
- Querying happens via **AI Playground** or **MLflow Deployments** (programmatic client).

##### User / Developer
- Runs notebooks in Databricks workspace
- Creates agents in Databricks UI
- Queries endpoints from Playground or code

##### Databricks Workspace
- **Notebooks** (the repo content executed on compute)
- **Serverless Compute / Cluster** (execution environment for notebooks)

##### Unity Catalog (Governance layer)
- **Catalog + Schema**
- **UC Volume**: `main.ab_demo.ka_data`
- Folder: `/knowledge` containing markdown files
- **UC Table**: `main.ab_demo.custom_llm_tiny` (custom LLM dataset)

##### Agent Bricks (Agents service)
- **Knowledge Assistant Agent**
- Knowledge source: UC Files (volume path ending in `/knowledge`)
- **Custom LLM Agent**
- Dataset: UC table with request/response columns
- **Deployed Agent Endpoints** (the runtime endpoints produced after build/sync/optimization)

##### Model/Embedding Services
- **Embedding endpoint**: `databricks-gte-large-en` (required for KA ingestion/indexing)
- **Foundation model access via `system.ai`** (used by the agent runtime)

##### Client Query Layer
- **AI Playground** (interactive testing)
- **MLflow Deployments client** (used in notebook `04_...` to call the endpoint)

Optional branch (call out as “not used in this repo but supported”):
- **Vector Search Index** as an alternative knowledge source to UC Files.

##### Flow 0: Prerequisite check
- `00_check_prereqs.py` → checks workspace context + availability of `databricks-gte-large-en`

##### Flow 1: Knowledge Assistant data preparation
- `01_setup_catalog_schema_volume.py`:
- `02_load_sample_docs_to_volume.py`:

##### Flow 2: KA agent build/sync + deployment (control plane action)
- User in **Agents → Knowledge Assistant**:

##### Flow 3: Custom LLM dataset + agent deployment
- `03_build_custom_llm_dataset.py`:
- User in **Agents → Custom LLM**:

##### Flow 4: Querying
- From Playground:
- From code:
- `04_query_knowledge_assistant_example.py`:

##### Architectural patterns to highlight (explicitly label these)
- **Notebook-driven provisioning**: notebooks act as “IaC-lite” for UC objects and sample data.
- **Governed data plane**: all knowledge sources live in **Unity Catalog** (Volumes/Tables).
- **Managed agent lifecycle**: build/sync/optimization/deploy handled by **Databricks Agent Bricks** (control plane).
- **RAG-style retrieval** for Knowledge Assistant:
- UC Files → embeddings (`databricks-gte-large-en`) → managed retrieval → LLM response.
- **Endpoint-based serving**:
- Both agents ultimately expose a **deployed endpoint** consumed by Playground or MLflow client.

- Databricks **Unity Catalog** (catalog/schema/volume/table)
- **UC Volumes** path convention `/Volumes/<catalog>/<schema>/<volume>/...`
- Agent Bricks **Knowledge Assistant** + **Custom LLM**
- Embeddings: `databricks-gte-large-en`
- LLM access: `system.ai`
- Query client: **MLflow Deployments**
- Execution: **Databricks Serverless compute** (or the compute mode you run notebooks on)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% =========================================================
    %% External Actor
    %% =========================================================
    U["User / Developer"]:::actor

    %% =========================================================
    %% Client / Local Repo Artifacts
    %% =========================================================
    subgraph RA["Repo Artifacts (This Demo)"]
        direction TB
        README["README (instructions & platform interactions)"]:::doc
        REQ["requirements.txt (client/job deps)"]:::doc
    end

    %% =========================================================
    %% Databricks Workspace (Authoring/Setup)
    %% =========================================================
    subgraph WS["Databricks Workspace (Authoring / Setup)"]
        direction TB

        subgraph COMP["Compute"]
            direction TB
            C["Serverless Compute / Cluster (Notebook execution)"]:::compute
        end

        subgraph NBS["Notebooks (IaC-lite + Data Bootstrapping)"]
            direction TB
            NB00["00_check_prereqs.py"]:::notebook
            NB01["01_setup_catalog_schema_volume.py"]:::notebook
            NB02["02_load_sample_docs_to_volume.py"]:::notebook
            NB03["03_build_custom_llm_dataset.py"]:::notebook
            NB04["04_query_knowledge_assistant_example.py"]:::notebook
        end

        subgraph AUI_SG["Agents UI (Control Plane Actions)"]
            direction TB
            AUI["Databricks UI: Agents (create/build/sync/optimize/deploy)"]:::control
        end
    end

    %% =========================================================
    %% Governance / Data Plane (Unity Catalog)
    %% =========================================================
    subgraph UC_SG["Unity Catalog (Governed Data Plane)"]
        direction TB

        UC["Unity Catalog (Catalog + Schema)"]:::governance

        subgraph KA_SRC["Knowledge Assistant Knowledge Source"]
            direction TB
            VOL[("UC Volume: main.ab_demo.ka_data")]:::storage
            KPATH["Folder: /Volumes/<catalog>/<schema>/<volume>/knowledge"]:::note
            DOCS["Markdown docs (sample knowledge)"]:::data
        end

        subgraph CLLM_SRC["Custom LLM Dataset"]
            direction TB
            TBL[("UC Table: main.ab_demo.custom_llm_tiny")]:::storage
            ROWS["Rows: (request, response) examples"]:::data
        end
    end

    %% =========================================================
    %% Platform AI Services
    %% =========================================================
    subgraph AI_SVC["Model / Embedding Services"]
        direction TB
        EMB["Embedding endpoint: databricks-gte-large-en"]:::service
        SYSAI["Foundation model access: system.ai"]:::service
    end

    %% =========================================================
    %% Agent Bricks (Build/Deploy) + Serving (Runtime)
    %% =========================================================
    subgraph AB["Agent Bricks (Managed Agent Lifecycle + Serving)"]
        direction TB

        subgraph AGENTS["Agent Definitions"]
            direction TB
            KA["Knowledge Assistant Agent (source: UC Volume files)"]:::agent
            CLLM["Custom LLM Agent (source: UC table dataset)"]:::agent
        end

        IDX[("Managed retrieval index / store (internal)")]:::storage

        subgraph EPS["Deployed Agent Endpoints (Runtime)"]
            direction TB
            KAEP["Deployed KA Endpoint"]:::endpoint
            CLLMEP["Deployed Custom LLM Endpoint"]:::endpoint
        end
    end

    %% =========================================================
    %% Query / Client Layer
    %% =========================================================
    subgraph CLIENTS["Query Clients"]
        direction TB
        PLAY["AI Playground"]:::client
        MLF["MLflow Deployments client"]:::client
    end

    %% =========================================================
    %% Control Flow (dashed)
    %% =========================================================
    U -.->|"reads_instructions"| README
    U -.->|"runs_notebooks"| C
    C -.->|"executes"| NB00
    C -.->|"executes"| NB01
    C -.->|"executes"| NB02
    C -.->|"executes"| NB03
    C -.->|"executes"| NB04

    U -.->|"creates_agents"| AUI
    AUI -.->|"create_KA"| KA
    AUI -.->|"create_Custom_LLM"| CLLM
    AUI -.->|"deploy"| KAEP
    AUI -.->|"deploy"| CLLMEP

    %% =========================================================
    %% Data Prep Flows (solid)
    %% =========================================================
    NB00 -->|"checks_availability"| EMB
    NB01 -->|"create/ensure"| UC
    NB01 -->|"provisions"| VOL
    NB02 -->|"writes_docs"| VOL
    VOL -->|"contains"| KPATH
    VOL -->|"stores"| DOCS

    NB03 -->|"writes_examples"| TBL
    TBL -->|"contains"| ROWS

    %% =========================================================
    %% KA Build/Sync + RAG retrieval
    %% =========================================================
    KA -->|"reads_uc_files"| VOL
    KA -->|"embeds_docs"| EMB
    KA -->|"builds_retrieval_index"| IDX
    KAEP -->|"retrieves_context"| IDX
    KAEP -->|"generates_response"| SYSAI

    %% =========================================================
    %% Custom LLM build/optimize
    %% =========================================================
    CLLM -->|"reads_dataset"| TBL
    CLLM -->|"optimize/evaluate_with"| SYSAI
    CLLMEP -->|"generates_response"| SYSAI

    %% =========================================================
    %% Querying paths
    %% =========================================================
    U -->|"interactive_test"| PLAY
    PLAY -->|"query"| KAEP
    PLAY -->|"query"| CLLMEP

    NB04 -->|"uses"| MLF
    MLF -->|"query"| KAEP

    %% =========================================================
    %% Optional / Not used branch
    %% =========================================================
    VS["Vector Search Index (optional KA source; not used here)"]:::optional
    KA -.->|"alt_source"| VS

    %% =========================================================
    %% Notes / Constraints
    %% =========================================================
    PR["Prereqs: Unity Catalog enabled • Serverless compute • system.ai access • embedding endpoint available"]:::note
    README -.->|"highlights_prereqs"| PR
    NB00 -.->|"validates"| PR
    KA -.->|"constraint:UC_Volume_files_only"| KPATH

    %% =========================================================
    %% Click Events (component_mapping)
    %% =========================================================
    click NB00 "https://github.com/mysterionrise/agentbricks-experiments/blob/main/notebooks/00_check_prereqs.py"
    click NB01 "https://github.com/mysterionrise/agentbricks-experiments/blob/main/notebooks/01_setup_catalog_schema_volume.py"
    click NB02 "https://github.com/mysterionrise/agentbricks-experiments/blob/main/notebooks/02_load_sample_docs_to_volume.py"
    click NB03 "https://github.com/mysterionrise/agentbricks-experiments/blob/main/notebooks/03_build_custom_llm_dataset.py"
    click NB04 "https://github.com/mysterionrise/agentbricks-experiments/blob/main/notebooks/04_query_knowledge_assistant_example.py"
    click REQ "https://github.com/mysterionrise/agentbricks-experiments/blob/main/requirements.txt"
    click README "https://github.com/mysterionrise/agentbricks-experiments/blob/main/README.md"

    %% =========================================================
    %% Styles
    %% =========================================================
    classDef actor fill:#1f2937,stroke:#0b1220,color:#ffffff,stroke-width:1px
    classDef doc fill:#fff7ed,stroke:#9a3412,color:#7c2d12,stroke-width:1px
    classDef compute fill:#ecfeff,stroke:#0891b2,color:#0e7490,stroke-width:1px
    classDef notebook fill:#eef2ff,stroke:#4338ca,color:#312e81,stroke-width:1px
    classDef governance fill:#f0fdf4,stroke:#16a34a,color:#14532d,stroke-width:1px
    classDef storage fill:#dcfce7,stroke:#15803d,color:#14532d,stroke-width:1px
    classDef data fill:#f0fdf4,stroke:#22c55e,color:#14532d,stroke-width:1px
    classDef service fill:#ede9fe,stroke:#6d28d9,color:#4c1d95,stroke-width:1px
    classDef control fill:#fff1f2,stroke:#be123c,color:#881337,stroke-width:1px
    classDef agent fill:#fef3c7,stroke:#d97706,color:#78350f,stroke-width:1px
    classDef endpoint fill:#fde68a,stroke:#b45309,color:#78350f,stroke-width:1px
    classDef client fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e,stroke-width:1px
    classDef note fill:#f8fafc,stroke:#64748b,color:#334155,stroke-width:1px,stroke-dasharray: 4 3
    classDef optional fill:#f1f5f9,stroke:#94a3b8,color:#475569,stroke-width:1px,stroke-dasharray: 6 4
```
