# chadmarkey/offline-notebooklm architecture

> Architecture diagram of the GitHub repository chadmarkey/offline-notebooklm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chadmarkey/offline-notebooklm
- Repository: https://github.com/chadmarkey/offline-notebooklm

## Overview

1. **What this repo is architecturally**
   - This is an offline, single-laptop clinical RAG stack built around a hard boundary: anything that might contain PHI must pass through a local de-identification gate before it reaches the notebook/index.
   - The repo is not just an app; it is a composed system of host tools, Docker services, a native ingest pipeline, and a few patches/customizations to keep every model local.

2. **The trust boundary and data ingress**
   - The main boundary is the `raw_pdfs/` folder. Files dropped there are treated as potentially sensitive and are the only intended ingress path for chart-derived material.
   - The UI’s direct “Add Source” path is intentionally a separate, unsafe boundary: it is only for sources you can guarantee are PHI-free.
   - After processing, originals are moved to `raw_pdfs/_processed/`, while the de-identified output goes to `clean_sources/`.

3. **Native preprocessing / de-identification layer**
   - `src/ingest.py` is the core ingest pipeline: it parses PDFs, runs de-identification, and prepares clean text for downstream indexing.
   - `src/phi_rules.py` contains rule-based corrections around the model output, especially edge cases like ages over 89 and address subunits.
   - The README indicates this layer uses Docling for PDF-to-markdown conversion plus OpenMed for PII stripping and NER tagging, all running natively in the Python environment, not in Docker.

4. **Clinical NLP gate and regression check**
   - `src/deid_redteam.py` is a safety gate for the ingest pipeline. It runs a fixed adversarial PHI test suite and must pass fully before the de-id layer is trusted.
   - Architecturally, this is a preflight control: if it fails, the system should not be used for ingestion.
   - The OpenMed models enrich the document, too, by attaching clinical entity tags into frontmatter before indexing.

5. **File watcher as the handoff mechanism**
   - `src/watch.py` is the operational bridge between the filesystem and the notebook system.
   - Its job is to notice new PDFs in `raw_pdfs/`, invoke the ingest/de-id pipeline, and then push cleaned content into the notebook backend through the API.
   - This watcher is what turns the repo from a batch tool into a continuously usable local workspace.

6. **Notebook / RAG application layer**
   - The main user-facing workspace is Open Notebook, run in Docker and exposed through the UI and API described in the README.
   - This layer owns notebooks, sources, notes, chats, retrieval, session context, and the citation-backed answer experience.
   - It is also where source selection, notebook separation, search, and transformation workflows live.

7. **Local model services and their roles**
   - LLM inference is split from the rest of the stack and runs natively in LM Studio on the host for Metal acceleration.
   - Embeddings and reranking run through Infinity in Docker; the custom `infinity-arm64/` build exists to support the required Apple-silicon runtime and model endpoints.
   - TTS/STT run in another Docker service (`Speaches`) so audio features stay local as well.
   - The architecture intentionally keeps model endpoints OpenAI-compatible so Open Notebook can talk to them through standard HTTP APIs.

8. **Storage and persistence boundaries**
   - SurrealDB is the persistent data store for notebook content and metadata.
   - Docker volumes and working directories are split by purpose: `surreal_data` for database state, `notebook_data` for notebook app state, and `infinity_cache` for model cache.
   - The sensitive content path is therefore: local parse/de-id → cleaned text → indexed/persisted in the notebook/search backend.

9. **Transformations as reusable content-processing jobs**
   - The `transformations/` folder defines reusable AI workflows that operate on sources, such as flashcards, board-style Q&A, clinical pearls, and patient-facing explanations.
   - These are not separate apps; they are content-generation templates plugged into the notebook layer.
   - `transformations/seed_transformations.py` bootstraps these presets into the system.

10. **Patches and repo-specific customization**
   - `patches/notebook.py` indicates the upstream Open Notebook code is patched locally, likely to adapt behavior for the offline/local-only build.
   - This is a key architectural clue: the repo is not a pure wrapper; it modifies the upstream application to fit the offline workflow and provider wiring.

11. **Docker composition and service topology**
   - `docker-compose.yml` is the orchestration layer for all containerized services: Open Notebook, SurrealDB, Infinity, and Speaches.
   - The network model is local-only service-to-service communication by container name, with the host LLM reached via `host.docker.internal`.
   - The only notable non-container runtime is LM Studio on the host; everything else is designed to stay on the machine and communicate over localhost or Docker’s internal network.

12. **Operational safety and offline enforcement**
   - The repo’s security model depends on process- and network-level controls outside the app itself, especially LuLu firewall rules and the “offline only” operating mode.
   - The README explicitly treats firewall blocking as the final step, meaning the system’s confidentiality posture is an operational property, not just a code property.
   - For an architecture diagram, this is important: there is a functional stack, but also an external enforcement boundary around egress.

13. **How the end-to-end flow fits together**
   - Input: PDF lands in `raw_pdfs/`.
   - Preprocess: `watch.py` triggers `ingest.py`, which uses Docling + OpenMed + rules to de-identify and tag.
   - Publish: cleaned content is sent through the Open Notebook REST API into the notebook workspace.
   - Retrieval: user query hits Open Notebook, which retrieves from SurrealDB/vector search via Infinity embeddings and reranks with the local reranker.
   - Generation: LM Studio produces the final cited answer, while optional audio features go through Speaches.

14. **What to show in an architecture diagram**
   - Draw three distinct zones: host-native services, Dockerized backend services, and the protected ingestion boundary around `raw_pdfs/`.
   - Show the native Python ingest path as the only sanctioned bridge from sensitive documents into the notebook.
   - Show Open Notebook as the orchestration/UI layer, with separate arrows to embeddings/reranking, storage, TTS/STT, and the host LLM.
   - Include the safety controls as an outer boundary, not as part of the core data path.

Main entry points: [`patches/notebook.py`](https://github.com/chadmarkey/offline-notebooklm/blob/main/patches/notebook.py) (Open Notebook), [`docker-compose.yml`](https://github.com/chadmarkey/offline-notebooklm/blob/main/docker-compose.yml) (Compose stack), [`src/ingest.py`](https://github.com/chadmarkey/offline-notebooklm/blob/main/src/ingest.py) (Ingest pipeline), [`src/watch.py`](https://github.com/chadmarkey/offline-notebooklm/blob/main/src/watch.py) (Watcher)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_boundary["Protected ingress"]
  node_raw_pdfs[("Raw PDFs<br/>sensitive ingress folder")]
  node_processed_pdfs[("Processed PDFs<br/>staging folder")]
  node_clean_sources[("Clean sources<br/>de-id output folder")]
end

subgraph group_host["Host runtime"]
  node_watch["Watcher<br/>filesystem bridge<br/>[watch.py]"]
  node_ingest["Ingest pipeline<br/>native preprocessing<br/>[ingest.py]"]
  node_phi_rules["PHI rules<br/>rule corrections<br/>[phi_rules.py]"]
  node_deid_redteam{{"Red-team check<br/>safety gate<br/>[deid_redteam.py]"}}
  node_docling_openmed["Docling + OpenMed<br/>local NLP stack"]
  node_lmstudio["LM Studio<br/>host LLM"]
  node_offline_control{{"Offline controls<br/>operational boundary<br/>[README.md]"}}
end

subgraph group_docker["Docker services"]
  node_open_notebook["Open Notebook<br/>workspace app<br/>[notebook.py]"]
  node_surrealdb[("SurrealDB<br/>persistent store")]
  node_infinity["Infinity<br/>embeddings/rerank"]
  node_speaches["Speaches<br/>audio service"]
  node_compose["Compose stack<br/>orchestration<br/>[docker-compose.yml]"]
end

subgraph group_content["Content workflows"]
  node_transformations["Transformations<br/>content workflows"]
  node_seed_transformations["Seed presets<br/>bootstrap job"]
end

node_raw_pdfs -->|"new PDF"| node_watch
node_watch -->|"preflight"| node_deid_redteam
node_deid_redteam -->|"pass"| node_ingest
node_watch -->|"invoke"| node_ingest
node_ingest -->|"convert + tag"| node_docling_openmed
node_ingest -->|"normalize"| node_phi_rules
node_ingest -->|"emit clean text"| node_clean_sources
node_ingest -->|"archive original"| node_processed_pdfs
node_clean_sources -->|"publish"| node_open_notebook
node_open_notebook -->|"persist"| node_surrealdb
node_open_notebook -->|"embed/re-rank"| node_infinity
node_open_notebook -->|"generate"| node_lmstudio
node_open_notebook -->|"audio"| node_speaches
node_transformations -->|"bootstrap"| node_seed_transformations
node_seed_transformations -->|"install"| node_open_notebook
node_compose -->|"orchestrate"| node_open_notebook
node_compose -->|"orchestrate"| node_surrealdb
node_compose -->|"orchestrate"| node_infinity
node_compose -->|"orchestrate"| node_speaches
node_compose -->|"connect host"| node_lmstudio
node_offline_control -.->|"egress guard"| node_compose

click node_watch "https://github.com/chadmarkey/offline-notebooklm/blob/main/src/watch.py"
click node_ingest "https://github.com/chadmarkey/offline-notebooklm/blob/main/src/ingest.py"
click node_phi_rules "https://github.com/chadmarkey/offline-notebooklm/blob/main/src/phi_rules.py"
click node_deid_redteam "https://github.com/chadmarkey/offline-notebooklm/blob/main/src/deid_redteam.py"
click node_open_notebook "https://github.com/chadmarkey/offline-notebooklm/blob/main/patches/notebook.py"
click node_infinity "https://github.com/chadmarkey/offline-notebooklm/tree/main/infinity-arm64"
click node_transformations "https://github.com/chadmarkey/offline-notebooklm/tree/main/transformations"
click node_seed_transformations "https://github.com/chadmarkey/offline-notebooklm/blob/main/transformations/seed_transformations.py"
click node_compose "https://github.com/chadmarkey/offline-notebooklm/blob/main/docker-compose.yml"
click node_offline_control "https://github.com/chadmarkey/offline-notebooklm/blob/main/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_raw_pdfs,node_processed_pdfs,node_clean_sources toneBlue
class node_watch,node_ingest,node_phi_rules,node_deid_redteam,node_docling_openmed,node_lmstudio,node_offline_control toneAmber
class node_open_notebook,node_surrealdb,node_infinity,node_speaches,node_compose toneMint
class node_transformations,node_seed_transformations toneRose
```

## Components

### Protected ingress

Sensitive documents enter here and must be de-identified before indexing.

- **Raw PDFs**: sensitive ingress folder · Main PHI-bearing drop zone for chart-derived documents.
- **Processed PDFs**: staging folder · Original files after ingestion are moved aside here.
- **Clean sources**: de-id output folder · De-identified text and source material ready for indexing.

### Host runtime

Native tools and local model runtime on the laptop.

- [Watcher](https://github.com/chadmarkey/offline-notebooklm/blob/main/src/watch.py): filesystem bridge · Detects new PDFs and hands them to the ingest pipeline. · `src/watch.py`
- [Ingest pipeline](https://github.com/chadmarkey/offline-notebooklm/blob/main/src/ingest.py): native preprocessing · Parses PDFs, runs de-identification, and prepares clean content. · `src/ingest.py`
- [PHI rules](https://github.com/chadmarkey/offline-notebooklm/blob/main/src/phi_rules.py): rule corrections · Post-processing fixes for age, address, and other edge cases. · `src/phi_rules.py`
- [Red-team check](https://github.com/chadmarkey/offline-notebooklm/blob/main/src/deid_redteam.py): safety gate · Fixed PHI regression suite that must pass before trusting ingest. · `src/deid_redteam.py`
- **Docling + OpenMed**: local NLP stack · PDF-to-markdown conversion, PII stripping, and clinical NER tagging in Python.
- **LM Studio**: host LLM · Local LLM inference endpoint used for answer generation.
- [Offline controls](https://github.com/chadmarkey/offline-notebooklm/blob/main/README.md): operational boundary · Firewall and local-only operating assumptions that enforce confidentiality. · `README.md`

### Docker services

Containerized notebook, storage, and local model services.

- [Open Notebook](https://github.com/chadmarkey/offline-notebooklm/blob/main/patches/notebook.py): workspace app · UI/API layer for notebooks, sources, chats, retrieval, and citations. · `patches/notebook.py`
- **SurrealDB**: persistent store · Database for notebook content, metadata, and session state.
- [Infinity](https://github.com/chadmarkey/offline-notebooklm/tree/main/infinity-arm64): embeddings/rerank · OpenAI-compatible local service for embeddings and reranking. · `infinity-arm64`
- **Speaches**: audio service · Local TTS/STT service for audio features.
- [Compose stack](https://github.com/chadmarkey/offline-notebooklm/blob/main/docker-compose.yml): orchestration · Defines the local service topology and volume boundaries. · `docker-compose.yml`

### Content workflows

Reusable transformation jobs plugged into the notebook layer.

- [Transformations](https://github.com/chadmarkey/offline-notebooklm/tree/main/transformations): content workflows · Reusable source-processing templates such as flashcards and clinical pearls. · `transformations`
- [Seed presets](https://github.com/chadmarkey/offline-notebooklm/blob/main/transformations/seed_transformations.py): bootstrap job · Installs transformation presets into the notebook system. · `transformations/seed_transformations.py`

## Connections

- Raw PDFs → Watcher: new PDF · Dropped files trigger ingestion.
- Watcher → Red-team check: preflight · Safety suite guards the ingest path.
- Red-team check → Ingest pipeline: pass · Only trusted ingest proceeds.
- Watcher → Ingest pipeline: invoke · Watcher starts the native preprocessing pipeline.
- Ingest pipeline → Docling + OpenMed: convert + tag · PDFs are converted, de-identified, and clinically tagged.
- Ingest pipeline → PHI rules: normalize · Rule layer corrects model edge cases.
- Ingest pipeline → Clean sources: emit clean text · De-identified output is written for downstream use.
- Ingest pipeline → Processed PDFs: archive original · Original PDFs are moved aside after processing.
- Clean sources → Open Notebook: publish · Clean content is pushed into the notebook backend.
- Open Notebook → SurrealDB: persist · Notebook state and content are stored durably.
- Open Notebook → Infinity: embed/re-rank · Retrieval uses local embedding and reranking services.
- Open Notebook → LM Studio: generate · Final answers come from the local LLM endpoint.
- Open Notebook → Speaches: audio · Optional speech features route to the audio service.
- Transformations → Seed presets: bootstrap · Preset jobs are seeded into the workspace.
- Seed presets → Open Notebook: install · Transforms become available in the notebook layer.
- Compose stack → Open Notebook: orchestrate · Docker Compose starts the notebook app.
- Compose stack → SurrealDB: orchestrate · Compose provides the database service.
- Compose stack → Infinity: orchestrate · Compose provides the embedding/rerank service.
- Compose stack → Speaches: orchestrate · Compose provides the audio service.
- Compose stack → LM Studio: connect host · Containerized services talk to the host LLM over local network plumbing.
- Offline controls → Compose stack: egress guard · Operational firewall rules and offline mode constrain the whole stack.
