How chadmarkey/offline-notebooklm works
1. What this repo is architecturally
- This is an offline, single-laptop clinical RAG stack built around a hard boundary: anything that might contain PHI must pass through a local de-identification gate before it reaches the notebook/index.
- The repo is not just an app; it is a composed system of host tools, Docker services, a native ingest pipeline, and a few patches/customizations to keep every model local.
2. The trust boundary and data ingress
- The main boundary is the raw_pdfs/ folder. Files dropped there are treated as potentially sensitive and are the only intended ingress path for chart-derived material.
- The UI’s direct “Add Source” path is intentionally a separate, unsafe boundary: it is only for sources you can guarantee are PHI-free.
- After processing, originals are moved to raw_pdfs/_processed/, while the de-identified output goes to clean_sources/.
3. Native preprocessing / de-identification layer
- src/ingest.py is the core ingest pipeline: it parses PDFs, runs de-identification, and prepares clean text for downstream indexing.
- src/phi_rules.py contains rule-based corrections around the model output, especially edge cases like ages over 89 and address subunits.
- The README indicates this layer uses Docling for PDF-to-markdown conversion plus OpenMed for PII stripping and NER tagging, all running natively in the Python environment, not in Docker.
4. Clinical NLP gate and regression check
- src/deid_redteam.py is a safety gate for the ingest pipeline. It runs a fixed adversarial PHI test suite and must pass fully before the de-id layer is trusted.
- Architecturally, this is a preflight control: if it fails, the system should not be used for ingestion.
- The OpenMed models enrich the document, too, by attaching clinical entity tags into frontmatter before indexing.
5. File watcher as the handoff mechanism
- src/watch.py is the operational bridge between the filesystem and the notebook system.
- Its job is to notice new PDFs in raw_pdfs/, invoke the ingest/de-id pipeline, and then push cleaned content into the notebook backend through the API.
- This watcher is what turns the repo from a batch tool into a continuously usable local workspace.
6. Notebook / RAG application layer
- The main user-facing workspace is Open Notebook, run in Docker and exposed through the UI and API described in the README.
- This layer owns notebooks, sources, notes, chats, retrieval, session context, and the citation-backed answer experience.
- It is also where source selection, notebook separation, search, and transformation workflows live.
7. Local model services and their roles
- LLM inference is split from the rest of the stack and runs natively in LM Studio on the host for Metal acceleration.
- Embeddings and reranking run through Infinity in Docker; the custom infinity-arm64/ build exists to support the required Apple-silicon runtime and model endpoints.
- TTS/STT run in another Docker service (Speaches) so audio features stay local as well.
- The architecture intentionally keeps model endpoints OpenAI-compatible so Open Notebook can talk to them through standard HTTP APIs.
8. Storage and persistence boundaries
- SurrealDB is the persistent data store for notebook content and metadata.
- Docker volumes and working directories are split by purpose: surreal_data for database state, notebook_data for notebook app state, and infinity_cache for model cache.
- The sensitive content path is therefore: local parse/de-id → cleaned text → indexed/persisted in the notebook/search backend.
9. Transformations as reusable content-processing jobs
- The transformations/ folder defines reusable AI workflows that operate on sources, such as flashcards, board-style Q&A, clinical pearls, and patient-facing explanations.
- These are not separate apps; they are content-generation templates plugged into the notebook layer.
- transformations/seed_transformations.py bootstraps these presets into the system.
10. Patches and repo-specific customization
- patches/notebook.py indicates the upstream Open Notebook code is patched locally, likely to adapt behavior for the offline/local-only build.
- This is a key architectural clue: the repo is not a pure wrapper; it modifies the upstream application to fit the offline workflow and provider wiring.
11. Docker composition and service topology
- docker-compose.yml is the orchestration layer for all containerized services: Open Notebook, SurrealDB, Infinity, and Speaches.
- The network model is local-only service-to-service communication by container name, with the host LLM reached via host.docker.internal.
- The only notable non-container runtime is LM Studio on the host; everything else is designed to stay on the machine and communicate over localhost or Docker’s internal network.
12. Operational safety and offline enforcement
- The repo’s security model depends on process- and network-level controls outside the app itself, especially LuLu firewall rules and the “offline only” operating mode.
- The README explicitly treats firewall blocking as the final step, meaning the system’s confidentiality posture is an operational property, not just a code property.
- For an architecture diagram, this is important: there is a functional stack, but also an external enforcement boundary around egress.
13. How the end-to-end flow fits together
- Input: PDF lands in raw_pdfs/.
- Preprocess: watch.py triggers ingest.py, which uses Docling + OpenMed + rules to de-identify and tag.
- Publish: cleaned content is sent through the Open Notebook REST API into the notebook workspace.
- Retrieval: user query hits Open Notebook, which retrieves from SurrealDB/vector search via Infinity embeddings and reranks with the local reranker.
- Generation: LM Studio produces the final cited answer, while optional audio features go through Speaches.
14. What to show in an architecture diagram
- Draw three distinct zones: host-native services, Dockerized backend services, and the protected ingestion boundary around raw_pdfs/.
- Show the native Python ingest path as the only sanctioned bridge from sensitive documents into the notebook.
- Show Open Notebook as the orchestration/UI layer, with separate arrows to embeddings/reranking, storage, TTS/STT, and the host LLM.
- Include the safety controls as an outer boundary, not as part of the core data path.
Main entry points: notebook.py Open Notebook · docker-compose.yml Compose stack · ingest.py Ingest pipeline · watch.py Watcher
Components 17
Protected ingress
Sensitive documents enter here and must be de-identified before indexing.
- Raw PDFs · Main PHI-bearing drop zone for chart-derived documents.
- Processed PDFs · Original files after ingestion are moved aside here.
- Clean sources · De-identified text and source material ready for indexing.
Host runtime
Native tools and local model runtime on the laptop.
- Watcher · Detects new PDFs and hands them to the ingest pipeline.
src/watch.py - Ingest pipeline · Parses PDFs, runs de-identification, and prepares clean content.
src/ingest.py - PHI rules · Post-processing fixes for age, address, and other edge cases.
src/phi_rules.py - Red-team check · Fixed PHI regression suite that must pass before trusting ingest.
src/deid_redteam.py - Docling + OpenMed · PDF-to-markdown conversion, PII stripping, and clinical NER tagging in Python.
- LM Studio · Local LLM inference endpoint used for answer generation.
- Offline controls · Firewall and local-only operating assumptions that enforce confidentiality.
README.md
Docker services
Containerized notebook, storage, and local model services.
- Open Notebook · UI/API layer for notebooks, sources, chats, retrieval, and citations.
patches/notebook.py - SurrealDB · Database for notebook content, metadata, and session state.
- Infinity · OpenAI-compatible local service for embeddings and reranking.
infinity-arm64 - Speaches · Local TTS/STT service for audio features.
- Compose stack · Defines the local service topology and volume boundaries.
docker-compose.yml
Content workflows
Reusable transformation jobs plugged into the notebook layer.
- Transformations · Reusable source-processing templates such as flashcards and clinical pearls.
transformations - Seed presets · Installs transformation presets into the notebook system.
transformations/seed_transformations.py
Connections 21
- Raw PDFs → Watcher: new PDF · Dropped files trigger ingestion.
- Watcher → Red-team check: preflight · Safety suite guards the ingest path.
- Red-team check → Ingest pipeline: pass · Only trusted ingest proceeds.
- Watcher → Ingest pipeline: invoke · Watcher starts the native preprocessing pipeline.
- Ingest pipeline → Docling + OpenMed: convert + tag · PDFs are converted, de-identified, and clinically tagged.
- Ingest pipeline → PHI rules: normalize · Rule layer corrects model edge cases.
- Ingest pipeline → Clean sources: emit clean text · De-identified output is written for downstream use.
- Ingest pipeline → Processed PDFs: archive original · Original PDFs are moved aside after processing.
- Clean sources → Open Notebook: publish · Clean content is pushed into the notebook backend.
- Open Notebook → SurrealDB: persist · Notebook state and content are stored durably.
- Open Notebook → Infinity: embed/re-rank · Retrieval uses local embedding and reranking services.
- Open Notebook → LM Studio: generate · Final answers come from the local LLM endpoint.
- Open Notebook → Speaches: audio · Optional speech features route to the audio service.
- Transformations → Seed presets: bootstrap · Preset jobs are seeded into the workspace.
- Seed presets → Open Notebook: install · Transforms become available in the notebook layer.
- Compose stack → Open Notebook: orchestrate · Docker Compose starts the notebook app.
- Compose stack → SurrealDB: orchestrate · Compose provides the database service.
- Compose stack → Infinity: orchestrate · Compose provides the embedding/rerank service.
- Compose stack → Speaches: orchestrate · Compose provides the audio service.
- Compose stack → LM Studio: connect host · Containerized services talk to the host LLM over local network plumbing.
- Offline controls → Compose stack: egress guard · Operational firewall rules and offline mode constrain the whole stack.