How databricks-industry-solutions/hls-llm-doc-qa works
This repository is a Databricks solution accelerator for biomedical question answering over custom document collections. Its main workflow prepares PDFs as chunked, embedded content in Vector Search, then retrieves relevant chunks to ground a LangChain answer from a served LLM. A separate work-in-progress notebook demonstrates registering and serving a custom Llama model. The sampled notebooks show the preparation and QA stages; the README describes the overall use case. Some preparation details and the WIP deployment flow are only partially excerpted, so uncertain connections are omitted.
Main entry points: 03-LLM-Chain-with-FMAPI.py Retrieval QA chain · 02-Data-prep-with-LangChain.py PDF ingestion · [WIP] 04-Deploy-Llama-3-to-GPU-Serving.py Model registry · create-update-serving-endpoint.py Endpoint management
Components 15
Document preparation
- PDF ingestion
02-Data-prep-with-LangChain.py - Text chunking
02-Data-prep-with-LangChain.py - Text embeddings
02-Data-prep-with-LangChain.py
Question answering
- Vector retriever
03-LLM-Chain-with-FMAPI.py - Retrieval QA chain
03-LLM-Chain-with-FMAPI.py - Grounding prompt
03-LLM-Chain-with-FMAPI.py
Model deployment
- Llama model wrapper
[WIP] 04-Deploy-Llama-3-to-GPU-Serving.py - Endpoint management
util/create-update-serving-endpoint.py
Databricks data and serving
- Document table
02-Data-prep-with-LangChain.py - Vector index
02-Data-prep-with-LangChain.py - Foundation model
03-LLM-Chain-with-FMAPI.py - Model registry
[WIP] 04-Deploy-Llama-3-to-GPU-Serving.py - Model serving endpoint
Other components
- Questioner
- PDF corpus
Connections 13
- PDF corpus → PDF ingestion: loads from
02-Data-prep-with-LangChain.py - PDF ingestion → Text chunking: splits into
02-Data-prep-with-LangChain.py - Text chunking → Text embeddings: embeds chunks
02-Data-prep-with-LangChain.py - Text embeddings → Document table: writes documents
02-Data-prep-with-LangChain.py - Document table → Vector index: syncs index
02-Data-prep-with-LangChain.py - Questioner → Retrieval QA chain: asks question
03-LLM-Chain-with-FMAPI.py - Retrieval QA chain → Vector retriever: retrieves chunks
03-LLM-Chain-with-FMAPI.py - Vector retriever → Vector index: searches index
03-LLM-Chain-with-FMAPI.py - Retrieval QA chain → Grounding prompt: formats context
03-LLM-Chain-with-FMAPI.py - Retrieval QA chain → Foundation model: invokes model
03-LLM-Chain-with-FMAPI.py - Llama model wrapper → Model registry: logs model
[WIP] 04-Deploy-Llama-3-to-GPU-Serving.py - Endpoint management → Model serving endpoint: creates or updates
util/create-update-serving-endpoint.py - Model registry → Model serving endpoint: deploys model
[WIP] 04-Deploy-Llama-3-to-GPU-Serving.py