# pathwaycom/llm-app architecture

> Architecture diagram of the GitHub repository pathwaycom/llm-app, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pathwaycom/llm-app
- Repository: https://github.com/pathwaycom/llm-app

## Overview

#### Purpose and Repository Shape
This is a catalog of independently deployable Python AI-pipeline templates built on Pathway Live Data Framework. It is not one unified application: each template owns its runtime entry point, dependencies, container image, and often deployment metadata. Primary executable entry points are template-local `app.py` files, e.g. `templates/question_answering_rag/app.py`, `templates/document_indexing/app.py`, and `templates/unstructured_to_sql_on_the_fly/app.py`.

#### Shared Execution Model
Templates use Pathway for continuously synchronized ingestion, in-memory transformation/indexing, and HTTP request serving. Pathway is a Python library backed by a Rust engine; it consolidates streaming data processing, retrieval index, cache, and API serving rather than requiring an external vector DB, Redis, or FastAPI service. Template containers are defined by paths such as `templates/question_answering_rag/Dockerfile` and `templates/multimodal_rag/Dockerfile`.

#### Live Ingestion and Indexing
The dominant flow is: connected document source → parsing/chunking/embedding → continuously updated search index → HTTP retrieval endpoint → optional LLM-generated answer. File-backed sample corpora illustrate source boundaries, including `templates/document_indexing/files-for-indexing/` and `templates/question_answering_rag/data/`. The framework supports filesystem and external live sources such as Google Drive, SharePoint, S3, Kafka, PostgreSQL, and real-time APIs.

#### RAG Application Variants
Core RAG examples provide alternate retrieval and generation designs:
- Basic hosted-model QA RAG: `templates/question_answering_rag/app.py`, with optional Streamlit client at `templates/question_answering_rag/ui/ui.py`.
- Adaptive RAG, intended to reduce retrieval/generation token cost: `templates/adaptive_rag/app.py`.
- Local/private RAG using Mistral and Ollama: `templates/private_rag/app.py`.
- GPT-4o-assisted multimodal document parsing and RAG: `templates/multimodal_rag/app.py`.
- Video retrieval using TwelveLabs descriptions and multimodal embeddings: `templates/video_rag_twelvelabs/app.py`.

#### Retrieval Service Boundary
`templates/document_indexing/app.py` is the focused live vector-store/retriever service, intended to be consumed by arbitrary frontends or LangChain/LlamaIndex applications rather than necessarily generating answers itself. Pathway’s built-in vector indexing uses usearch; hybrid/full-text retrieval uses Tantivy. A related MCP-facing document-store variant is exposed from `templates/document_store_mcp_server/app.py`, with local multi-service execution in `templates/document_store_mcp_server/docker-compose.yml`.

#### Structured-Data and Domain Pipelines
The unstructured-to-SQL template separates document extraction from relational query execution: financial PDFs are transformed into structured data, loaded into PostgreSQL, and queried through LLM-generated SQL. Its pipeline entry is `templates/unstructured_to_sql_on_the_fly/app.py`; database initialization is `templates/unstructured_to_sql_on_the_fly/postgres/init-db.sql`; orchestration is `templates/unstructured_to_sql_on_the_fly/docker-compose.yml`. This is the main template with a material external persistence boundary.

#### Deployment and User-Facing Surfaces
Most templates package backend deployments with `Dockerfile`, Python requirements, and Pathway deployment descriptors (`app.yaml`), for example `templates/adaptive_rag/app.yaml` and `templates/slides_ai_search/app.yaml`. Multi-container stacks use Compose where a UI, reverse proxy, or database is needed: `templates/slides_ai_search/docker-compose.yml` includes an Nginx component at `templates/slides_ai_search/nginx/nginx.conf`; UI-capable templates use Streamlit or lightweight Python servers such as `templates/drive_alert/ui/server.py`.

Main entry points: [`templates/document_indexing/app.py`](https://github.com/pathwaycom/llm-app/blob/main/templates/document_indexing/app.py) (Document indexing service), [`templates/unstructured_to_sql_on_the_fly/app.py`](https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/app.py) (Unstructured-to-SQL), [`templates/unstructured_to_sql_on_the_fly/docker-compose.yml`](https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/docker-compose.yml) (Local multi-service stacks), [`templates/question_answering_rag/app.py`](https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/app.py) (Hosted QA RAG)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_framework["Shared runtime"]
  node_sources(("Live document sources<br/>source connectors"))
  node_pathway{{"Pathway runtime<br/>Python + Rust engine"}}
  node_processing["Parse, chunk, embed<br/>ingestion pipeline"]
  node_search[("Live search index<br/>usearch + Tantivy")]
  node_http["HTTP retrieval API<br/>serving endpoint"]
end

subgraph group_rag["RAG templates"]
  node_qa_rag["Hosted QA RAG<br/>template app<br/>[app.py]"]
  node_qa_ui["QA Streamlit UI<br/>client UI<br/>[ui.py]"]
  node_adaptive["Adaptive RAG<br/>template app<br/>[app.py]"]
  node_private["Private RAG<br/>template app<br/>[app.py]"]
  node_multimodal["Multimodal RAG<br/>template app<br/>[app.py]"]
  node_video["Video RAG<br/>template app<br/>[app.py]"]
end

subgraph group_services["Specialized pipelines"]
  node_indexing["Document indexing service<br/>retriever template<br/>[app.py]"]
  node_mcp["MCP document store<br/>MCP server template<br/>[app.py]"]
  node_sql_pipeline["Unstructured-to-SQL<br/>template pipeline<br/>[app.py]"]
  node_postgres[("PostgreSQL<br/>external persistence<br/>[init-db.sql]")]
end

subgraph group_deployment["Deployment surfaces"]
  node_containers["Template containers<br/>Docker images"]
  node_compose["Local multi-service stacks<br/>Docker Compose<br/>[docker-compose.yml]"]
  node_nginx["Slides reverse proxy<br/>Nginx<br/>[nginx.conf]"]
end

node_sources -->|"streams into"| node_pathway
node_pathway -->|"runs"| node_processing
node_processing -->|"updates"| node_search
node_search -->|"serves retrieval"| node_http
node_qa_rag -->|"uses"| node_pathway
node_qa_ui -->|"queries"| node_qa_rag
node_adaptive -->|"uses"| node_pathway
node_private -->|"uses"| node_pathway
node_multimodal -->|"uses"| node_pathway
node_video -->|"uses"| node_pathway
node_indexing -->|"exposes"| node_http
node_indexing -->|"builds on"| node_pathway
node_mcp -.->|"MCP-facing variant"| node_indexing
node_sql_pipeline -->|"extracts with"| node_pathway
node_sql_pipeline -->|"loads and queries"| node_postgres
node_containers -.->|"packages"| node_qa_rag
node_compose -.->|"orchestrates"| node_sql_pipeline
node_compose -.->|"starts"| node_postgres
node_nginx -.->|"included by slides stack"| node_compose

click node_qa_rag "https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/app.py"
click node_qa_ui "https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/ui/ui.py"
click node_adaptive "https://github.com/pathwaycom/llm-app/blob/main/templates/adaptive_rag/app.py"
click node_private "https://github.com/pathwaycom/llm-app/blob/main/templates/private_rag/app.py"
click node_multimodal "https://github.com/pathwaycom/llm-app/blob/main/templates/multimodal_rag/app.py"
click node_video "https://github.com/pathwaycom/llm-app/blob/main/templates/video_rag_twelvelabs/app.py"
click node_indexing "https://github.com/pathwaycom/llm-app/blob/main/templates/document_indexing/app.py"
click node_mcp "https://github.com/pathwaycom/llm-app/blob/main/templates/document_store_mcp_server/app.py"
click node_sql_pipeline "https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/app.py"
click node_postgres "https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/postgres/init-db.sql"
click node_containers "https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/Dockerfile"
click node_compose "https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/docker-compose.yml"
click node_nginx "https://github.com/pathwaycom/llm-app/blob/main/templates/slides_ai_search/nginx/nginx.conf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sources,node_pathway,node_processing,node_search,node_http toneBlue
class node_qa_rag,node_qa_ui,node_adaptive,node_private,node_multimodal,node_video toneAmber
class node_indexing,node_mcp,node_sql_pipeline,node_postgres toneMint
class node_containers,node_compose,node_nginx toneRose
```

## Components

### Shared runtime

Pathway provides continuous processing, indexing, and HTTP serving.

- **Live document sources**: source connectors · Filesystem and external sources continuously feed templates.
- **Pathway runtime**: Python + Rust engine · Combines streaming transforms, retrieval, cache, and HTTP serving.
- **Parse, chunk, embed**: ingestion pipeline
- **Live search index**: usearch + Tantivy · Continuously updated vector, hybrid, and full-text retrieval.
- **HTTP retrieval API**: serving endpoint

### RAG templates

Independently deployable retrieval and answer-generation examples.

- [Hosted QA RAG](https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/app.py): template app · `templates/question_answering_rag/app.py`
- [QA Streamlit UI](https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/ui/ui.py): client UI · `templates/question_answering_rag/ui/ui.py`
- [Adaptive RAG](https://github.com/pathwaycom/llm-app/blob/main/templates/adaptive_rag/app.py): template app · Retrieval and generation design optimized for token cost. · `templates/adaptive_rag/app.py`
- [Private RAG](https://github.com/pathwaycom/llm-app/blob/main/templates/private_rag/app.py): template app · Local Mistral and Ollama-based RAG. · `templates/private_rag/app.py`
- [Multimodal RAG](https://github.com/pathwaycom/llm-app/blob/main/templates/multimodal_rag/app.py): template app · GPT-4o-assisted document parsing and RAG. · `templates/multimodal_rag/app.py`
- [Video RAG](https://github.com/pathwaycom/llm-app/blob/main/templates/video_rag_twelvelabs/app.py): template app · TwelveLabs descriptions and multimodal embeddings. · `templates/video_rag_twelvelabs/app.py`

### Specialized pipelines

Focused retrieval, MCP, and structured-data templates.

- [Document indexing service](https://github.com/pathwaycom/llm-app/blob/main/templates/document_indexing/app.py): retriever template · Live vector-store API for arbitrary clients and frameworks. · `templates/document_indexing/app.py`
- [MCP document store](https://github.com/pathwaycom/llm-app/blob/main/templates/document_store_mcp_server/app.py): MCP server template · `templates/document_store_mcp_server/app.py`
- [Unstructured-to-SQL](https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/app.py): template pipeline · Extracts financial PDFs and serves LLM-generated relational queries. · `templates/unstructured_to_sql_on_the_fly/app.py`
- [PostgreSQL](https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/postgres/init-db.sql): external persistence · Structured extracted data and SQL execution boundary. · `templates/unstructured_to_sql_on_the_fly/postgres/init-db.sql`

### Deployment surfaces

Containers and multi-service local stacks.

- [Template containers](https://github.com/pathwaycom/llm-app/blob/main/templates/question_answering_rag/Dockerfile): Docker images · Each template owns its runtime image and dependencies. · `templates/question_answering_rag/Dockerfile`
- [Local multi-service stacks](https://github.com/pathwaycom/llm-app/blob/main/templates/unstructured_to_sql_on_the_fly/docker-compose.yml): Docker Compose · Orchestrates UI, proxy, database, or MCP companion services. · `templates/unstructured_to_sql_on_the_fly/docker-compose.yml`
- [Slides reverse proxy](https://github.com/pathwaycom/llm-app/blob/main/templates/slides_ai_search/nginx/nginx.conf): Nginx · `templates/slides_ai_search/nginx/nginx.conf`

## Connections

- Live document sources → Pathway runtime: streams into
- Pathway runtime → Parse, chunk, embed: runs
- Parse, chunk, embed → Live search index: updates
- Live search index → HTTP retrieval API: serves retrieval
- Hosted QA RAG → Pathway runtime: uses
- QA Streamlit UI → Hosted QA RAG: queries
- Adaptive RAG → Pathway runtime: uses
- Private RAG → Pathway runtime: uses
- Multimodal RAG → Pathway runtime: uses
- Video RAG → Pathway runtime: uses
- Document indexing service → HTTP retrieval API: exposes
- Document indexing service → Pathway runtime: builds on
- MCP document store → Document indexing service: MCP-facing variant
- Unstructured-to-SQL → Pathway runtime: extracts with
- Unstructured-to-SQL → PostgreSQL: loads and queries
- Template containers → Hosted QA RAG: packages
- Local multi-service stacks → Unstructured-to-SQL: orchestrates
- Local multi-service stacks → PostgreSQL: starts
- Slides reverse proxy → Local multi-service stacks: included by slides stack
