# datastax/graph-rag-example architecture

> Architecture diagram of the GitHub repository datastax/graph-rag-example, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datastax/graph-rag-example
- Repository: https://github.com/datastax/graph-rag-example

## Overview

##### Project type and purpose
- This is a Python-based RAG (Retrieval-Augmented Generation) demo app that:
- Ingests structured content (movie pages) from TMDB via a loader script.
- Stores embeddings and graph relationships in a Cassandra-backed graph vector store (DataStax Astra DB).
- Exposes a Dash UI for real-time comparison of vector similarity vs. graph traversal queries, powered by an OpenAI LLM.

##### File structure
- Top-level scripts:
- load_data.py – ingestion pipeline (scraping, processing, upserting into vector/graph store)
- app.py – Dash application entry point
- search_executor.py – encapsulates search logic (similarity & graph traversal)
- Utilities in util/:
- config.py – environment variable parsing (Astra DB credentials, OpenAI key)
- scrub.py – text cleaning before embedding
- visualization.py – helpers for building UI graphs or charts
- assets/: static files (CSS, sample movies.json)
- requirements.txt & .env.example – dependencies and configuration boilerplate

##### README for additional insights
- Confirms tech stack: Python 3, Dash (Plotly), LangChain graph_vectorstores, CassandraGraphVectorStore (DataStax Astra), OpenAI API
- Usage flow:
1. load_data.py → pulls & preprocesses TMDB content
2. app.py → runs Dash UI for user queries, delegates to search_executor

a. Main components:
1. External Data Source (TMDB API)
2. Data Ingestion Layer (load_data.py)
3. Data Processing & Utilities (util/scrub.py, util/config.py)
4. Graph Vector Store (DataStax Astra Cassandra)
5. Search Layer (search_executor.py)
6. LLM Service (OpenAI API)
7. Presentation Layer (Dash UI in app.py)
b. Relationships & interactions:
- load_data.py → scrub.py (text cleaning) → CassandraGraphVectorStore (“upsert embeddings & graph edges”)
- User (browser) → Dash UI (HTTP) → search_executor
- search_executor → CassandraGraphVectorStore (two paths: similarity search & graph traversal)
- search_executor → OpenAI API (prompting & response generation)
- search_executor → Dash UI (returns results for rendering)
c. Architectural patterns & principles:
- Monolithic service divided into clear layers: ingestion, storage, search logic, presentation
- RAG pattern: combine vector similarity & structured graph traversal
- Single source of truth for config (.env → util/config.py)
- Separation of concerns: utilities (scrubbing, visualization) factored out
d. Key technologies/frameworks:
- Python 3 + LangChain graph_vectorstores
- DataStax Astra DB (CassandraGraphVectorStore)
- OpenAI LLM API
- Dash (Plotly) for UI

##### Detailed capture of essential elements
- Open ports or protocols where relevant (HTTP, REST API calls)
- If desired, annotate critical configuration parameters (Astra DB endpoint, OpenAI key) adjacent to util/config.py

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Sources
    TMDB["TMDB API"]:::external

    %% Ingestion Layer
    subgraph "Ingestion Layer"
        direction TB
        LoadData["load_data.py"]:::ingestion
        ConfigIngest["util/config.py"]:::util
        Scrub["util/scrub.py"]:::util
    end

    %% Storage Layer
    Cassandra["CassandraGraphVectorStore"]:::storage

    %% Search & LLM Layer
    subgraph "Search Layer"
        direction TB
        SearchExec["search_executor.py"]:::search
        ConfigSearch["util/config.py"]:::util
        Visualization["util/visualization.py"]:::util
        OpenAI["OpenAI API"]:::external
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        DashUI["app.py (Dash UI)"]:::ui
        AssetsCSS["assets/globals.css"]:::ui
        AssetsData["assets/movies.json"]:::ui
    end

    %% Configuration & Dependencies
    Env[".env.example"]:::config
    Req["requirements.txt"]:::config

    %% Flows
    TMDB -->|"fetch raw pages"| LoadData
    LoadData -->|"load config"| ConfigIngest
    LoadData -->|"clean text"| Scrub
    LoadData -->|"upsert embeddings & graph edges"| Cassandra

    DashUI -->|"HTTP request (user query)"| SearchExec
    SearchExec -->|"load config"| ConfigSearch
    SearchExec -->|"vector similarity & graph traverse"| Cassandra
    SearchExec -->|"prompt & response"| OpenAI
    SearchExec -->|"build visuals"| Visualization
    SearchExec -->|"return results"| DashUI

    Env -->|".env vars"| ConfigIngest
    Env -->|".env vars"| ConfigSearch
    Req -->|"install deps"| LoadData
    Req -->|"install deps"| DashUI
    Req -->|"install deps"| SearchExec

    %% Click Events
    click LoadData "https://github.com/datastax/graph-rag-example/blob/main/load_data.py"
    click ConfigIngest "https://github.com/datastax/graph-rag-example/blob/main/util/config.py"
    click Scrub "https://github.com/datastax/graph-rag-example/blob/main/util/scrub.py"
    click Visualization "https://github.com/datastax/graph-rag-example/blob/main/util/visualization.py"
    click SearchExec "https://github.com/datastax/graph-rag-example/blob/main/search_executor.py"
    click DashUI "https://github.com/datastax/graph-rag-example/blob/main/app.py"
    click AssetsCSS "https://github.com/datastax/graph-rag-example/blob/main/assets/globals.css"
    click AssetsData "https://github.com/datastax/graph-rag-example/blob/main/assets/movies.json"
    click Env "https://github.com/datastax/graph-rag-example/blob/main/.env.example"
    click Req "https://github.com/datastax/graph-rag-example/blob/main/requirements.txt"

    %% Styles
    classDef external fill:#f2f2f2,stroke:#999,stroke-width:1px
    classDef ingestion fill:#add8e6,stroke:#0366d6,stroke-width:2px
    classDef util fill:#e0f7fa,stroke:#00796b,stroke-width:1px
    classDef storage fill:#98fb98,stroke:#2e7d32,stroke-width:2px
    classDef search fill:#ffa500,stroke:#e65100,stroke-width:2px
    classDef ui fill:#dda0dd,stroke:#6a1b9a,stroke-width:2px
    classDef config fill:#fffacd,stroke:#f9a825,stroke-width:1px
```
