# joswikkha/ml-docs-rag architecture

> Architecture diagram of the GitHub repository joswikkha/ml-docs-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/joswikkha/ml-docs-rag
- Repository: https://github.com/joswikkha/ml-docs-rag

## Overview

This is a Streamlit RAG product for asking questions about HuggingFace model cards and scikit-learn documentation, then comparing dense, hybrid, and reranked retrieval. Its main path is document ingestion and chunking, local vector indexing, question retrieval and Groq answer generation with sources. A separate benchmark path runs RAGAS scoring and performance measurements, writes results, and presents them in a dashboard. The README documents capabilities beyond the sampled excerpts; edges are limited to relationships directly shown there or in source references.

Main entry points: [`src/run_benchmark.py`](https://github.com/joswikkha/ml-docs-rag/blob/main/src/run_benchmark.py) (Benchmark runner), [`src/rag_chain.py`](https://github.com/joswikkha/ml-docs-rag/blob/main/src/rag_chain.py) (RAG answer chain), [`src/retrievers.py`](https://github.com/joswikkha/ml-docs-rag/blob/main/src/retrievers.py) (Retrieval strategies), [`data/processed`](https://github.com/joswikkha/ml-docs-rag/tree/main/data/processed) (Processed chunks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_corpus["Corpus preparation"]
  node_ingest["Fetch and chunk<br/>[ingest.py]"]
  node_chunks["Processed chunks"]
  node_embed["Local embeddings<br/>[vectorstore.py]"]
  node_index[("Chroma index")]
end

subgraph group_answering["Question answering"]
  node_retrieval["Retrieval strategies<br/>[retrievers.py]"]
  node_rag["RAG answer chain<br/>[rag_chain.py]"]
end

subgraph group_benchmark["Benchmarking"]
  node_runner["Benchmark runner<br/>[run_benchmark.py]"]
  node_evaluation["RAGAS evaluation<br/>[evaluate.py]"]
  node_evaldata["Evaluation questions<br/>[eval_dataset.json]"]
  node_results["Benchmark results"]
end

subgraph group_presentation["Streamlit interface"]
  node_chat["QA chat page<br/>[QA_Chat.py]"]
  node_dashboard["Results dashboard"]
end

node_user(("Documentation user"))
node_hf["HuggingFace Hub"]
node_sklearn["scikit-learn docs"]
node_groq["Groq generator"]

node_hf -->|"fetches cards"| node_ingest
node_sklearn -->|"scrapes docs"| node_ingest
node_ingest -->|"saves chunks"| node_chunks
node_chunks -->|"loads chunks"| node_embed
node_embed -->|"builds index"| node_index
node_retrieval -->|"dense search"| node_index
node_chunks -->|"loads BM25 text"| node_retrieval
node_user -->|"asks question"| node_chat
node_chat -->|"invokes chain"| node_rag
node_rag -->|"retrieves context"| node_retrieval
node_rag -->|"generates answer"| node_groq
node_runner -->|"selects strategy"| node_retrieval
node_runner -->|"builds chain"| node_rag
node_runner -->|"scores answers"| node_evaluation
node_evaluation -->|"loads questions"| node_evaldata
node_runner -->|"writes results"| node_results
node_dashboard -->|"reads results"| node_results

click node_ingest "https://github.com/joswikkha/ml-docs-rag/blob/main/src/ingest.py"
click node_chunks "https://github.com/joswikkha/ml-docs-rag/tree/main/data/processed"
click node_embed "https://github.com/joswikkha/ml-docs-rag/blob/main/src/vectorstore.py"
click node_index "https://github.com/joswikkha/ml-docs-rag/tree/main/data/chroma_db"
click node_retrieval "https://github.com/joswikkha/ml-docs-rag/blob/main/src/retrievers.py"
click node_rag "https://github.com/joswikkha/ml-docs-rag/blob/main/src/rag_chain.py"
click node_chat "https://github.com/joswikkha/ml-docs-rag/blob/main/pages/QA_Chat.py"
click node_runner "https://github.com/joswikkha/ml-docs-rag/blob/main/src/run_benchmark.py"
click node_evaluation "https://github.com/joswikkha/ml-docs-rag/blob/main/src/evaluate.py"
click node_evaldata "https://github.com/joswikkha/ml-docs-rag/blob/main/data/eval/eval_dataset.json"
click node_results "https://github.com/joswikkha/ml-docs-rag/blob/main/results/benchmark_results.csv"
click node_dashboard "https://github.com/joswikkha/ml-docs-rag/blob/main/pages/Benchmark_Dashboard.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest,node_chunks,node_embed,node_index,node_user toneBlue
class node_retrieval,node_rag toneAmber
class node_runner,node_evaluation,node_evaldata,node_results toneMint
class node_chat,node_dashboard toneRose
class node_hf,node_sklearn,node_groq toneIndigo
```

## Components

### Corpus preparation

- [Fetch and chunk](https://github.com/joswikkha/ml-docs-rag/blob/main/src/ingest.py): `src/ingest.py`
- [Processed chunks](https://github.com/joswikkha/ml-docs-rag/tree/main/data/processed): `data/processed`
- [Local embeddings](https://github.com/joswikkha/ml-docs-rag/blob/main/src/vectorstore.py): `src/vectorstore.py`
- [Chroma index](https://github.com/joswikkha/ml-docs-rag/tree/main/data/chroma_db): `data/chroma_db`

### Question answering

- [Retrieval strategies](https://github.com/joswikkha/ml-docs-rag/blob/main/src/retrievers.py): `src/retrievers.py`
- [RAG answer chain](https://github.com/joswikkha/ml-docs-rag/blob/main/src/rag_chain.py): `src/rag_chain.py`

### Benchmarking

- [Benchmark runner](https://github.com/joswikkha/ml-docs-rag/blob/main/src/run_benchmark.py): `src/run_benchmark.py`
- [RAGAS evaluation](https://github.com/joswikkha/ml-docs-rag/blob/main/src/evaluate.py): `src/evaluate.py`
- [Evaluation questions](https://github.com/joswikkha/ml-docs-rag/blob/main/data/eval/eval_dataset.json): `data/eval/eval_dataset.json`
- [Benchmark results](https://github.com/joswikkha/ml-docs-rag/blob/main/results/benchmark_results.csv): `results/benchmark_results.csv`

### Streamlit interface

- [QA chat page](https://github.com/joswikkha/ml-docs-rag/blob/main/pages/QA_Chat.py): `pages/QA_Chat.py`
- [Results dashboard](https://github.com/joswikkha/ml-docs-rag/blob/main/pages/Benchmark_Dashboard.py): `pages/Benchmark_Dashboard.py`

### Other components

- **Documentation user**
- **HuggingFace Hub**
- **scikit-learn docs**
- **Groq generator**

## Connections

- HuggingFace Hub → Fetch and chunk: fetches cards (evidence: `src/ingest.py`)
- scikit-learn docs → Fetch and chunk: scrapes docs (evidence: `src/ingest.py`)
- Fetch and chunk → Processed chunks: saves chunks (evidence: `src/ingest.py`)
- Processed chunks → Local embeddings: loads chunks (evidence: `src/vectorstore.py`)
- Local embeddings → Chroma index: builds index (evidence: `src/vectorstore.py`)
- Retrieval strategies → Chroma index: dense search (evidence: `src/retrievers.py`)
- Processed chunks → Retrieval strategies: loads BM25 text (evidence: `src/retrievers.py`)
- Documentation user → QA chat page: asks question (evidence: `pages/QA_Chat.py`)
- QA chat page → RAG answer chain: invokes chain (evidence: `pages/QA_Chat.py`)
- RAG answer chain → Retrieval strategies: retrieves context (evidence: `src/rag_chain.py`)
- RAG answer chain → Groq generator: generates answer (evidence: `src/rag_chain.py`)
- Benchmark runner → Retrieval strategies: selects strategy (evidence: `src/run_benchmark.py`)
- Benchmark runner → RAG answer chain: builds chain (evidence: `src/run_benchmark.py`)
- Benchmark runner → RAGAS evaluation: scores answers (evidence: `src/run_benchmark.py`)
- RAGAS evaluation → Evaluation questions: loads questions (evidence: `src/evaluate.py`)
- Benchmark runner → Benchmark results: writes results (evidence: `src/run_benchmark.py`)
- Results dashboard → Benchmark results: reads results (evidence: `pages/Benchmark_Dashboard.py`)
