# tarun1125/csaiml-capstone-project-20 architecture

> Architecture diagram of the GitHub repository tarun1125/csaiml-capstone-project-20, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tarun1125/csaiml-capstone-project-20
- Repository: https://github.com/tarun1125/csaiml-capstone-project-20

## Overview

This is a benchmark and demonstration project for natural-language-to-PyMongo query generation, comparing zero-shot, retrieval-augmented, and LoRA-adapted Qwen arms. Its main workflow takes verified question/query pairs, creates a held-out set, generates queries, and measures correctness by executing them against MongoDB Atlas and comparing results with executed gold answers. RAG retrieval and fine-tuning are distinct preparation paths. The README documents the overall workflow; sampled source confirms parts of generation, demo inference, and execution/scoring, so uncertain implementation links are omitted.

Main entry points: [`rag/data/rag_test.json`](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_test.json) (Held-out test cases), [`evaluation/execute_queries.py`](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/evaluation/execute_queries.py) (Execute and compare), [`rag/build_prompts.py`](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_prompts.py) (Retrieved prompts), [`normalize.py`](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/normalize.py) (Query normalization)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_corpus["Dataset and gold"]
  node_queries["Verified query pairs"]
  node_gold["Gold executions<br/>[gold_results.json]"]
  node_split["Few-shot and test split<br/>[build_split.py]"]
  node_pool["Few-shot pool"]
  node_test["Held-out test cases<br/>[rag_test.json]"]
end

subgraph group_rag["RAG arm"]
  node_index[("FAISS retrieval index")]
  node_metadata["Retrieval metadata"]
  node_prompts["Retrieved prompts<br/>[build_prompts.py]"]
  node_ragprompts["RAG prompt set<br/>[rag_prompts.json]"]
end

subgraph group_models["Generation arms"]
  node_baseline["Baseline generation"]
  node_raggen["RAG generation"]
  node_ftdata["LoRA training data"]
  node_training["LoRA training"]
  node_adapter["23-database adapter"]
  node_ftgen["Fine-tuned generation"]
end

subgraph group_evaluation["Execution evaluation"]
  node_normalize["Query normalization<br/>[normalize.py]"]
  node_execute["Execute and compare<br/>[execute_queries.py]"]
  node_ragscore["RAG scoring<br/>[score_rag.py]"]
  node_results["Execution results"]
end

subgraph group_demo["Interactive demo"]
  node_demo["Streamlit demo<br/>[app.py]"]
  node_live["Live inference wrapper<br/>[live_inference.py]"]
  node_curated["Curated examples"]
end

node_user(("Engineer / evaluator"))
node_atlas[("MongoDB Atlas")]

node_queries -->|"executes gold queries"| node_gold
node_queries -->|"splits cases"| node_split
node_split -->|"produces pool"| node_pool
node_split -->|"produces holdout"| node_test
node_pool -->|"embeds questions"| node_index
node_index -->|"retrieves neighbors"| node_prompts
node_metadata -->|"resolves neighbors"| node_prompts
node_test -->|"builds prompts"| node_prompts
node_prompts -->|"writes prompts"| node_ragprompts
node_test -->|"provides cases"| node_baseline
node_ragprompts -->|"provides prompts"| node_raggen
node_pool -->|"forms training data"| node_ftdata
node_ftdata -->|"trains on cases"| node_training
node_training -->|"produces adapter"| node_adapter
node_test -->|"provides cases"| node_ftgen
node_adapter -->|"loads adapter"| node_ftgen
node_baseline -->|"normalizes generated queries"| node_normalize
node_raggen -->|"normalizes generated queries"| node_normalize
node_ftgen -->|"normalizes generated queries"| node_normalize
node_normalize -->|"provides normalized queries"| node_execute
node_execute -->|"executes queries"| node_atlas
node_execute -->|"compares results"| node_gold
node_execute -->|"writes scores"| node_results
node_ragscore -->|"executes RAG and baseline"| node_atlas
node_ragscore -->|"compares outputs"| node_gold
node_ragscore -->|"writes scores"| node_results
node_user -->|"chooses arm"| node_demo
node_demo -->|"renders examples"| node_curated
node_demo -->|"dispatches inference"| node_live
node_live -.->|"optionally executes query"| node_atlas

click node_queries "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/data/reference_queries.json"
click node_gold "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/data/gold_results.json"
click node_split "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_split.py"
click node_pool "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_fewshot_pool.json"
click node_test "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_test.json"
click node_index "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_retrieval_index.py"
click node_metadata "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/fewshot_metadata.json"
click node_prompts "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_prompts.py"
click node_ragprompts "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_prompts.json"
click node_baseline "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/generate_baseline_mlx.py"
click node_raggen "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/generate_rag_mlx.py"
click node_ftdata "https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/fine_tuning/data_23db"
click node_training "https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/fine_tuning"
click node_adapter "https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/fine_tuning/adapters_23db_1000iter"
click node_ftgen "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/fine_tuning/generate_predictions_23db.py"
click node_normalize "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/normalize.py"
click node_execute "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/evaluation/execute_queries.py"
click node_ragscore "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/score_rag.py"
click node_results "https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/data"
click node_demo "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/demo_ui/app.py"
click node_live "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/demo_ui/live_inference.py"
click node_curated "https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/demo_ui/curated_examples.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_queries,node_gold,node_split,node_pool,node_test toneBlue
class node_index,node_metadata,node_prompts,node_ragprompts,node_atlas toneAmber
class node_baseline,node_raggen,node_ftdata,node_training,node_adapter,node_ftgen toneMint
class node_normalize,node_execute,node_ragscore,node_results toneRose
class node_demo,node_live,node_curated,node_user toneIndigo
```

## Components

### Dataset and gold

- [Verified query pairs](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/data/reference_queries.json): `data/reference_queries.json`
- [Gold executions](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/data/gold_results.json): `data/gold_results.json`
- [Few-shot and test split](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_split.py): `rag/build_split.py`
- [Few-shot pool](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_fewshot_pool.json): `rag/data/rag_fewshot_pool.json`
- [Held-out test cases](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_test.json): `rag/data/rag_test.json`

### RAG arm

- [FAISS retrieval index](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_retrieval_index.py): `rag/build_retrieval_index.py`
- [Retrieval metadata](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/fewshot_metadata.json): `rag/data/fewshot_metadata.json`
- [Retrieved prompts](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/build_prompts.py): `rag/build_prompts.py`
- [RAG prompt set](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/data/rag_prompts.json): `rag/data/rag_prompts.json`

### Generation arms

- [Baseline generation](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/generate_baseline_mlx.py): `generate_baseline_mlx.py`
- [RAG generation](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/generate_rag_mlx.py): `rag/generate_rag_mlx.py`
- [LoRA training data](https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/fine_tuning/data_23db): `fine_tuning/data_23db`
- [LoRA training](https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/fine_tuning): `fine_tuning`
- [23-database adapter](https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/fine_tuning/adapters_23db_1000iter): `fine_tuning/adapters_23db_1000iter`
- [Fine-tuned generation](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/fine_tuning/generate_predictions_23db.py): `fine_tuning/generate_predictions_23db.py`

### Execution evaluation

- [Query normalization](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/normalize.py): `normalize.py`
- [Execute and compare](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/evaluation/execute_queries.py): `evaluation/execute_queries.py`
- [RAG scoring](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/rag/score_rag.py): `rag/score_rag.py`
- [Execution results](https://github.com/tarun1125/csaiml-capstone-project-20/tree/main/data): `data`

### Interactive demo

- [Streamlit demo](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/demo_ui/app.py): `demo_ui/app.py`
- [Live inference wrapper](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/demo_ui/live_inference.py): `demo_ui/live_inference.py`
- [Curated examples](https://github.com/tarun1125/csaiml-capstone-project-20/blob/main/demo_ui/curated_examples.py): `demo_ui/curated_examples.py`

### Other components

- **Engineer / evaluator**
- **MongoDB Atlas**

## Connections

- Verified query pairs → Gold executions: executes gold queries
- Verified query pairs → Few-shot and test split: splits cases
- Few-shot and test split → Few-shot pool: produces pool
- Few-shot and test split → Held-out test cases: produces holdout
- Few-shot pool → FAISS retrieval index: embeds questions
- FAISS retrieval index → Retrieved prompts: retrieves neighbors
- Retrieval metadata → Retrieved prompts: resolves neighbors
- Held-out test cases → Retrieved prompts: builds prompts
- Retrieved prompts → RAG prompt set: writes prompts
- Held-out test cases → Baseline generation: provides cases
- RAG prompt set → RAG generation: provides prompts
- Few-shot pool → LoRA training data: forms training data
- LoRA training data → LoRA training: trains on cases
- LoRA training → 23-database adapter: produces adapter
- Held-out test cases → Fine-tuned generation: provides cases
- 23-database adapter → Fine-tuned generation: loads adapter
- Baseline generation → Query normalization: normalizes generated queries
- RAG generation → Query normalization: normalizes generated queries
- Fine-tuned generation → Query normalization: normalizes generated queries
- Query normalization → Execute and compare: provides normalized queries
- Execute and compare → MongoDB Atlas: executes queries
- Execute and compare → Gold executions: compares results
- Execute and compare → Execution results: writes scores
- RAG scoring → MongoDB Atlas: executes RAG and baseline
- RAG scoring → Gold executions: compares outputs
- RAG scoring → Execution results: writes scores
- Engineer / evaluator → Streamlit demo: chooses arm
- Streamlit demo → Curated examples: renders examples
- Streamlit demo → Live inference wrapper: dispatches inference
- Live inference wrapper → MongoDB Atlas: optionally executes query