# sameershaikh55/utility-aware-rag-opm architecture

> Architecture diagram of the GitHub repository sameershaikh55/utility-aware-rag-opm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sameershaikh55/utility-aware-rag-opm
- Repository: https://github.com/sameershaikh55/utility-aware-rag-opm

## Overview

This repository is an interactive utility-aware RAG evaluator. A user submits a query through the Streamlit dashboard; the OPM engine retrieves corpus evidence, elicits and aggregates grounded model samples, then selects an answer or abstention using expected utility. The dashboard also exposes benchmark telemetry, while the evaluation runner computes calibration and utility metrics. The graph follows the sampled calls and README capabilities; unsampled implementation details are represented only where the README supports them.

Main entry points: [`src/opm_engine.py`](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/opm_engine.py) (OPM engine), [`src/eval.py`](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/eval.py) (Benchmark evaluation), [`src/app.py`](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/app.py) (Streamlit dashboard), [`src/retriever.py`](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/retriever.py) (Corpus retrieval)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["Query experience"]
  node_dashboard["Streamlit dashboard<br/>[app.py]"]
end

subgraph group_decision["OPM decision pipeline"]
  node_engine["OPM engine<br/>[opm_engine.py]"]
  node_retrieval["Corpus retrieval<br/>[retriever.py]"]
  node_corpus["Domain corpus"]
  node_prompting["Prompt templates<br/>[prompts.py]"]
  node_prompt_yaml["Versioned prompt<br/>[v1.yaml]"]
  node_groq{{"Groq models"}}
end

subgraph group_evaluation["Evaluation and telemetry"]
  node_benchmark["Benchmark evaluation<br/>[eval.py]"]
  node_golden["Golden queries<br/>[golden_eval.jsonl]"]
  node_results["Evaluation reports"]
end

subgraph group_support["Shared runtime support"]
  node_contracts["Result contracts<br/>[schema.py]"]
  node_config["Runtime configuration<br/>[config.py]"]
  node_io["File I/O<br/>[io_utils.py]"]
  node_logger["Structured logging<br/>[logger.py]"]
end

node_user(("User"))

node_user -->|"submits query"| node_dashboard
node_dashboard -->|"runs query"| node_engine
node_engine -->|"retrieves chunks"| node_retrieval
node_retrieval -->|"reads corpus"| node_corpus
node_retrieval -->|"reads Markdown"| node_io
node_engine -->|"loads prompts"| node_prompting
node_prompting -->|"loads YAML"| node_prompt_yaml
node_engine -->|"requests samples"| node_groq
node_engine -->|"produces decision"| node_contracts
node_dashboard -->|"renders results"| node_contracts
node_engine -->|"uses settings"| node_config
node_retrieval -->|"uses corpus path"| node_config
node_engine -->|"logs events"| node_logger
node_prompting -->|"uses prompt path"| node_config
node_dashboard -->|"runs telemetry"| node_benchmark
node_benchmark -->|"loads records"| node_golden
node_benchmark -->|"evaluates queries"| node_engine
node_benchmark -->|"validates reports"| node_contracts
node_benchmark -->|"reads and writes"| node_io
node_benchmark -->|"saves report"| node_results
node_benchmark -->|"logs report"| node_logger

click node_dashboard "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/app.py"
click node_engine "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/opm_engine.py"
click node_retrieval "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/retriever.py"
click node_corpus "https://github.com/sameershaikh55/utility-aware-rag-opm/tree/main/data/corpus"
click node_prompting "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/prompts.py"
click node_prompt_yaml "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/prompts/rag_opm/v1.yaml"
click node_contracts "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/schema.py"
click node_config "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/config.py"
click node_io "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/io_utils.py"
click node_logger "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/logger.py"
click node_benchmark "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/eval.py"
click node_golden "https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/data/golden_eval.jsonl"
click node_results "https://github.com/sameershaikh55/utility-aware-rag-opm/tree/main/results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_user toneBlue
class node_engine,node_retrieval,node_corpus,node_prompting,node_prompt_yaml,node_groq toneAmber
class node_benchmark,node_golden,node_results toneMint
class node_contracts,node_config,node_io,node_logger toneRose
```

## Components

### Query experience

- [Streamlit dashboard](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/app.py): `src/app.py`

### OPM decision pipeline

- [OPM engine](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/opm_engine.py): `src/opm_engine.py`
- [Corpus retrieval](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/retriever.py): `src/retriever.py`
- [Domain corpus](https://github.com/sameershaikh55/utility-aware-rag-opm/tree/main/data/corpus): `data/corpus`
- [Prompt templates](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/prompts.py): `src/prompts.py`
- [Versioned prompt](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/prompts/rag_opm/v1.yaml): `prompts/rag_opm/v1.yaml`
- **Groq models**

### Evaluation and telemetry

- [Benchmark evaluation](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/eval.py): `src/eval.py`
- [Golden queries](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/data/golden_eval.jsonl): `data/golden_eval.jsonl`
- [Evaluation reports](https://github.com/sameershaikh55/utility-aware-rag-opm/tree/main/results): `results`

### Shared runtime support

- [Result contracts](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/schema.py): `src/schema.py`
- [Runtime configuration](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/config.py): `src/config.py`
- [File I/O](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/io_utils.py): `src/io_utils.py`
- [Structured logging](https://github.com/sameershaikh55/utility-aware-rag-opm/blob/main/src/logger.py): `src/logger.py`

### Other components

- **User**

## Connections

- User → Streamlit dashboard: submits query (evidence: `README.md`)
- Streamlit dashboard → OPM engine: runs query (evidence: `src/app.py`)
- OPM engine → Corpus retrieval: retrieves chunks (evidence: `src/opm_engine.py`)
- Corpus retrieval → Domain corpus: reads corpus (evidence: `README.md`)
- Corpus retrieval → File I/O: reads Markdown (evidence: `src/retriever.py`)
- OPM engine → Prompt templates: loads prompts (evidence: `src/opm_engine.py`)
- Prompt templates → Versioned prompt: loads YAML (evidence: `src/prompts.py`)
- OPM engine → Groq models: requests samples (evidence: `src/opm_engine.py`)
- OPM engine → Result contracts: produces decision (evidence: `src/opm_engine.py`)
- Streamlit dashboard → Result contracts: renders results (evidence: `src/app.py`)
- OPM engine → Runtime configuration: uses settings (evidence: `src/opm_engine.py`)
- Corpus retrieval → Runtime configuration: uses corpus path (evidence: `src/retriever.py`)
- OPM engine → Structured logging: logs events (evidence: `src/opm_engine.py`)
- Prompt templates → Runtime configuration: uses prompt path (evidence: `src/prompts.py`)
- Streamlit dashboard → Benchmark evaluation: runs telemetry (evidence: `README.md`)
- Benchmark evaluation → Golden queries: loads records (evidence: `src/eval.py`)
- Benchmark evaluation → OPM engine: evaluates queries (evidence: `src/eval.py`)
- Benchmark evaluation → Result contracts: validates reports (evidence: `src/eval.py`)
- Benchmark evaluation → File I/O: reads and writes (evidence: `src/eval.py`)
- Benchmark evaluation → Evaluation reports: saves report (evidence: `src/eval.py`)
- Benchmark evaluation → Structured logging: logs report (evidence: `src/eval.py`)