# rishabhjain2003/shl-copilot architecture

> Architecture diagram of the GitHub repository rishabhjain2003/shl-copilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rishabhjain2003/shl-copilot
- Repository: https://github.com/rishabhjain2003/shl-copilot

## Overview

1. **System purpose**
   - This repo is a **stateless conversational recommender** for SHL Individual Test Solutions.
   - It takes a chat history, classifies the user’s intent, retrieves candidate assessments from a local catalog, and returns grounded recommendations.

2. **Runtime and service boundary**
   - The online service is a **FastAPI** app in `app/main.py`.
   - Exposed endpoints are minimal: a lightweight **health check** and a **chat** endpoint.
   - The app is designed to run in a container via `Dockerfile` and `Procfile`, with deployment settings for platforms like Fly.io.

3. **Conversation orchestration**
   - The core control plane is an **explicit state machine** implemented across `app/state.py` and prompt/model definitions.
   - The readme names five branches: **CLARIFY**, **RETRIEVE_AND_RECOMMEND**, **REFINE**, **COMPARE**, and **REFUSE**.
   - This means the LLM is not free-form end-to-end; it is steered into one of a small number of action paths.

4. **Request-to-response flow**
   - A `/chat` request carries the full message list.
   - The app interprets the latest user intent, decides the action branch, optionally retrieves catalog candidates, then generates a response and structured recommendation list.
   - The response includes both natural-language text and machine-readable recommendation objects, plus an `end_of_conversation` flag.

5. **Retrieval subsystem**
   - `app/retrieval.py` is the main matching layer.
   - It uses **hybrid retrieval**: TF-IDF lexical search plus sentence-transformer semantic embeddings, combined with a weighted fusion.
   - The repository includes prebuilt indices in `data/tfidf.pkl` and `data/embeddings.npy`, so retrieval runs locally without needing external search infrastructure.

6. **Catalog and grounding**
   - `app/catalog.py` and `data/catalog.json` represent the SHL product catalog as the authoritative reference set.
   - The system enforces grounding by validating URLs and restricting final selection to retrieved catalog candidates.
   - The LLM is therefore a selector and summarizer, not the source of truth for product identity.

7. **LLM integration and failover**
   - `app/llm.py` handles model invocation and provider fallback.
   - The readme describes a **dynamic query-time failover chain**: Groq first, then Gemini Flash, then OpenAI GPT-4o-mini on rate-limit or availability failure.
   - This creates a latency/availability boundary between the app and external model providers.

8. **Prompting and response shape**
   - `app/prompts.py` contains **separate prompts per action**, matching the state-machine branches.
   - `app/models.py` likely defines request/response schemas and typed internal objects used by FastAPI and the orchestration layer.
   - This separation keeps intent routing, retrieval, and answer generation decoupled.

9. **Security and validation boundary**
   - `app/security.py` is the likely place for URL, catalog, or output validation rules.
   - The architecture emphasizes **programmatic grounding checks** after generation, rather than trusting the model output directly.
   - That boundary is important: external LLM output is filtered before being returned to the client.

10. **Offline build pipeline**
   - `scripts/scrape.py` populates the catalog from SHL sources into the local dataset.
   - `scripts/build_embeddings.py` builds the semantic and lexical artifacts used at runtime.
   - This means the service is mostly read-only at runtime; expensive preparation happens offline.

11. **Packaging and startup behavior**
   - The container is optimized to avoid startup downloads by baking model caches into the image.
   - `README.md` notes `HF_HOME=/app/.cache`, and startup eagerly loads embedding resources so the first request does not pay model-loading latency.
   - This affects the deployment architecture more than the application code does.

12. **Testing and evaluation**
   - `tests/test_api.py` covers endpoint behavior, `tests/test_behavior.py` probes decision behavior, and `tests/test_traces.py` replays traces with Recall@10.
   - The presence of trace replay and recall measurement shows the system is evaluated as a retrieval-and-recommendation pipeline, not just a chat app.

13. **Data and sample assets**
   - `sample_conversations/` contains example dialogue traces used to validate or demonstrate the state machine and retrieval behavior.
   - `data/` contains the runtime artifacts that the service depends on directly, making the repo self-contained after preprocessing.

14. **High-level architecture summary**
   - The overall structure is: **FastAPI request layer → state-machine intent routing → hybrid retrieval over local catalog/indexes → LLM generation with provider failover → grounding/validation → structured chat response**.
   - The key architectural constraint is that all recommendation decisions remain tied to the local catalog and retrieval results, with external LLMs used only as controlled reasoning/generation components.

Main entry points: [`app/state.py`](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/state.py) (State machine), [`app/retrieval.py`](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/retrieval.py) (Hybrid retrieval), [`app/main.py`](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/main.py) (FastAPI app), [`app/catalog.py`](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/catalog.py) (Catalog layer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_app_main["FastAPI app<br/>api server<br/>[main.py]"]
  node_app_state{{"State machine<br/>dialog control<br/>[state.py]"}}
  node_app_prompts["Branch prompts<br/>prompt set<br/>[prompts.py]"]
  node_app_models["Schemas<br/>typed models<br/>[models.py]"]
  node_app_retrieval["Hybrid retrieval<br/>search fusion<br/>[retrieval.py]"]
  node_app_catalog[("Catalog layer<br/>grounding data<br/>[catalog.py]")]
  node_app_llm["LLM gateway<br/>provider failover<br/>[llm.py]"]
  node_app_security{{"Validation<br/>grounding checks<br/>[security.py]"}}
end

subgraph group_data["Data"]
  node_catalog_json[("Catalog JSON<br/>product source<br/>[catalog.json]")]
  node_tfidf_idx[("TF-IDF index<br/>lexical artifact<br/>[tfidf.pkl]")]
  node_embed_idx[("Embeddings<br/>semantic artifact<br/>[embeddings.npy]")]
end

subgraph group_offline["Offline prep"]
  node_scrape_script["Scraper<br/>offline ingestion<br/>[scrape.py]"]
  node_build_script["Embedding build<br/>offline indexing"]
end

subgraph group_qa["Evaluation"]
  node_api_tests["API tests<br/>endpoint checks<br/>[test_api.py]"]
  node_behavior_tests["Behavior tests<br/>decision checks<br/>[test_behavior.py]"]
  node_trace_tests["Trace replay<br/>retrieval eval<br/>[test_traces.py]"]
end

subgraph group_examples["Examples"]
  node_sample_convos["Sample traces<br/>conversation examples"]
end

node_app_main -->|"uses schemas"| node_app_models
node_app_main -->|"routes chat"| node_app_state
node_app_state -->|"selects prompt"| node_app_prompts
node_app_state -->|"needs candidates"| node_app_retrieval
node_app_state -->|"generates reply"| node_app_llm
node_app_retrieval -->|"reads catalog"| node_app_catalog
node_app_catalog -->|"loads data"| node_catalog_json
node_app_retrieval -->|"lexical search"| node_tfidf_idx
node_app_retrieval -->|"semantic search"| node_embed_idx
node_app_llm -->|"validated by"| node_app_security
node_app_security -->|"checks grounding"| node_app_catalog
node_app_main -->|"guards response"| node_app_security
node_scrape_script -->|"builds catalog"| node_catalog_json
node_build_script -->|"builds index"| node_tfidf_idx
node_build_script -->|"builds vectors"| node_embed_idx
node_sample_convos -->|"feeds examples"| node_behavior_tests
node_sample_convos -->|"replays traces"| node_trace_tests
node_api_tests -->|"tests endpoints"| node_app_main
node_behavior_tests -->|"tests routing"| node_app_state
node_trace_tests -->|"measures recall"| node_app_retrieval

click node_app_main "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/main.py"
click node_app_state "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/state.py"
click node_app_prompts "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/prompts.py"
click node_app_models "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/models.py"
click node_app_retrieval "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/retrieval.py"
click node_app_catalog "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/catalog.py"
click node_app_llm "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/llm.py"
click node_app_security "https://github.com/rishabhjain2003/shl-copilot/blob/main/app/security.py"
click node_catalog_json "https://github.com/rishabhjain2003/shl-copilot/blob/main/data/catalog.json"
click node_tfidf_idx "https://github.com/rishabhjain2003/shl-copilot/blob/main/data/tfidf.pkl"
click node_embed_idx "https://github.com/rishabhjain2003/shl-copilot/blob/main/data/embeddings.npy"
click node_scrape_script "https://github.com/rishabhjain2003/shl-copilot/blob/main/scripts/scrape.py"
click node_build_script "https://github.com/rishabhjain2003/shl-copilot/blob/main/scripts/build_embeddings.py"
click node_api_tests "https://github.com/rishabhjain2003/shl-copilot/blob/main/tests/test_api.py"
click node_behavior_tests "https://github.com/rishabhjain2003/shl-copilot/blob/main/tests/test_behavior.py"
click node_trace_tests "https://github.com/rishabhjain2003/shl-copilot/blob/main/tests/test_traces.py"
click node_sample_convos "https://github.com/rishabhjain2003/shl-copilot/tree/main/sample_conversations/GenAI_SampleConversations"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_main,node_app_state,node_app_prompts,node_app_models,node_app_retrieval,node_app_catalog,node_app_llm,node_app_security toneBlue
class node_catalog_json,node_tfidf_idx,node_embed_idx toneAmber
class node_scrape_script,node_build_script toneMint
class node_api_tests,node_behavior_tests,node_trace_tests toneRose
class node_sample_convos toneIndigo
```

## Components

### Runtime

Online API, orchestration, retrieval, and validation

- [FastAPI app](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/main.py): api server · Entry point with chat and health endpoints · `app/main.py`
- [State machine](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/state.py): dialog control · Routes each chat turn into one of the action branches · `app/state.py`
- [Branch prompts](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/prompts.py): prompt set · Prompt templates for CLARIFY, RETRIEVE_AND_RECOMMEND, REFINE, COMPARE, and REFUSE · `app/prompts.py`
- [Schemas](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/models.py): typed models · Request, response, and internal data structures · `app/models.py`
- [Hybrid retrieval](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/retrieval.py): search fusion · Combines lexical and semantic candidate ranking · `app/retrieval.py`
- [Catalog layer](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/catalog.py): grounding data · Loads and serves the authoritative product catalog · `app/catalog.py`
- [LLM gateway](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/llm.py): provider failover · Invokes models with Groq to Gemini to OpenAI fallback · `app/llm.py`
- [Validation](https://github.com/rishabhjain2003/shl-copilot/blob/main/app/security.py): grounding checks · Validates URLs and filters model output against catalog rules · `app/security.py`

### Data

Local catalog and prebuilt retrieval artifacts

- [Catalog JSON](https://github.com/rishabhjain2003/shl-copilot/blob/main/data/catalog.json): product source · Local authoritative assessment catalog · `data/catalog.json`
- [TF-IDF index](https://github.com/rishabhjain2003/shl-copilot/blob/main/data/tfidf.pkl): lexical artifact · Prebuilt lexical retrieval index · `data/tfidf.pkl`
- [Embeddings](https://github.com/rishabhjain2003/shl-copilot/blob/main/data/embeddings.npy): semantic artifact · Prebuilt sentence-transformer vectors · `data/embeddings.npy`

### Offline prep

Scraping and embedding build pipeline

- [Scraper](https://github.com/rishabhjain2003/shl-copilot/blob/main/scripts/scrape.py): offline ingestion · Builds the local catalog from SHL sources · `scripts/scrape.py`
- [Embedding build](https://github.com/rishabhjain2003/shl-copilot/blob/main/scripts/build_embeddings.py): offline indexing · Generates retrieval artifacts for runtime · `scripts/build_embeddings.py`

### Evaluation

Behavioral and trace-based checks

- [API tests](https://github.com/rishabhjain2003/shl-copilot/blob/main/tests/test_api.py): endpoint checks · Exercises service behavior and responses · `tests/test_api.py`
- [Behavior tests](https://github.com/rishabhjain2003/shl-copilot/blob/main/tests/test_behavior.py): decision checks · Probes routing and recommendation behavior · `tests/test_behavior.py`
- [Trace replay](https://github.com/rishabhjain2003/shl-copilot/blob/main/tests/test_traces.py): retrieval eval · Replays conversations and measures Recall@10 · `tests/test_traces.py`

### Examples

Sample conversation traces

- [Sample traces](https://github.com/rishabhjain2003/shl-copilot/tree/main/sample_conversations/GenAI_SampleConversations): conversation examples · Example dialogues used to inspect the system · `sample_conversations/GenAI_SampleConversations`

## Connections

- FastAPI app → Schemas: uses schemas · FastAPI request and response typing
- FastAPI app → State machine: routes chat · Turns incoming messages into a controlled action branch
- State machine → Branch prompts: selects prompt · Chooses the branch-specific prompt template
- State machine → Hybrid retrieval: needs candidates · Retrieves catalog matches when the branch requires grounding
- State machine → LLM gateway: generates reply · Produces the final natural-language response
- Hybrid retrieval → Catalog layer: reads catalog · Retrieval operates over the local product set
- Catalog layer → Catalog JSON: loads data · Authoritative catalog source
- Hybrid retrieval → TF-IDF index: lexical search · TF-IDF candidate scoring
- Hybrid retrieval → Embeddings: semantic search · Embedding-based similarity search
- LLM gateway → Validation: validated by · Model output is checked before returning
- Validation → Catalog layer: checks grounding · Constrains recommendations to catalog-backed items
- FastAPI app → Validation: guards response · Final API response passes validation and grounding checks
- Scraper → Catalog JSON: builds catalog · Populates the local dataset from external sources
- Embedding build → TF-IDF index: builds index · Creates lexical retrieval artifacts
- Embedding build → Embeddings: builds vectors · Creates semantic retrieval artifacts
- Sample traces → Behavior tests: feeds examples · Example dialogue traces support behavioral inspection
- Sample traces → Trace replay: replays traces · Used for evaluation against known conversations
- API tests → FastAPI app: tests endpoints · Verifies service contract
- Behavior tests → State machine: tests routing · Checks branch selection and state transitions
- Trace replay → Hybrid retrieval: measures recall · Evaluates retrieval quality over recorded traces
