How kishoraditya/wexa_demo works
1. This is a FastAPI-based RAG backend for AWS Well-Architected Framework Q&A, with a small Streamlit frontend and separate evaluation/fine-tuning assets. The repo is organized around two main flows: offline ingestion of PDFs into a local vector index, and online question answering over that index.
2. The backend entrypoint is backend/main.py, which wires app startup/lifespan, middleware, and route registration. Configuration is centralized in backend/core/config.py and driven by config.yaml plus .env values, so most tunable behavior is externalized rather than hardcoded.
3. The online API surface is split into three route groups in backend/routes/: generate.py for /generate, ingest.py for /ingest and /ingest/upload, and health.py for liveness/readiness endpoints. This separation reflects the system boundary between request-time inference and document/index lifecycle management.
4. The ingestion subsystem lives in backend/services/ingestion.py. It loads the source PDFs, chunks text, normalizes metadata such as pillar/page/source, and deduplicates chunks before indexing. The README indicates this is an offline or manual-trigger pipeline, not an always-on crawler.
5. The retrieval layer is implemented in backend/services/vector_store.py. It uses local embeddings plus a FAISS index persisted on disk, with a metadata sidecar for source attribution. The architecture relies on local, file-backed storage rather than an external managed vector database, which keeps deployment self-contained but limits horizontal sharing of the index.
6. The generation pipeline is orchestrated in backend/services/rag_pipeline.py. It combines retrieval, reranking, prompt assembly, model routing, and post-generation checks into one request path. The prompt templates themselves are versioned in backend/services/prompts.py, which helps keep answer style and refusal behavior consistent.
7. Model selection is handled by backend/services/llm_manager.py. The primary path is a fine-tuned Phi-3-mini adapter, with transparent fallback to OpenAI GPT-4o-mini when the local model or adapter is unavailable. That router is a key architectural boundary because it turns model availability into a runtime concern rather than a deployment blocker.
8. Safety and answer quality are enforced in backend/services/guardrails.py. The system applies input checks before retrieval/generation and output grounding/PII-style checks after generation, with refusal behavior when retrieval confidence is too low or the question is out of scope. This is central to the repo’s “no hallucination” contract.
9. Cross-cutting runtime concerns are in backend/core/: cache.py provides a two-level cache for embeddings and responses, logging.py sets up structured loguru logging, metrics.py exposes Prometheus counters/histograms, and dependencies.py provides FastAPI dependency wiring. These modules make observability and performance part of the core architecture rather than an afterthought.
10. The data model boundary is in backend/models/schemas.py, which defines the request/response payloads for generation, ingestion, and health endpoints. This is where validation constraints such as query length, top_k, and pillar filtering are enforced at the API edge.
11. The frontend in frontend/app.py is a thin Streamlit client over the backend API. It is not the system of record; its role is to provide a user-facing chat/query surface and render sources, confidence, and latency returned by the API.
12. Testing is concentrated under backend/tests/ and is organized by subsystem: ingestion, vector store, RAG pipeline, observability, and API integration. That test layout mirrors the runtime boundaries and suggests the architecture was designed to be validated end-to-end as well as at the service level.
13. The eval/ directory is a separate quality-assurance lane, not part of the serving path. It contains a gold dataset, a RAGAS evaluation runner, and latency benchmarking scripts, which means the repository distinguishes production request handling from offline scoring and regression measurement.
14. The notebooks/fine_tuning_phi3_qlora.ipynb notebook documents the adapter-training workflow used to produce the local generation model. It is operationally separate from serving: the app can run with the adapter if available, but the fine-tuning process itself is outside the backend runtime.
15. Deployment is currently “local first” in code, but Dockerfile and docs/deployment_plan.md describe a production split between stateless API routing and heavier inference workers, with shared cache/storage services in the target cloud design. That means the repo contains both the current monolithic implementation and a forward-looking production topology.
Main entry points: rag_pipeline.py RAG pipeline · main.py App entry · generate.py Generate API · ingest.py Ingest API
Components 20
Backend
FastAPI app, services, and API contracts
- App entry · Wires lifespan, middleware, and route registration.
backend/main.py - Config · Loads YAML and env-driven runtime configuration.
backend/core/config.py - Dependencies · FastAPI dependency injection and shared service wiring.
backend/core/dependencies.py - Cache · Caches embeddings and responses.
backend/core/cache.py - Logging · Structured application logging setup.
backend/core/logging.py - Metrics · Counters and histograms for runtime observability.
backend/core/metrics.py - Schemas · Request and response validation for API endpoints.
backend/models/schemas.py - Generate API · Question answering endpoint group.
backend/routes/generate.py - Ingest API · Document ingestion and upload endpoints.
backend/routes/ingest.py - Health API · Liveness and readiness endpoints.
backend/routes/health.py - Ingestion · Loads PDFs, chunks text, normalizes metadata, and deduplicates.
backend/services/ingestion.py - Vector store · Local embeddings, FAISS retrieval, and persisted metadata.
backend/services/vector_store.py - RAG pipeline · Retrieval, reranking, prompt assembly, generation, and checks.
backend/services/rag_pipeline.py - Prompts · Versioned prompt templates and response style rules.
backend/services/prompts.py - LLM router · Selects local Phi-3 adapter or OpenAI fallback.
backend/services/llm_manager.py - Guardrails · Input, grounding, refusal, and output quality checks.
backend/services/guardrails.py
Client
Thin UI and user-facing entrypoint
- Streamlit app · Thin chat/query interface over the backend API.
frontend/app.py
Offline workflows
Evaluation and fine-tuning assets outside serving path
- Eval suite · Gold data, RAGAS scoring, and latency benchmarks.
eval/run_ragas.py - Fine-tuning · Phi-3 QLoRA training workflow for the local adapter.
notebooks/fine_tuning_phi3_qlora.ipynb
Docs
Architecture and deployment references
- Architecture docs · System diagrams, deployment plan, and tradeoff discussion.
docs/architecture.mmd
Connections 25
- App entry → Config: loads · Startup reads centralized configuration.
- App entry → Dependencies: wires · App startup registers shared dependencies.
- App entry → Generate API: mounts · Registers generation routes.
- App entry → Ingest API: mounts · Registers ingestion routes.
- App entry → Health API: mounts · Registers health routes.
- Generate API → Schemas: validates · Uses schemas for request and response payloads.
- Ingest API → Schemas: validates · Uses schemas for ingestion payloads.
- Generate API → RAG pipeline: calls · Generation requests enter the RAG pipeline.
- Ingest API → Ingestion: calls · Ingestion endpoints trigger document processing.
- Health API → Metrics: reports · Health endpoints expose runtime status.
- Ingestion → Vector store: indexes · Processed chunks are stored in the local vector index.
- RAG pipeline → Vector store: retrieves · RAG fetches relevant context from FAISS.
- RAG pipeline → Prompts: assembles · Prompt templates shape answer construction.
- RAG pipeline → LLM router: generates · Model router produces the answer.
- RAG pipeline → Guardrails: checks · Applies pre- and post-generation safety checks.
- RAG pipeline → Cache: uses · Caches embeddings and responses.
- Vector store → Cache: shares · Vector operations benefit from shared caching.
- LLM router → Config: reads · Model selection and fallbacks are config-driven.
- Guardrails → Config: reads · Safety thresholds and policies come from config.
- Streamlit app → Generate API: queries · UI sends user questions to the backend.
- Streamlit app → Ingest API: uploads · UI can trigger ingestion/upload flows.
- Streamlit app → Health API: checks · UI or operator can probe service health.
- Eval suite → RAG pipeline: benchmarks · Offline evaluation measures retrieval and answer quality.
- Fine-tuning → LLM router: produces · Training workflow yields the local adapter used by the model router.
- Architecture docs → App entry: describes · Docs capture the current service architecture.