nicolasarmientor/tf-rag

Purpose and entry points

TensorFlow-RAG is a compact retrieval-augmented Q&A system over TensorFlow Guide documentation. Its two operational entry points are the FastAPI server, src/api/main.py, and offline vector-store construction via python -m src.retrieval.vector_store in src/retrieval/vector_store.py. Evaluation runs separately through src/eval/evaluate.py.

Offline ingestion and indexing

The build pipeline loads Markdown and notebook source files from the externally supplied TensorFlow docs tree, extracts content, and creates source-attributed chunks. src/ingest/loader.py owns document loading; src/ingest/chunker.py performs token-aware chunking with overlap and preserves fenced code blocks intact. The resulting chunks and metadata are embedded and persisted by src/retrieval/vector_store.py.

Retrieval boundary

ChromaDB is both the persistent vector store and embedding provider, using its bundled ONNX Runtime all-MiniLM-L6-v2 model rather than a direct Torch/sentence-transformers runtime. Query retrieval is isolated in src/retrieval/retriever.py, which requests top-k candidates but applies a distance threshold. This is the key grounding boundary: irrelevant questions yield no usable context rather than unrelated nearest neighbors.

Generation and safety

src/generation/generator.py converts retrieved chunks into a constrained Claude prompt, calls Anthropic, and returns answers with source information. The generator is expected to refuse when retrieval produces no qualifying context. Because model output is rendered in the browser, generated HTML is sanitized before presentation. Anthropic Claude is also used as an external evaluation judge.

API and BYOK UI

src/api/main.py composes retrieval and generation behind two interfaces: JSON POST /ask, using server-side ANTHROPIC_API_KEY, and form-oriented POST /ask-ui, which renders an HTMX HTML fragment. Request/response contracts reside in src/api/schemas.py. The browser UI is server-rendered from src/api/templates/index.html and src/api/templates/answer.html; for the UI route, a visitor-provided API key is kept client-side/session-scoped rather than persisted by the application.

Evaluation loop

src/eval/testset.py defines a hand-authored set of answerable and negative questions. src/eval/evaluate.py exercises both stages: whether retrieval finds the expected source and whether a second Claude call judges the answer against a reference summary. This makes refusal behavior a first-class quality criterion, not merely prompt guidance.

Deployment and runtime

The Python application runs under FastAPI/Uvicorn. Dockerfile fetches TensorFlow documentation and builds chroma_data/ during image build, producing a runtime container with no source-data volume requirement. It preserves the Chroma/ONNX cache under /app, then runs non-root. The intended hosting target is Render; runtime port selection honors its PORT environment variable. CI is defined in .github/workflows/ci.yml.