How bhanuprasanna2001/kanyarasi works
1. Top-level shape
- The repo is a two-surface product built around one backend retrieval engine:
- apps/web: Next.js 16 frontend for the “Probe” chat and “Live” meeting experience.
- apps/api: FastAPI backend that handles auth, ingestion, retrieval, chat orchestration, and live verification.
- supabase/: Postgres schema/migrations, including pgvector and tables for users, sources, blocks, sessions, and traces.
2. Frontend boundary
- apps/web is a thin product shell plus feature UI.
- The main routes are organized by user workflow: dashboard, sources, indexing, chat, sessions, live, and settings.
- Most business logic lives in apps/web/lib and apps/web/features, which consume API endpoints rather than talking directly to the database or connectors.
3. Backend API boundary
- apps/api/app/main.py wires together FastAPI routers, dependencies, and observability.
- routers/ exposes product-facing HTTP endpoints for auth, chat, search, connections, ingestion, sessions, and provider-specific connector flows.
- services/ contains the core application logic; repositories/ is the persistence layer over Supabase/Postgres.
4. Data ingestion and connector subsystem
- Source ingestion is connector-driven: Google Drive, Slack, and GitHub are first-class providers.
- app/connectors/ and app/services/*_oauth.py handle provider-specific OAuth, API access, and document discovery.
- Ingestion is asynchronous: the API records jobs and a separate worker (app/jobs/ingestion_worker.py) processes them, turning external content into indexed source items and blocks.
5. Document modeling and storage
- The database stores normalized source metadata, extracted documents, chunk/block records, and ingestion run state.
- Migrations show a focus on stable block identity, chunk headers/metadata, idempotent session turns, live sessions, and policy/tracing tables.
- This means the system is designed to re-ingest and re-chunk content without duplicating semantic records.
6. Retrieval brain
- Retrieval is a multi-stage pipeline in app/services/retrieval/ and related services.
- The README indicates the key strategy: hybrid retrieval using pgvector HNSW plus BM25, then RRF fusion, then reranking.
- Supporting pieces include query parsing/classification/expansion, contextual prefixes, evidence assembly, and citation building.
7. Chat / Probe flow
- The chat surface routes through chat_pipeline, answer_generation, contextualizer, evidence, and citation_builder.
- User input is parsed, expanded, and classified, then retrieval produces candidate chunks, which are synthesized into a grounded answer with citations.
- Conversations and sessions are persisted in the API database layer so the web app can list history and reopen prior chats.
8. Live meeting fact-checking flow
- The live product is a separate backend feature area under app/features/live.
- The pipeline is explicitly staged: segment → extract claims → policy gate → retrieve → judge, with streamed updates over SSE.
- Supporting modules handle prompts, extraction, policy rules, a tracer, claim timelines, and verdict generation.
9. Deepgram and streaming
- features/deepgram/ and the web live hooks/components indicate real-time audio/transcription integration.
- The frontend live experience uses streaming hooks and UI components for microphone control, transcripts, claim cards, verdict bars, and trace drawers.
- SSE is the key transport boundary for incremental live feedback from backend to browser.
10. Auth, identity, and security
- Authentication is centralized in the backend with Google login for the user account and separate OAuth flows for Drive, Slack, and GitHub connections.
- session_tokens, crypto, and repository modules suggest encrypted session/auth artifacts and persisted connection state.
- Supabase RLS migrations indicate the database is protected by row-level security rather than trusting the app alone.
11. External service dependencies
- The architecture materially depends on:
- Supabase Postgres + pgvector for storage and vector search.
- OpenAI for embeddings and generation (gpt-5-mini, text-embedding-3-small).
- Cohere reranking as part of retrieval.
- Deepgram for live transcription.
- Google / Slack / GitHub as content sources and auth providers.
12. Evaluation and quality loop
- app/eval/ is a separate evaluation harness for retrieval and answer quality.
- It includes corpus loading, golden sets, judge logic, metrics, retriever variants, and a CLI.
- This is important architecturally because it gives the retrieval brain an offline test loop independent of the live product.
13. Deployment topology
- The intended deployment is split:
- Web on Vercel.
- API + worker on Railway as separate services.
- Database via local or hosted Supabase.
- There is also an ngrok-based tunnel path for Slack/GitHub webhook development.
14. How to draw the architecture
- Draw three major zones: browser UI, API/worker backend, and Supabase database.
- Show the API as the central orchestrator with two distinct runtime paths:
- synchronous chat/search requests
- asynchronous ingestion/live streaming workflows
- Add provider edges for Drive/Slack/GitHub/Google/OpenAI/Deepgram/Cohere to reflect the real external dependencies and the retrieval pipeline’s inputs and outputs.
Main entry points: service.py Retrieval Brain · main.py API App · client.ts API Client · layout.tsx App Shell
Components 24
Browser UI
Next.js product shell and workflow screens.
- App Shell · Top-level app shell, navigation, and shared UI chrome.
apps/web/app/(app)/layout.tsx - Chat UI · Probe chat routes and chat experience components.
apps/web/app/(app)/chat/page.tsx - Live UI · Meeting fact-checking screens, transcript, claims, and verdict views.
apps/web/app/(app)/live/page.tsx - Sources UI · Source browsing, provider selection, and indexing status screens.
apps/web/app/(app)/sources/page.tsx - Sessions UI · Conversation and live session history routes.
apps/web/app/(app)/sessions/page.tsx - API Client · Typed API wrappers, hooks, and endpoint definitions used by the UI.
apps/web/lib/api/client.ts
API Backend
FastAPI app, services, jobs, and feature pipelines.
- API App · Main application wiring for routers, dependencies, and observability.
apps/api/app/main.py - Auth & Identity · Google login, session tokens, crypto, and user management.
apps/api/app/routers/auth.py - Chat Pipeline · Chat request handling, parsing, retrieval, evidence, and answer generation.
apps/api/app/services/chat_pipeline.py - Retrieval Brain · Hybrid retrieval, ranking, query expansion, reranking, and citation support.
apps/api/app/services/retrieval/service.py - Ingestion Worker · Job processing that turns connector content into indexed source blocks.
apps/api/app/jobs/ingestion_worker.py - Connectors · Provider-specific discovery and OAuth for Drive, Slack, and GitHub.
apps/api/app/connectors/registry.py - Live Facts · Segment, claim extraction, policy gating, retrieval, judging, and SSE streaming.
apps/api/app/features/live/service.py - Eval Harness · Retrieval and answer quality evaluation runner, metrics, and judges.
apps/api/app/eval/cli.py
Supabase
Postgres schema, vector search, and app data.
- Users · Authenticated users and connection records.
supabase/migrations/20260518204424_create app users.sql - Sources & Blocks · Normalized source metadata, source documents, blocks, chunk metadata, and ingestion state.
supabase/migrations/20260519211956_source documents and blocks.sql - Sessions · Chat sessions, turns, live sessions, claim timelines, and trace records.
supabase/migrations/20260520012249_live sessions.sql - Vector Search · HNSW tuning and indexed retrieval support.
supabase/migrations/20260520080339_context prefix and hnsw tune.sql
External Services
Identity, content sources, retrieval, and transcription providers.
- Google · Google login and Google Drive content source.
- Slack · Slack OAuth and message/document discovery.
- GitHub · GitHub OAuth and repository/discussion/document discovery.
- OpenAI · Embeddings and answer generation models used by retrieval and chat.
- Cohere · Reranking stage in the retrieval pipeline.
- Deepgram · Real-time audio transcription for live sessions.
Connections 31
- App Shell → Chat UI: routes to · Shell hosts the major workflow surfaces.
- App Shell → Live UI: routes to · Shell hosts live meeting experience.
- App Shell → Sources UI: routes to · Shell hosts source management views.
- App Shell → Sessions UI: routes to · Shell hosts history and session views.
- Chat UI → API Client: fetches data via · Chat UI uses typed API hooks and client wrappers.
- Live UI → API Client: streams via · Live UI consumes SSE-backed live endpoints.
- Sources UI → API Client: fetches data via · Source screens use API hooks and selection state.
- Sessions UI → API Client: fetches data via · Session history and detail views load from API.
- API Client → API App: HTTP API · Frontend talks to the FastAPI backend.
- API App → Auth & Identity: includes · Main app wires auth and identity routes.
- API App → Chat Pipeline: includes · Main app exposes chat orchestration routes.
- API App → Ingestion Worker: includes · Main app exposes ingestion endpoints and background job entrypoints.
- API App → Live Facts: includes · Main app exposes live fact-checking routes and SSE.
- API App → Retrieval Brain: uses · Chat and live features share retrieval services.
- Chat Pipeline → Retrieval Brain: calls · Chat pipeline expands, retrieves, reranks, and assembles evidence.
- Chat Pipeline → Sessions: persists · Chat sessions and turns are stored in Postgres.
- Ingestion Worker → Connectors: pulls from · Worker reads connector jobs and discovers source content.
- Ingestion Worker → Sources & Blocks: writes · Ingestion creates and updates source documents and blocks.
- Retrieval Brain → Sources & Blocks: queries · Retrieval reads indexed blocks and source metadata.
- Retrieval Brain → Vector Search: searches · Hybrid vector and keyword retrieval uses pgvector-backed indexing.
- Live Facts → Sessions: persists · Live sessions, turns, claims, and traces are stored.
- Live Facts → Retrieval Brain: calls · Live pipeline retrieves evidence for claims.
- Live Facts → Deepgram: streams audio to · Audio transcription is sourced from Deepgram.
- Connectors → Google: auth/discover · Google Drive OAuth and discovery.
- Connectors → Slack: auth/discover · Slack OAuth and discovery.
- Connectors → GitHub: auth/discover · GitHub OAuth and discovery.
- Auth & Identity → Google: login via · User sign-in uses Google identity.
- Retrieval Brain → OpenAI: embeddings/gen · Embedding and answer generation rely on OpenAI.
- Retrieval Brain → Cohere: reranks with · Candidate passages are reranked by Cohere.
- Eval Harness → Retrieval Brain: benchmarks · Offline eval compares retrieval variants and quality.
- Eval Harness → OpenAI: judges/models · Evaluation can use model-backed judging and generation.