How sylphai-inc/githubchat works
What this project is (type + purpose)
GithubChat is a RAG (Retrieval-Augmented Generation) assistant that lets a user “chat with” an arbitrary GitHub repository.
Architecturally it’s a hybrid full-stack project with two possible UIs:
• React + Vite frontend (frontend/) that talks to a FastAPI backend (api.py)
• A Streamlit UI (app.py) that likely talks to the same RAG core directly (or could call the FastAPI, depending on implementation)
The “core” backend logic lives in src/:
• src/data_pipeline.py: ingest repo → chunk/index → persist/reload (DatabaseManager)
• src/rag.py: orchestrates “prepare repo + retrieve + generate answer” lifecycle (RAG class)
External dependencies implied by README:
• OpenAI API (via OPENAI_API_KEY) for embeddings and/or generation
• GitHub as the upstream content source (repo URL)
How to read the file tree as architecture hints
Top-level:
• api.py → FastAPI HTTP entrypoint (serves /query)
• app.py → Streamlit entrypoint (interactive UI, separate runtime)
• config.py → config management (keys, model params, paths)
• src/ → backend domain logic (RAG + pipeline)
• frontend/ → React UI (Vite, Tailwind)
• tests/test_rag.py → confirms RAG is a first-class unit (core component)
Frontend details:
• frontend/src/components/GitHubChat.tsx → key UI component (calls /query)
• Vite config + Tailwind → typical SPA build pipeline
Backend details:
• src/data_pipeline.py (DatabaseManager) strongly suggests:
• some persistent store (vector DB, sqlite, local files, etc.)
• “prepare repo, persist from reloading” (README) → there’s a cache/index reuse path
• src/rag.py → the orchestrator that API/Streamlit call into
React Web App (Vite)
• frontend/src/App.tsx
• frontend/src/components/GitHubChat.tsx
• Talks to FastAPI: POST /query
Streamlit App
• app.py
• Calls RAG core (direct import) *or* calls FastAPI (show whichever is true in code; if unknown, show both paths but mark one as “likely”)
FastAPI Server
• api.py
• Endpoint: POST /query taking { repo_url, query } returning { rationale, answer, contexts }
RAG Orchestrator
• src/rag.py (RAG class)
• “Repo readiness check / load existing index”
• “Retrieve contexts”
• “Prompt + generate answer”
• “Return rationale + contexts + answer”
Data Pipeline / Index Manager
• src/data_pipeline.py (DatabaseManager)
• Responsibilities:
• Fetch/parse repo content
• Chunking / preprocessing
• Embedding computation (likely)
• Persist and reload index
Config
• config.py + .streamlit/secrets.toml (runtime secrets)
• As a cross-cutting dependency feeding API/Streamlit/RAG (model name, paths, API keys)
GitHub (External)
• Input: repo_url
• Data: repo files, README, code
Vector Store / Index Persistence (Local or DB)
• Not explicitly named in tree, but README says “persist from reloading”
• “Embeddings + chunks + metadata”
• “Repo fingerprint / commit hash” (if implemented)
• “local disk / sqlite / chroma / etc.” once confirmed
LLM / Embeddings Provider (OpenAI)
• External API called by RAG pipeline
• Two logical calls:
• embeddings (indexing + maybe query embedding)
• chat/completions (answer generation)
RAG → DatabaseManager
• If repo not indexed: “prepare_repo(repo_url)” → triggers fetch/chunk/embed/persist
• Else: “load_index(repo_url)” → reuse persisted store
Alternate flow (Streamlit)
• If Streamlit instead calls FastAPI, show Streamlit → FastAPI like the React path.
Architectural patterns/design principles to call out
• Separation of concerns
• UI (React/Streamlit) vs API (FastAPI) vs Core (src/) vs Storage/External
• RAG lifecycle split
• *Ingestion/indexing* (DatabaseManager) vs *Query-time retrieval+generation* (RAG)
• Caching/persistence
• Key feature: “persist from reloading” → important stateful component (vector store/index) even if the service is stateless
• Plug-in/external provider boundary
Labels
• Use exact names from repo where possible: RAG, DatabaseManager, POST /query, GitHubChat.tsx
• Responses: answer, rationale, contexts
• Clients (blue): React, Streamlit
• Services (green): FastAPI
• Core modules (teal): RAG, DatabaseManager
• External services (gray): GitHub, OpenAI
Checklist of project-specific details not to miss
• Two entrypoints: api.py and app.py (must both appear)
• /query contract (must label request/response fields)
• Persistence/caching concept (vector store/index reuse)
• OpenAI key management via .streamlit/secrets.toml + config.py
• That the “repo URL” drives a data ingestion pipeline prior to retrieval
• exact storage technology (Chroma/FAISS/sqlite/files)
• whether ingestion is triggered per-request or via an explicit “prepare” step
• whether repo cloning uses git, GitHub API, or raw HTTP fetches