How peiyun2260/scorerag works
Project Type and Purpose
• ScoreRAG is a full-stack news-generation application consisting of:
• A FastAPI backend pipeline that retrieves, scores, and summarizes news.
• A React + Vite frontend UI for query input and result visualization.
• Data stores (ChromaDB for embeddings, MongoDB for raw articles).
• Integration with LLM providers (OpenAI, Groq) and HuggingFace embeddings.
File-Structure Analysis
• Top-level: backend/, frontend/, config files (Makefile, pyproject.toml, requirements.txt).
• backend/app/ subdirectories:
• api/ (FastAPI routers)
• services/ (retrieval, scoring, summary, generation business logic)
• db/ (Chroma & Mongo connectors)
• llm_clients/ (OpenAI & Groq abstractions)
• schemas/ (Pydantic models for request/response)
• backend/scripts/: command-line ingestion and pipeline test utilities.
• backend/tests/: unit tests for API routes.
• frontend/: standard React + Vite structure (src/components, App.jsx, index.html, public/sample_output.json).
README Insights
• Purpose: build “retrieval-augmented generation” pipeline for news.
• Key tech: FastAPI, React/Vite, ChromaDB, MongoDB, OpenAI/Groq, HuggingFace embeddings.
• Flow:
1. Ingest news into ChromaDB & MongoDB.
2. API POST /api/query → retrieve top-K docs via embedding search.
3. Re-rank with multi-round GPT scoring.
4. Produce summary and full article with citations.
5. Frontend displays results or sample JSON in demo mode.
Components
a. Main Components
• Frontend (React/Vite SPA)
• Backend (FastAPI app)
• Data Stores: ChromaDB, MongoDB
• External LLM Services: OpenAI, Groq
• Embedding Service (HuggingFace model)
• CLI Scripts (ingest, test pipeline)
b. Relationships & Data Flow
• User interacts with Frontend → sends HTTP POST to FastAPI API Gateway.
• Router (news_router.py) invokes Service layer:
– EmbeddingService queries ChromaDB for top-K vectors.
– CoTService and ScoringService call OpenAI/Groq clients.
– SummaryService and GenerationService orchestrate LLM calls.
• DB connectors read/write from MongoDB (raw data) & ChromaDB (embeddings).
• Results flow back through API → Frontend renders summaries, references.
• CLI Scripts bypass UI for bulk ingestion or pipeline tests.
c. Architectural Patterns & Principles
• Layered architecture: API layer → Service (business logic) → Data/External integrations.
• Dependency inversion: LLM clients & DB connectors abstracted behind interfaces.
• Single Responsibility: each service class focuses on one pipeline stage.
• Config-driven: environment variables for keys and DB URIs (.env).
d. Key Technologies & Frameworks
• Backend: Python 3.9+, FastAPI, Pydantic, ChromaDB, PyMongo
• Frontend: React 18+, Vite, JSX
• LLMs: OpenAI GPT, Groq, HuggingFace embeddings
• Dev tools: Makefile, uv (uvicorn/virtualenv), pytest