How skhapijulhossen/gsearch works
Project type and purpose
• This is a full-stack Retrieval-Augmented Generation (RAG) chatbot for HR teams.
• Front-end: Streamlit application (app/streamlit_app.py)
• Back-end: FastAPI service (app/api/main.py)
• Core pipeline: data loading, FAISS vector store, LLM chain (via Ollama), hybrid search.
File structure
• app/api/main.py → FastAPI entrypoint & HTTP endpoints
• app/streamlit_app.py → Streamlit UI
• app/core/
• config.py → application settings (e.g., thresholds, model names)
• schemas.py → Pydantic models for requests/responses
• prompts.py → prompt templates for the LLM
• app/services/
• data_service.py → load/filter employee JSON (data/employees.json)
• retriever_service.py → wrap FAISS index (employee_faiss_index/index.faiss), hybrid search logic
• llm_service.py → assemble LangChain QA chain, call Ollama LLM, handle context & formatting
• data/ → raw employee JSON
• employee_faiss_index/ → persisted FAISS index
• requirements.txt → Python dependencies
Extract key architecture insights from the README
• Four logical stages: Data Loading → Vector Store → LLM Chain → API
• Hybrid search: similarity + keyword matching
• Open-source stack: FastAPI, FAISS, LangChain, Ollama, Streamlit
a. Main components
• Streamlit UI (front-end)
• FastAPI server (API gateway)
• Service layer (DataService, RetrieverService, LLMService)
• Data Store (employees.json)
• Vector Store (FAISS index)
• External LLM (Ollama)
b. Relationships & data flow
1. User enters query in Streamlit → HTTP POST to FastAPI
2. FastAPI controller validates via schemas → passes to DataService (optional filters)
3. RetrieverService queries FAISS index → retrieves top-k embeddings + metadata
4. LLMService builds prompt (using prompts.py), invokes Ollama via LangChain QA chain
5. Ollama returns structured response → LLMService formats + attaches citations
6. FastAPI returns JSON → Streamlit renders response
c. Architectural patterns & principles
• Layered architecture: UI → API → Services → Storage/External
• Separation of concerns: data loading, retrieval, LLM orchestration
• RAG pipeline pattern (index, retrieve, generate)
d. Key technologies & frameworks
• FastAPI (web framework)
• Streamlit (UI)
• FAISS (vector similarity search)
• LangChain (RAG orchestration)
• Ollama (local LLM host)
• Pydantic (schemas & validation)