How ntdat02092002/lol-chatbot works
Project Type and Purpose
• This is a full-stack, RAG-based chatbot application with two main concerns:
a. Automated data pipeline to crawl, embed, and store League of Legends patch notes.
b. Interactive Streamlit UI that uses a vector DB and LLM to answer user questions about the latest patch.
File Structure
• Top-level scripts and their roles:
• chatbot.py & main.py – entrypoint for the Streamlit UI and orchestrator of the chat interactions.
• pinecone_data_flow.py – defines a Prefect flow to crawl patch notes, embed text, and upsert into Pinecone.
• prefect_deployment.py – registers and deploys the Prefect flow to Prefect Cloud.
• prompt_template.py – holds reusable prompt formats for LangChain.
• utils.py & test.py – utility functions and unit tests.
• config.json – model/configuration settings for different LLMs/embeddings.
• Infrastructure as code / CI files in .github/workflows
• Requirements.txt, .python-version for environment reproducibility
Extract Insights from README
• Tech Stack: Streamlit for UI, LangChain + Google AI for LLM & embeddings, Pinecone for vector store, Prefect for workflow orchestration, BeautifulSoup for crawling, WandB for logging.
• Data Flow: daily crawl → embed → Pinecone index → RAG retrieval → Streamlit UI → LLM → user.
• External Dependencies: Google AI Studio (LLM & embeddings), Pinecone, Prefect Cloud, Weights & Biases.
Main Components & Their Interactions
a. User Interface (Streamlit)
• Accepts user queries, invokes the retrieval chain, displays LLM responses.
b. Retrieval & Generation Layer (LangChain)
• Self-query retriever against Pinecone, constructs prompts via prompt_template.py, calls Google AI LLM.
c. Vector Database (Pinecone)
• Stores embeddings of patch notes, supports similarity search.
d. Workflow Orchestrator (Prefect)
• Scheduled/Manual flows to crawl (BeautifulSoup), process, embed, and upsert into Pinecone.
e. External Services
• Google AI Studio API, Pinecone API, Prefect Cloud API, WandB API for logging interactions.
Architectural Patterns & Design Principles
• Separation of concerns: data ingestion pipeline decoupled from UI.
• RAG (Retrieval-Augmented Generation) pattern for context-aware Q&A.
• Modular design: prompt templates, utilities, and config abstracted out.
• Workflow scheduling via Prefect, enabling reproducibility and visibility.
Final Checklist
• All components distinctly labeled.
• Annotation of external dependencies and environment variables (e.g., PINECONE_API_KEY).
• Coverage of both the automated flow and the real-time chat flow.