How pr0mila/groqstreamchain works
Project type and purpose
• This is a full-stack, real-time AI chat application (“GroqStreamChain”) built with FastAPI (backend), WebSockets (real-time communication), LangChain + Groq (LLM service), and a minimal HTML/CSS/JS frontend.
File structure
• Top-level:
• server.py – FastAPI app with HTTP & WebSocket routes
• config.py & system_prompts.py – environment/configuration and prompt templates
• models/chat.py – Pydantic models for request/response schemas
• services/llm_service.py – abstraction over the Groq & LangChain client
• static/ & templates/ – frontend assets (HTML, JS, CSS)
• test_groq.py – standalone script to validate Groq API connectivity
• No external database, so sessions are ephemeral/in‐memory within the running server.
Key insights from README
• Real-time streaming: WebSocket connection for bi-directional, low-latency chat.
• Chunked LLM responses: streamed back through the same socket.
• Clean separation: presentation (static/, templates/) vs. API layer (server.py) vs. LLM service (services/llm_service.py).
Main components
a. Frontend (Browser)
• index.html + main.js establishes a WebSocket to the backend
• Renders messages, shows typing indicators, streams incoming chunks
b. API Layer (FastAPI / server.py)
• HTTP endpoint to serve the HTML page
• WebSocket endpoint to receive user messages & push LLM replies
• Manages chat sessions (in-memory or by session ID)
c. Business Logic / Service Layer
• llm_service.py orchestrates calls to the Groq/LangChain client
• system_prompts.py provides prompt templates
• config.py supplies API key & model name from environment (.env)
d. External LLM Provider
• Groq API (via LangChain’s Groq integration)
• Streams token chunks back to llm_service
e. Test Harness (Optional)
• test_groq.py independently tests connectivity to Groq API
• Can be shown as a side-branch for validation, not on the critical path
Relationships and data flow
1. Browser loads index.html from FastAPI HTTP route
2. main.js opens WebSocket → FastAPI server
3. User message → WebSocket “message” event → server.py receives payload (chat model)
4. server.py calls llm_service.generate_response(…)
5. llm_service uses LangChain + Groq client → Groq API
6. Groq streams token chunks → llm_service forwards each chunk
7. server.py pushes each chunk through WebSocket → browser UI renders stream
8. End of stream closes the loop; client shows complete message
Architectural patterns & design principles
• Layered architecture:
• Presentation (static + templates)
• API/Controller (FastAPI endpoints)
• Service (LLM orchestration)
• Configuration (config & prompts)
• Single responsibility: each module has a focused job
• Streaming architecture: asynchronous, non-blocking WebSockets for chunked data
• Externalize configuration: .env for API keys & model names
Technologies & frameworks
• Frontend: HTML5, vanilla JS (WebSocket API), CSS
• Backend: FastAPI (async), Uvicorn (ASGI), Pydantic models
• AI Service: LangChain, Groq Python client
• Utilities: python-dotenv for config, Jinja2 for templating