How thelalit/nlweb works
Project type and purpose
• NLWeb is an open-source, full-stack Python service that implements a simple “natural language” REST/MCP API for websites.
• It provides:
• A backend pipeline (pre-retrieval → retrieval → embedding/LLM → ranking/generation)
• Connectors to vector stores (Qdrant, Snowflake, Milvus, Azure AI Search) and LLM providers (OpenAI, Azure OAI, Gemini, Anthropic, Inception)
• A lightweight webserver + static JavaScript UI for demos
• Data-loading tools for schema.org, RSS, JSONL ingestion into vector indexes
File structure
• Top-level directories:
• code/ – main Python application
– config/ – YAML and Python config loader
– core/ – request router, state machine, handlers, ranking, answer generation
– pre_retrieval/ – query analysis, memory, decontextualization, relevance detection
– retrieval/ – client abstractions for Qdrant, Milvus, Snowflake, Azure Search
– embedding/ – abstraction layer over embedding providers
– llm/ – abstraction over LLM providers
– prompts/ – prompt templates and runner
– webserver/ – HTTP/MCP server entrypoint and static handler
– tools/ – scripts for loading data into vector stores
– utils/ – logging, Snowflake helper, generic utilities
• static/ – JavaScript/HTML chat UI and widgets
• docs/, demo/, data/ – documentation, examples, sample schemas
• Key config files: .env.template, config_*.yaml
• Entry points: code/WebServer.py (server), code/chatbot_interface.py (CLI/test harness)
README insights
• Two main pieces: the protocol definition (REST/MCP) and a reference implementation
• Encourages community to reuse protocol and swap in production-grade UIs or data sources
• Emphasizes modular connectors for storage/LLMs and a clear “life of a chat query” flow
1. Client Layer
• Web UI (static JS/chat widgets)
• External agents/CLI (MCP clients)
2. Webserver / API Layer
• HTTP / MCP endpoint (WebServer.py, static_file_handler.py)
• Config loader & environment (.env)
3. Core Pipeline Controller
• Request router (baseHandler.py, mcp_handler.py)
• State management (state.py)
4. Pre-Retrieval Module
• Query analyzer, decontextualizer, memory (analyze_query.py, memory.py, etc.)
5. Retrieval Module
• Vector store clients (Qdrant, Milvus, Snowflake, Azure Search)
• Retriever orchestration
6. Embedding Module
• Embedding provider abstraction (OpenAI, Gemini, Azure OAI, Snowflake)
7. LLM Module
• LLM provider abstraction (OpenAI, Azure, Anthropic, Inception)
8. Post-Processing
• Ranking (post_ranking.py, ranking.py), answer generation (generate_answer.py)
9. Tools & Data Ingestion
• Scripts for data loading (RSS→schema.org JSONL → vector DB)
10. Observability & Config
• Logging, metrics, config/*.yaml, utils/logger.py
1. Client → Webserver (HTTP/MCP ask request)
2. Webserver → Core Router
3. Core Router → Pre-Retrieval → Retrieval → Embedding → LLM
4. LLM → Post-Processing → Core Router → Webserver
5. Webserver → Client (JSON + schema.org output)
6. Tools → Data Stores (vector DB / Snowflake) feeding Retrieval
c. Architectural patterns & design principles
• Pipeline pattern: clear, pluggable stages (pre-retrieval, retrieval, embed, LLM, ranking)
• Adapter/plugin architecture: connectors for each LLM and vector store live behind a common interface
• Configuration-driven: YAML + environment variables for swapping providers
• Single code base supports multiple deployment targets (laptop → cloud)
d. Key technologies/frameworks
• Python 3
• HTTP/MCP interface (custom server in WebServer.py)
• Vector stores: Qdrant, Milvus, Snowflake, Azure Search
• LLM APIs: OpenAI, Azure OAI, Gemini, Anthropic, Inception
• Static JS UI (vanilla JS + HTML)