How microsoft/nlweb works
Project Type and Purpose
• NLWeb is a full-stack, Python-based conversational interface server for websites.
• It implements a simple REST/MCP protocol, connectors to multiple LLMs and vector stores, data-loading tools, and a static-file web UI.
• Its purpose is to let websites expose content (Schema.org JSON, RSS, etc.) through a natural-language “ask” endpoint.
File Structure
• Top-level:
• code/ – Python modules for core logic, embedding, LLMs, retrieval, pre-retrieval, tools, config, webserver
• static/ & webserver/ – front-end assets and the HTTP server
• data/ & demo/ – sample data and scripts
• docs/ – protocol, control-flow, prompts, deployment guides
• Architectural patterns:
• Pipeline of request handling: pre_retrieval → retrieval → embedding/LLM → post_ranking
• Plugin-style providers: code/embedding, code/llm, code/retrieval subdirectories
• Config-driven behavior via YAML and config.py
• Simple webserver hosting static UI + REST/MCP endpoints
README Insights
• Two main components: (1) protocol + JSON/Schema layer, (2) “proof-of-concept” implementation.
• It uses MCP’s “ask” method exposed over HTTP.
• Connectors for Qdrant, Snowflake, Milvus, Azure Search, and LLMs (OpenAI, Anthropic, Gemini, etc.).
Main Components
a. Client Layer
• Browser UI (static HTML/JS assets)
• External agents (MCP-aware bots)
b. API Layer
• Python WebServer (WebServer.py, StreamingWrapper) exposing REST/MCP endpoints
c. Core Service Pipeline
• Pre-retrieval (analyze_query, relevance_detection, memory, etc.)
• Retrieval layer (retriever.py, azure_search_client, qdrant, milvus…)
• Embedding services (openai_embedding, gemini_embedding, anthropic_embedding…)
• LLM providers (openai.py, anthropic.py, azure_oai.py, gemini.py…)
• Answer generation & post-ranking (generate_answer.py, post_ranking.py)
• State management (state.py, baseHandler.py, mcp_handler.py)
d. Data Stores & External Services
• Vector DBs: Qdrant, Snowflake, Milvus, Azure Cognitive Search
• LLM APIs: OpenAI, Anthropic, Gemini, Azure OAI, Inception
• Schema.org/RSS content in JSONL or live DB
e. Tools & CLI
• Data loaders (db_load, qdrant_load, rss2schema, etc.)
• Shell scripts and setup scripts for local/CI deployment
f. Configuration & Utilities
• YAML configs (config_embedding, config_llm, config_retrieval, etc.)
• Logger and logging_config_helper
• Environment management (.env.template)
Relationships and Interactions
• Browser/UI → HTTP POST “/ask” → WebServer
• WebServer → Core Handler → Pre-retrieval → Retrieval → Embedding → LLM → generate_answer
• Core → Vector DBs & LLM APIs (bidirectional over HTTP/gRPC)
• Core → Post-ranking → Response shaped as Schema.org JSON → WebServer → UI/agent
• Tools interact offline to load data into vector stores
Architectural Patterns & Principles
• Pipeline pattern for request processing
• Plugin architecture for easy addition of LLMs, embeddings, retrieval engines
• Config-driven behavior for extensibility
• Separation of concerns: UI vs. API vs. core logic vs. data connectors
Key Technologies & Frameworks
• Python 3.x
• YAML for configs
• Vector stores: Qdrant, Snowflake, Milvus, Azure Cognitive Search
• LLM APIs: OpenAI, Anthropic, Google Gemini, Azure OAI, etc.
• Static-file web UI with JS for streaming chat
• Shell scripts for setup and CLI