# microsoft/nlweb architecture

> Architecture diagram of the GitHub repository microsoft/nlweb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/nlweb
- Repository: https://github.com/microsoft/nlweb

## Overview

##### Project Type and Purpose
- NLWeb is a full-stack, Python-based conversational interface server for websites.
- It implements a simple REST/MCP protocol, connectors to multiple LLMs and vector stores, data-loading tools, and a static-file web UI.
- Its purpose is to let websites expose content (Schema.org JSON, RSS, etc.) through a natural-language “ask” endpoint.

##### File Structure
- Top-level:
- code/ – Python modules for core logic, embedding, LLMs, retrieval, pre-retrieval, tools, config, webserver
- static/ & webserver/ – front-end assets and the HTTP server
- data/ & demo/ – sample data and scripts
- docs/ – protocol, control-flow, prompts, deployment guides
- Architectural patterns:
- Pipeline of request handling: pre_retrieval → retrieval → embedding/LLM → post_ranking
- Plugin-style providers: code/embedding, code/llm, code/retrieval subdirectories
- Config-driven behavior via YAML and config.py
- Simple webserver hosting static UI + REST/MCP endpoints

##### README Insights
- Two main components: (1) protocol + JSON/Schema layer, (2) “proof-of-concept” implementation.
- It uses MCP’s “ask” method exposed over HTTP.
- Connectors for Qdrant, Snowflake, Milvus, Azure Search, and LLMs (OpenAI, Anthropic, Gemini, etc.).

##### Main Components
a. Client Layer
- Browser UI (static HTML/JS assets)
- External agents (MCP-aware bots)
b. API Layer
- Python WebServer (WebServer.py, StreamingWrapper) exposing REST/MCP endpoints
c. Core Service Pipeline
- Pre-retrieval (analyze_query, relevance_detection, memory, etc.)
- Retrieval layer (retriever.py, azure_search_client, qdrant, milvus…)
- Embedding services (openai_embedding, gemini_embedding, anthropic_embedding…)
- LLM providers (openai.py, anthropic.py, azure_oai.py, gemini.py…)
- Answer generation & post-ranking (generate_answer.py, post_ranking.py)
- State management (state.py, baseHandler.py, mcp_handler.py)
d. Data Stores & External Services
- Vector DBs: Qdrant, Snowflake, Milvus, Azure Cognitive Search
- LLM APIs: OpenAI, Anthropic, Gemini, Azure OAI, Inception
- Schema.org/RSS content in JSONL or live DB
e. Tools & CLI
- Data loaders (db_load, qdrant_load, rss2schema, etc.)
- Shell scripts and setup scripts for local/CI deployment
f. Configuration & Utilities
- YAML configs (config_embedding, config_llm, config_retrieval, etc.)
- Logger and logging_config_helper
- Environment management (.env.template)

##### Relationships and Interactions
- Browser/UI → HTTP POST “/ask” → WebServer
- WebServer → Core Handler → Pre-retrieval → Retrieval → Embedding → LLM → generate_answer
- Core → Vector DBs & LLM APIs (bidirectional over HTTP/gRPC)
- Core → Post-ranking → Response shaped as Schema.org JSON → WebServer → UI/agent
- Tools interact offline to load data into vector stores

##### Architectural Patterns & Principles
- Pipeline pattern for request processing
- Plugin architecture for easy addition of LLMs, embeddings, retrieval engines
- Config-driven behavior for extensibility
- Separation of concerns: UI vs. API vs. core logic vs. data connectors

##### Key Technologies & Frameworks
- Python 3.x
- YAML for configs
- Vector stores: Qdrant, Snowflake, Milvus, Azure Cognitive Search
- LLM APIs: OpenAI, Anthropic, Google Gemini, Azure OAI, etc.
- Static-file web UI with JS for streaming chat
- Shell scripts for setup and CLI

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        ClientUI["Browser UI"]:::frontend
        ExtAgents["External MCP Agents"]:::external
    end

    %% API Layer
    subgraph "API Layer"
        direction TB
        WebServer["WebServer.py"]:::backend
        StreamWrap["StreamingWrapper.py"]:::backend
        StaticHandler["static_file_handler.py"]:::backend
    end

    %% Core Entry & State
    subgraph "Core Entry"
        direction TB
        BaseHandler["baseHandler.py"]:::backend
        MCPHandler["mcp_handler.py"]:::backend
        StateMgr["state.py"]:::backend
    end

    %% Pre-retrieval Stage
    subgraph "Pre-retrieval" 
        direction TB
        Analyze["analyze_query.py"]:::pipeline
        Relevance["relevance_detection.py"]:::pipeline
        Decontext["decontextualize.py"]:::pipeline
        Memory["memory.py"]:::pipeline
        ReqInfo["required_info.py"]:::pipeline
    end

    %% Retrieval Layer
    subgraph "Retrieval" 
        direction TB
        Retriever["retriever.py"]:::pipeline
        QdrantRet["qdrant_retrieve.py"]:::pipeline
        AzureSearch["azure_search_client.py"]:::pipeline
        Milvus["milvus_client.py"]:::pipeline
        SnowflakeRet["snowflake_client.py"]:::pipeline
    end

    %% Embedding Services
    subgraph "Embedding Plugins" 
        direction TB
        OpenAIEmb["openai_embedding.py"]:::plugin
        AnthropicEmb["anthropic_embedding.py"]:::plugin
        GeminiEmb["gemini_embedding.py"]:::plugin
        AzureEmb["azure_oai_embedding.py"]:::plugin
        SnowflakeEmb["snowflake_embedding.py"]:::plugin
    end

    %% LLM Providers
    subgraph "LLM Plugins"
        direction TB
        OpenAI["openai.py"]:::plugin
        Anthropic["anthropic.py"]:::plugin
        Gemini["gemini.py"]:::plugin
        AzureOAI["azure_oai.py"]:::plugin
        AzureDS["azure_deepseek.py"]:::plugin
        AzureLL["azure_llama.py"]:::plugin
        Inception["inception.py"]:::plugin
        SnowflakeLLM["snowflake.py"]:::plugin
        LLMProv["llm_provider.py"]:::plugin
        LLMBase["llm.py"]:::plugin
    end

    %% Answer Generation & Ranking
    subgraph "Answer Gen & Ranking"
        direction TB
        GenAns["generate_answer.py"]:::backend
        Ranking["ranking.py"]:::backend
        PostRank["post_ranking.py"]:::backend
    end

    %% Protocol Interface
    ChatIfc["chatbot_interface.py"]:::backend

    %% Config & Utils
    subgraph "Config & Utilities"
        direction TB
        ConfigAll["config/*.yaml & config.py"]:::config
        Logger["logger.py"]:::utils
        LogHelper["logging_config_helper.py"]:::utils
        SetLevel["set_log_level.py"]:::utils
    end

    %% Tools & CLI
    subgraph "Tools & CLI"
        direction TB
        DBLoad["db_load.py"]:::tools
        QdrantLoad["qdrant_load.py"]:::tools
        RSS2Schema["rss2schema.py"]:::tools
        JSONAnalysis["json_analysis.py"]:::tools
        EmbeddingTool["embedding.py"]:::tools
        CLI["nlweb.sh & setup.sh"]:::tools
    end

    %% External Services
    subgraph "External Services"
        direction TB
        VectorDBs["Vector DBs"]:::external
        LLMAPIs["LLM APIs"]:::external
        ContentStore["Schema.org/RSS Content"]:::external
    end

    %% Connections
    ClientUI -->|"HTTP POST /ask"| WebServer
    ExtAgents -->|"MCP ask"| WebServer
    WebServer --> StreamWrap
    WebServer --> StaticHandler
    WebServer --> BaseHandler
    BaseHandler --> MCPHandler
    MCPHandler --> StateMgr
    MCPHandler --> Analyze
    Analyze --> Relevance --> Decontext --> Memory --> ReqInfo
    ReqInfo --> Retriever
    Retriever --> QdrantRet
    Retriever --> AzureSearch
    Retriever --> Milvus
    Retriever --> SnowflakeRet
    QdrantRet --> VectorDBs
    AzureSearch --> VectorDBs
    Milvus --> VectorDBs
    SnowflakeRet --> VectorDBs
    Retriever --> OpenAIEmb
    Retriever --> AnthropicEmb
    Retriever --> GeminiEmb
    Retriever --> AzureEmb
    Retriever --> SnowflakeEmb
    OpenAIEmb --> OpenAI
    AnthropicEmb --> Anthropic
    GeminiEmb --> Gemini
    AzureEmb --> AzureOAI
    SnowflakeEmb --> SnowflakeLLM
    OpenAI --> GenAns
    Anthropic --> GenAns
    Gemini --> GenAns
    AzureOAI --> GenAns
    AzureDS --> GenAns
    AzureLL --> GenAns
    Inception --> GenAns
    SnowflakeLLM --> GenAns
    GenAns --> Ranking --> PostRank
    PostRank --> ChatIfc
    ChatIfc --> WebServer
    VectorDBs --> Retriever
    LLMAPIs --> OpenAI
    LLMAPIs --> Anthropic
    LLMAPIs --> Gemini
    LLMAPIs --> AzureOAI
    LLMAPIs --> AzureDS
    LLMAPIs --> AzureLL
    LLMAPIs --> Inception
    ContentStore --> Retriever

    %% Click Events
    click ClientUI "https://github.com/microsoft/nlweb/tree/main/static/"
    click WebServer "https://github.com/microsoft/nlweb/blob/main/code/webserver/WebServer.py"
    click StreamWrap "https://github.com/microsoft/nlweb/blob/main/code/webserver/StreamingWrapper.py"
    click StaticHandler "https://github.com/microsoft/nlweb/blob/main/code/webserver/static_file_handler.py"
    click BaseHandler "https://github.com/microsoft/nlweb/blob/main/code/core/baseHandler.py"
    click MCPHandler "https://github.com/microsoft/nlweb/blob/main/code/core/mcp_handler.py"
    click StateMgr "https://github.com/microsoft/nlweb/blob/main/code/core/state.py"
    click GenAns "https://github.com/microsoft/nlweb/blob/main/code/core/generate_answer.py"
    click PostRank "https://github.com/microsoft/nlweb/blob/main/code/core/post_ranking.py"
    click Ranking "https://github.com/microsoft/nlweb/blob/main/code/core/ranking.py"
    click Analyze "https://github.com/microsoft/nlweb/blob/main/code/pre_retrieval/analyze_query.py"
    click Relevance "https://github.com/microsoft/nlweb/blob/main/code/pre_retrieval/relevance_detection.py"
    click Decontext "https://github.com/microsoft/nlweb/blob/main/code/pre_retrieval/decontextualize.py"
    click Memory "https://github.com/microsoft/nlweb/blob/main/code/pre_retrieval/memory.py"
    click ReqInfo "https://github.com/microsoft/nlweb/blob/main/code/pre_retrieval/required_info.py"
    click Retriever "https://github.com/microsoft/nlweb/blob/main/code/retrieval/retriever.py"
    click QdrantRet "https://github.com/microsoft/nlweb/blob/main/code/retrieval/qdrant_retrieve.py"
    click AzureSearch "https://github.com/microsoft/nlweb/blob/main/code/retrieval/azure_search_client.py"
    click Milvus "https://github.com/microsoft/nlweb/blob/main/code/retrieval/milvus_client.py"
    click SnowflakeRet "https://github.com/microsoft/nlweb/blob/main/code/retrieval/snowflake_client.py"
    click OpenAIEmb "https://github.com/microsoft/nlweb/blob/main/code/embedding/openai_embedding.py"
    click AnthropicEmb "https://github.com/microsoft/nlweb/blob/main/code/embedding/anthropic_embedding.py"
    click GeminiEmb "https://github.com/microsoft/nlweb/blob/main/code/embedding/gemini_embedding.py"
    click AzureEmb "https://github.com/microsoft/nlweb/blob/main/code/embedding/azure_oai_embedding.py"
    click SnowflakeEmb "https://github.com/microsoft/nlweb/blob/main/code/embedding/snowflake_embedding.py"
    click OpenAI "https://github.com/microsoft/nlweb/blob/main/code/llm/openai.py"
    click Anthropic "https://github.com/microsoft/nlweb/blob/main/code/llm/anthropic.py"
    click Gemini "https://github.com/microsoft/nlweb/blob/main/code/llm/gemini.py"
    click AzureOAI "https://github.com/microsoft/nlweb/blob/main/code/llm/azure_oai.py"
    click AzureDS "https://github.com/microsoft/nlweb/blob/main/code/llm/azure_deepseek.py"
    click AzureLL "https://github.com/microsoft/nlweb/blob/main/code/llm/azure_llama.py"
    click Inception "https://github.com/microsoft/nlweb/blob/main/code/llm/inception.py"
    click SnowflakeLLM "https://github.com/microsoft/nlweb/blob/main/code/llm/snowflake.py"
    click LLMProv "https://github.com/microsoft/nlweb/blob/main/code/llm/llm_provider.py"
    click LLMBase "https://github.com/microsoft/nlweb/blob/main/code/llm/llm.py"
    click ChatIfc "https://github.com/microsoft/nlweb/blob/main/code/chatbot_interface.py"
    click ConfigAll "https://github.com/microsoft/nlweb/blob/main/code/config/config.py"
    click Logger "https://github.com/microsoft/nlweb/blob/main/code/utils/logger.py"
    click LogHelper "https://github.com/microsoft/nlweb/blob/main/code/utils/logging_config_helper.py"
    click SetLevel "https://github.com/microsoft/nlweb/blob/main/code/utils/set_log_level.py"
    click DBLoad "https://github.com/microsoft/nlweb/blob/main/code/tools/db_load.py"
    click QdrantLoad "https://github.com/microsoft/nlweb/blob/main/code/tools/qdrant_load.py"
    click RSS2Schema "https://github.com/microsoft/nlweb/blob/main/code/tools/rss2schema.py"
    click JSONAnalysis "https://github.com/microsoft/nlweb/blob/main/code/tools/json_analysis.py"
    click EmbeddingTool "https://github.com/microsoft/nlweb/blob/main/code/tools/embedding.py"

    %% Styles
    classDef frontend fill:#f9f,stroke:#333,stroke-width:1px;
    classDef backend fill:#bbf,stroke:#333,stroke-width:1px;
    classDef pipeline fill:#ffb,stroke:#333,stroke-width:1px;
    classDef plugin fill:#bfb,stroke:#333,stroke-width:1px;
    classDef external fill:#eee,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5;
    classDef config fill:#fcf,stroke:#333,stroke-width:1px;
    classDef utils fill:#cff,stroke:#333,stroke-width:1px;
    classDef tools fill:#fcc,stroke:#333,stroke-width:1px;
```
