How nitin-mane/intel-ai-assistant-builder works
Project type and purpose
• This is an open-source Gen-AI reference design platform—a Windows desktop application (Intel® AI Assistant Builder, a.k.a. SuperBuilder) that exposes a local API service for building custom AI assistants.
• It runs entirely on-device, leveraging Intel® OpenVINO for model inference, supports LLMs, RAG pipelines, agentic workflows, and offers sample client code in C#, Go, and Python.
File structure
• Top-level docs: README.md, basic_usage.md, special_query.md describe features, usage, API.
• example/ contains three language‐specific client samples:
• example/csharp: .NET Core client calling AssistantService via gRPC
• example/golang: Go client with .pb.go stubs
• example/python: Python client with generated superbuilder_middleware_pb2/_grpc, helpers, and tests
• The presence of .proto-generated code in Go and Python indicates a gRPC API layer.
• No service source code here—clients target a prebuilt binary/service.
• Media/ holds UI screenshots used in docs—reference front-end design.
README for architectural insights
• The core entry point is AssistantService (gRPC/REST API)
• Key modules: model management (load, convert, upload), knowledge ingestion/RAG, parameter config, special query functions, agentic workflows
• Hardware acceleration: CPU, GPU (OpenVINO), NPU
Main components
a. Front-End Reference UI (Windows Desktop App or Custom Web UI)
• Provides chat interface, settings pages, model management screens
b. SuperBuilder Service API (AssistantService)
• gRPC/REST server exposing endpoints for chat, config, ingestion, conversion, model mgmt
c. Core Modules inside the service
• Model Manager & Converter (OpenVINO, ModelScope, Hugging Face integrations)
• Knowledge Base / RAG Pipeline (file ingestion, embedding store, retriever)
• Chat Engine (LLM orchestration, context management, conversation history)
• Special Query Modules (tabular data, image, resume match, etc.)
• Agentic Workflow Engine (multi-step pipelines, MCP support upcoming)
d. Data Stores
• Local File System: uploaded docs, model binaries, converted models, configuration profiles
• (Optionally) Embedded DB or key-value store for embeddings/indexes
e. External Libraries & Services
• Intel® OpenVINO runtime
• gRPC Framework & Protobuf stubs
• OS Drivers (Intel® Graphics & NPU drivers)
f. Hardware Acceleration
• CPU, GPU, NPU blocks invoked by OpenVINO at inference time
g. Sample Clients (C#, Go, Python)
• Integration via generated stubs calling AssistantService
Relationships and data flow
• User interacts with Front-End → sends gRPC/HTTP requests to AssistantService
• AssistantService routes requests to appropriate module:
• Chat request → Chat Engine → Model Manager → OpenVINO → Hardware
• Ingestion request → Knowledge Base → File System + Embedding Store
• Conversion/upload requests → Model Manager → OpenVINO conversion utilities
• Special Query → Special Query Modules → Data Parsers → Model Manager
• Outputs flow back through AssistantService → Front-End → User
Architectural patterns & design principles
• Modular Monolith: discrete modules/plugins within a single service process
• API-First Design: gRPC/REST interface as the integration contract
• Layered Architecture: Presentation (UI) → API Layer → Domain Modules → Infrastructure (OpenVINO, FS)
• Plugin/Extension Points: special queries, workflows, model conversion, sample clients
Essential architectural elements
• Core modules (model mgmt, chat engine, RAG, special queries)
• Hardware acceleration path (OpenVINO → GPU/NPU/CPU)
• Plugin nature of sample clients and extensibility points