How canner/wrenai works
1. Purpose and Delivery Surfaces
WrenAI is an open-source generative-BI context and execution stack: agents use governed semantic definitions to plan SQL, execute it against customer data platforms, recall business knowledge, and generate browser-side dashboards. The maintained product is a multi-language monorepo, not a monolithic web application.
Primary entry points:
• Python CLI/runtime: core/wren/src/wren/cli.py, core/wren/src/wren/engine.py
• Rust semantic planning engine: core/wren-core/core/src/lib.rs
• Python native bindings: core/wren-core-py/src/lib.rs
• Browser/WASM SDK: core/wren-core-wasm/src/lib.rs, core/wren-core-wasm/sdk/src/index.ts
2. Semantic Contract: MDL
MDL is the system boundary between business context and physical data sources. It represents models, datasets, relationships, views, cubes, metrics, functions, dialect behavior, lineage, and type planning. A shared Rust base defines manifest types, building, validation/migration utilities, and Python-oriented manifest support; the JSON schema is distributed separately.
Key paths:
• core/wren-core-base/src/mdl/manifest.rs
• core/wren-core-base/src/mdl/builder.rs
• core/wren-core/core/src/mdl/mod.rs
• core/wren-mdl/mdl.schema.json
Project-local MDL, instructions, example queries, and knowledge files are intended to be versioned with a user’s project rather than stored in a hosted service.
3. Governed SQL Planning Engine
wren-core is the Rust/DataFusion semantic engine. It transforms SQL expressed over MDL concepts into source-executable SQL: construct context and semantic objects, analyze logical plans, expand views and model relations, enforce scope/access-control stages where enabled, optimize type/timestamp behavior, then unparse into a target dialect.
Key paths:
• core/wren-core/core/src/mdl/context.rs
• core/wren-core/core/src/logical_plan/analyze/plan.rs
• core/wren-core/core/src/logical_plan/analyze/expand_view.rs
• core/wren-core/core/src/logical_plan/optimize/mod.rs
• core/wren-core/core/src/logical_plan/unparser.rs
This engine is a planning/rewrite layer; data execution remains behind connectors and target warehouses.
4. Python Runtime, CLI, and Connectors
The wrenai Python package is the agent-facing operational layer. Its CLI commands manage projects/context, profile connections, query through the MDL engine, serve local capabilities, expose MCP, and package agent prompts/skills. engine.py bridges CLI workflows to planning and query execution; context.py manages project context.
Connectors share a base/factory boundary and implement backend-specific connection, SQL handling, limits, and metadata behavior for DataFusion/DuckDB and remote databases.
Key paths:
• core/wren/src/wren/cli.py
• core/wren/src/wren/engine.py
• core/wren/src/wren/connector/base.py
• core/wren/src/wren/connector/factory.py
• core/wren/src/wren/mcp_server.py
Material external systems include PostgreSQL, BigQuery, Snowflake, ClickHouse, Redshift, Databricks, Athena, Trino, Spark, MySQL, SQL Server, Oracle, DuckDB, and DataFusion.
5. Context Retrieval and Agent Interfaces
The AI context layer combines structured MDL with markdown/business knowledge and reusable query examples. The memory subsystem indexes and retrieves this content using embeddings and a local LanceDB-backed index. Agent-facing access is available through MCP, CLI skill delivery, and framework SDKs; the agent owns orchestration and LLM selection.
Key paths:
• core/wren/src/wren/memory/schema_indexer.py
• core/wren/src/wren/memory/store.py
• core/wren/src/wren/skills_delivery.py
• sdk/wren-langchain/src/wren_langchain/_toolkit.py
• sdk/wren-pydantic/src/wren_pydantic/_toolkit.py
6. Dashboard Build and Deployment Path
GenBI builds static/browser-side dashboard applications from project context and uses the WASM engine for in-browser query/planning capabilities. The Python GenBI subsystem composes app artifacts, validates them, and deploys to the user’s own Vercel or Cloudflare Pages account.
Key paths:
• core/wren/src/wren/genbi/composer.py
• core/wren/src/wren/genbi/verify.py
• core/wren/src/wren/genbi/providers/vercel.py
• core/wren/src/wren/genbi/providers/cloudflare.py
• core/wren-core-wasm/sdk/src/index.ts
7. Runtime Boundaries and Packaging
Rust is the authoritative planner implementation, exposed to Python through wren-core-py and to browsers through wren-core-wasm. Python provides local CLI, connectors, project/context management, memory, and deployment orchestration. TypeScript is a thin WASM SDK surface. Releases publish independently versioned Rust crates, Python packages, WASM/npm artifacts, and agent SDKs.
Key paths:
• core/wren-core/Cargo.toml
• core/wren-core-py/Cargo.toml
• core/wren-core-wasm/Cargo.toml
• core/wren/pyproject.toml
Main entry points: lib.rs Wren Core · engine.py Runtime engine · mcp_server.py MCP server · manifest.rs MDL manifest builder
Components 20
Semantic Planning
- MDL schema
core/wren-mdl/mdl.schema.json - MDL manifest builder · Defines manifest types, construction, validation, and migration.
core/wren-core-base/src/mdl/manifest.rs - Wren Core · Governed SQL rewrite layer built on DataFusion.
core/wren-core/core/src/lib.rs - Semantic context
core/wren-core/core/src/mdl/context.rs - Logical-plan analysis · Analyzes plans and expands semantic views and relations.
core/wren-core/core/src/logical_plan/analyze/plan.rs - Plan optimization · Applies type and timestamp behavior.
core/wren-core/core/src/logical_plan/optimize/mod.rs - Source SQL output · Emits executable SQL for the target dialect.
core/wren-core/core/src/logical_plan/unparser.rs
Python Runtime
- Python native bindings
core/wren-core-py/src/lib.rs - wrenai CLI · Manages projects, profiles, queries, local serving, and agent capabilities.
core/wren/src/wren/cli.py - Runtime engine · Bridges CLI workflows, planning, and query execution.
core/wren/src/wren/engine.py - Connector factory · Selects backend-specific connection and SQL behavior.
core/wren/src/wren/connector/factory.py - Context indexer · Indexes MDL, knowledge, and example queries.
core/wren/src/wren/memory/schema_indexer.py - Local memory store
core/wren/src/wren/memory/store.py
Agent & Dashboard Surfaces
- MCP server
core/wren/src/wren/mcp_server.py - Framework SDKs · LangChain and Pydantic agent integrations.
sdk/wren-langchain/src/wren_langchain/_toolkit.py - GenBI composer · Composes static dashboard application artifacts.
core/wren/src/wren/genbi/composer.py - Browser WASM SDK · Exposes in-browser planning and query capabilities.
core/wren-core-wasm/sdk/src/index.ts
Other components
- Versioned project context · Project-local MDL, instructions, knowledge, and examples.
- Customer data platforms · Remote databases plus local DuckDB and DataFusion execution.
- User-owned hosting · Vercel or Cloudflare Pages accounts.
Connections 23
- Versioned project context → MDL schema: conforms to
- Versioned project context → MDL manifest builder: loads MDL
- MDL schema → MDL manifest builder: distributes contract
- MDL manifest builder → Semantic context: builds
- Semantic context → Logical-plan analysis: supplies semantic objects
- Logical-plan analysis → Plan optimization: rewrites plan
- Plan optimization → Source SQL output: optimized plan
- Wren Core → Semantic context: contains
- Wren Core → Logical-plan analysis: runs
- Wren Core → Source SQL output: produces
- Python native bindings → Wren Core: native access
- wrenai CLI → Runtime engine: invokes
- Runtime engine → Python native bindings: plans through
- Runtime engine → Connector factory: executes through
- Connector factory → Customer data platforms: queries
- Versioned project context → Context indexer: indexes context
- Context indexer → Local memory store: stores embeddings
- MCP server → Runtime engine: agent query access
- MCP server → Local memory store: retrieves context
- Framework SDKs → MCP server: agent tools
- GenBI composer → Browser WASM SDK: builds browser app with
- GenBI composer → User-owned hosting: deploys to
- Browser WASM SDK → Wren Core: WASM planner build