How leonxu6/datasteward works
DataSteward is a self-hosted governed data platform where an agent or external MCP client asks questions through a shared governance kernel rather than receiving direct database access. The main path runs from a user through the LangGraph agent and governed tools to governed data and a response. The graph also retains the documented ingestion, warehouse/modeling, orchestration, writeback, and governance-console capabilities. Some subsystems are represented at their documented directory paths because their implementation excerpts were not provided; those links rely on README claims rather than inspected call sites.
Main entry points: tools Governed tools · graph.py LangGraph agent · store.py StarRocks warehouse · app.py Streamlit console
Components 23
User surfaces
- LangGraph agent
src/dm/agent/graph.py - DingTalk channel
src/dm/channels/dingtalk.py - Streamlit console
src/dm/app/app.py - MCP server
src/dm/connector/mcp_server.py
Data platform
- Source connectors
src/dm/connect - U8 batch sync
src/dm/connect/u8_mapping.py - Flink CDC
src/dm/pipeline - Postgres source
- StarRocks warehouse
src/dm/warehouse/store.py - dbt modeling
transform/dbt - Dataset lineage
src/dm/datasets/model.py - Metrics layer
src/dm/ontology/metrics.py - Document RAG
src/dm/tools/documents.py - Knowledge graph
src/dm/kg
Governance kernel
- Governed tools
src/dm/tools - Access policy
src/dm/security/policy.py - Access principal
src/dm/tools/principal.py - Append-only audit
src/dm/tools/audit.py - Ontology actions
src/dm/ontology/actions.py
Platform operations
- Dagster orchestration
src/dm/orchestration
Other components
- User
- MCP client
- OpenAI-compatible LLM
Connections 29
- User → LangGraph agent: asks
README.md - LangGraph agent → OpenAI-compatible LLM: reasons with
README.md - LangGraph agent → Governed tools: calls in-process
src/dm/agent/graph.py - Governed tools → LangGraph agent: returns results
README.md - LangGraph agent → DingTalk channel: serves through
README.md - User → Streamlit console: uses
README.md - Streamlit console → LangGraph agent: offers agent UI
- MCP client → MCP server: connects via
README.md - MCP server → Governed tools: delegates to
src/dm/connector/mcp_server.py - Source connectors → Postgres source: extracts from
README.md - U8 batch sync → StarRocks warehouse: batch loads
src/dm/connect/u8_mapping.py - Postgres source → Flink CDC: streams changes
README.md - Flink CDC → StarRocks warehouse: replicates to
README.md - StarRocks warehouse → dbt modeling: feeds models
README.md - dbt modeling → Metrics layer: defines measures for
README.md - dbt modeling → Dataset lineage: registers lineage
src/dm/datasets/model.py - Metrics layer → Governed tools: serves governed queries
README.md - StarRocks warehouse → Governed tools: serves governed reads
README.md - Document RAG → Governed tools: serves search
README.md - Knowledge graph → Governed tools: serves graph queries
README.md - Governed tools → Access policy: enforces access
src/dm/tools/data.py - Governed tools → Access principal: uses identity
src/dm/tools/__init__.py - Governed tools → Append-only audit: records calls
src/dm/tools/data.py - Append-only audit → StarRocks warehouse: stores log data
src/dm/tools/audit.py - Governed tools → Ontology actions: exposes writebacks
src/dm/tools/__init__.py - Ontology actions → Postgres source: writes after approval
README.md - Dagster orchestration → dbt modeling: orchestrates builds
README.md - Streamlit console → Dataset lineage: surfaces lineage
README.md - Streamlit console → Append-only audit: replays governance
README.md