How yuyuejia/datay works
DataY is a lightweight data platform: users configure sources and ETL flows, execute and schedule jobs, model data and metrics, then query dashboards or ask questions in natural language. The map also preserves the documented data-map, CDC, warehouse, and MCP capabilities. Edges use README statements for documented product flows and sampled source excerpts for visible service calls; unsampled implementations are represented only at their supported directory paths.
Main entry points: ETLFlowTask.java Flow Runtime · assistant AI Assistants · mcp MCP Endpoint · component ETL Designer
Components 23
Data Integration
- ETL Designer
datay-core/src/main/java/com/data/job/component - Source Connectors
datay-core/src/main/java/com/data/datasource/provider - Schema Metadata
datay-core/src/main/java/com/data/metadata - CDC Collectors
datay-core/src/main/java/com/data/job/component/cdc
Execution and Storage
- Flow Runtime
datay-core/src/main/java/com/data/job/ETLFlowTask.java - Job Scheduler
datay-web/src/main/java/com/data/datafusion/service/scheduler - Run Status
datay-core/src/main/java/com/data/status/StatusStorageStrategy.java - File Storage
datay-core/src/main/java/com/data/file/FileStorageStrategy.java
Modeling and Analytics
- Analytical Engines
datay-core/src/main/java/com/data/job/DuckDBEngine.java - Dimensional Models
datay-web/src/main/java/com/data/datafusion/domain - Metric Queries
datay-web/src/main/java/com/data/datafusion/service/MetricQueryService.java - Datasets and Dashboards
datay-web/src/main/java/com/data/datafusion/service/dashboard/DatasetQueryService.java - Data Map
datay-web/src/main/java/com/data/datafusion/service/datamap/DataMapQueryService.java
AI and MCP
- AI Assistants
datay-web/src/main/java/com/data/datafusion/ai/assistant - Metric RAG
datay-web/src/main/java/com/data/datafusion/ai/rag/MetricRagService.java - Agent Tools
datay-web/src/main/java/com/data/datafusion/ai/tool - MCP Endpoint
datay-web/src/main/java/com/data/datafusion/mcp
Platform Services
- Source Query Service
datay-web/src/main/java/com/data/datafusion/service/DataSourceQueryService.java - LLM Configuration
datay-web/src/main/java/com/data/datafusion/ai/llm/LlmConfigService.java
Other components
- Platform User
- External Data Stores
- LLM Provider
- MCP Client
Connections 26
- Platform User → ETL Designer: designs flows
- ETL Designer → Flow Runtime: configures
- Flow Runtime → Source Connectors: runs connectors
- Flow Runtime → CDC Collectors: runs CDC
- Source Connectors → Schema Metadata: reads schemas
datay-core/src/main/java/com/data/datasource/provider/TdengineDataSourceProvider.java - Flow Runtime → Analytical Engines: processes data
- Job Scheduler → Flow Runtime: dispatches jobs
- Flow Runtime → Run Status: records status
- Flow Runtime → File Storage: writes results
- Analytical Engines → External Data Stores: loads data
- Platform User → Dimensional Models: defines models
- Dimensional Models → Metric Queries: supplies semantics
datay-web/src/main/java/com/data/datafusion/service/MetricQueryService.java - Platform User → Datasets and Dashboards: builds dashboards
- Datasets and Dashboards → Source Query Service: executes SQL
datay-web/src/main/java/com/data/datafusion/service/dashboard/DatasetQueryService.java - Source Query Service → External Data Stores: queries
datay-web/src/main/java/com/data/datafusion/service/DataSourceQueryService.java - Platform User → Metric Queries: queries metrics
- Platform User → Data Map: searches assets
- Platform User → AI Assistants: asks questions
- AI Assistants → Metric RAG: retrieves context
datay-web/src/main/java/com/data/datafusion/ai/rag/MetricRagService.java - AI Assistants → Agent Tools: uses tools
- AI Assistants → Metric Queries: queries metrics
- LLM Configuration → LLM Provider: configures client
- MCP Client → MCP Endpoint: invokes tools
- MCP Endpoint → AI Assistants: shares query logic
- MCP Endpoint → ETL Designer: manages tasks
- MCP Endpoint → Dimensional Models: manages models