How ahmednasser5/datapilot-ai works
DataPilot AI is a Streamlit-based, deterministic-first analytics platform: users upload datasets, inspect profiles and quality findings, optionally approve explainable cleaning changes, then analyze, query, visualize, and export results. WorkflowService coordinates analysis and re-runs downstream analysis after a cleaned version is promoted. The README documents the major product capabilities; source excerpts clarify workflow, cleaning, and state ownership, but many engines and pages were not sampled, so their internal call relationships are not asserted.
Main entry points: cleaning_service.py Cleaning service · workflow_service.py Workflow service · 05_Cleaning_Studio.py Cleaning studio · analytics_service.py Analytics and AI
Components 23
User experience
- Streamlit interface
streamlit_app/app.py - Upload and pages
streamlit_app/pages/02_Upload_Data.py - Cleaning studio
streamlit_app/pages/05_Cleaning_Studio.py
Application workflow
- Workflow service
services/workflow_service.py - Dataset service
services/dataset_service.py - Cleaning service
services/cleaning_service.py
Data engines
- Format loaders
ingestion/loader_registry.py - Profiling
profiling/profiler.py - Quality assessment
quality/quality_engine.py - Cleaning planner
cleaning/cleaning_planner.py - Cleaning preview
cleaning/preview.py - Cleaning executor
cleaning/cleaning_executor.py - Cleaning validation
cleaning/cleaning_validator.py - SQL analytics
sql/query_executor.py - DuckDB engine
sql/duckdb_engine.py
Analysis and AI
- Analytics and AI
services/analytics_service.py - Domain KPIs
kpi/kpi_calculator.py - Dashboard planning
services/dashboard_service.py - AI reasoning
ai/analyst_agent.py
Outputs and state
- Versioned dataset
models/dataset.py - Exports
services/export_service.py - Shared result models
models/analysis.py
Other components
- Data analyst
Connections 29
- Data analyst → Streamlit interface: uses
README.md - Streamlit interface → Upload and pages: offers pages
- Upload and pages → Dataset service: submits dataset
- Dataset service → Format loaders: loads data
- Format loaders → Versioned dataset: normalizes into
README.md - Streamlit interface → Workflow service: starts workflows
README.md - Workflow service → Cleaning service: evaluates quality
services/workflow_service.py - Cleaning service → Profiling: profiles dataset
services/cleaning_service.py - Cleaning service → Quality assessment: evaluates issues
services/cleaning_service.py - Cleaning studio → Workflow service: runs pipeline
streamlit_app/pages/05_Cleaning_Studio.py - Cleaning studio → Cleaning service: requests plan
streamlit_app/pages/05_Cleaning_Studio.py - Cleaning service → Cleaning planner: builds plan
services/cleaning_service.py - Cleaning studio → Cleaning service: previews changes
streamlit_app/pages/05_Cleaning_Studio.py - Cleaning service → Cleaning preview: generates preview
services/cleaning_service.py - Cleaning studio → Cleaning service: applies approvals
streamlit_app/pages/05_Cleaning_Studio.py - Cleaning service → Cleaning executor: executes changes
services/cleaning_service.py - Cleaning service → Cleaning validation: validates cleaning
services/cleaning_service.py - Cleaning service → Versioned dataset: derives version
services/cleaning_service.py - Workflow service → Analytics and AI: detects domain and KPIs
services/workflow_service.py - Workflow service → Dashboard planning: builds dashboard plan
services/workflow_service.py - Analytics and AI → Domain KPIs: recommends and calculates
- Data analyst → SQL analytics: asks SQL questions
README.md - SQL analytics → DuckDB engine: executes queries
README.md - Data analyst → AI reasoning: asks analyst
README.md - Analytics and AI → AI reasoning: generates explanations
services/analytics_service.py - Workflow service → Versioned dataset: tracks versions
services/workflow_service.py - Data analyst → Exports: requests exports
README.md - Exports → Versioned dataset: exports dataset
- Analytics and AI → Shared result models: produces analysis results
services/analytics_service.py