How mouleshgs/autonomous-data-engineering works
Orbital is an autonomous data-engineering MVP: users upload datasets, inspect profiles and quality, run an adaptive cleaning pipeline, and review processed output, decisions, and analytics. The documented pipeline runs through a LangGraph workflow; its stages load and profile data, plan safe operations, clean and validate, then persist results and optionally benchmark a selected model. The graph and tool excerpts support the workflow and data operations; several dashboard/API connections and optional integrations are documented or only partly visible, so uncertain links are marked dashed or omitted.
Main entry points: agent_graph.py LangGraph Workflow · App.tsx React Dashboard · main.py HTTP API · core.py Runtime Registry
Components 16
User Experience
- React Dashboard
frontend/src/App.tsx
API Runtime
- HTTP API
backend/app/main.py - Runtime Registry
backend/app/core.py - Pipeline State
backend/app/schemas/pipeline_state.py
Data Pipeline
- LangGraph Workflow
backend/app/graph/agent_graph.py - Source Analysis
backend/app/graph/agent_graph.py - Data Profiling
backend/app/graph/agent_graph.py - Adaptive Planning
backend/app/graph/agent_graph.py - Cleaning and Validation
backend/app/graph/agent_graph.py - Output Transformation
backend/app/graph/agent_graph.py - Model Benchmark
backend/app/agents/model_benchmark.py
Data Operations
- Deterministic Data Tools
backend/app/tools/data_tools.py - Agent Logging
backend/app/agents/base.py - Processed CSV
backend/app/agents/storage.py
Other components
- Dataset User
- Ollama Model
Connections 15
- Dataset User → React Dashboard: uploads and runs
README.md - React Dashboard → HTTP API: requests
- HTTP API → Runtime Registry: uses runtime
backend/app/main.py - Runtime Registry → LangGraph Workflow: runs pipeline
backend/app/core.py - LangGraph Workflow → Source Analysis: dispatches
backend/app/graph/agent_graph.py - LangGraph Workflow → Data Profiling: dispatches
README.md - LangGraph Workflow → Adaptive Planning: dispatches
README.md - LangGraph Workflow → Cleaning and Validation: dispatches
README.md - LangGraph Workflow → Output Transformation: dispatches
README.md - LangGraph Workflow → Model Benchmark: optionally benchmarks
- LangGraph Workflow → Deterministic Data Tools: calls data tools
backend/app/graph/agent_graph.py - LangGraph Workflow → Pipeline State: updates state
backend/app/graph/agent_graph.py - Output Transformation → Processed CSV: persists CSV
backend/app/graph/agent_graph.py - Adaptive Planning → Ollama Model: optionally plans with
README.md - Source Analysis → Agent Logging: records decisions