How business-science/ai-data-science-team works
The project is a Python library and set of applications built around a multi-agent architecture that streamlines common data science tasks. Its purpose is to provide an “AI Data Science Team” where each agent specializes in activities such as data cleaning, wrangling, feature engineering, SQL queries, exploratory data analysis, machine learning (via H2O and MLflow), and even multi-agent coordination (for combined tasks like data analysis with visualization).
Main Components
• Core Library (“ai_data_science_team”): This is the heart of the project. It houses:
• The Agents submodule (found under ai_data_science_team/agents) that contains individual agents for data cleaning, loading, visualization, wrangling, feature engineering, and SQL operations.
• The Data Science Agents (ds_agents) responsible for exploratory data analysis (EDA) and related functions.
• Machine Learning Agents (ml_agents) which include agents interfacing with H2O and MLflow for ML model building and deployment.
• Multi-Agents (multiagents) that combine functionalities (for example, a Pandas data analyst that integrates data wrangling with visualization or SQL analysis).
• Supporting Modules:
• Tools (data loaders, EDA utilities, SQL connectors, etc. in ai_data_science_team/tools).
• Templates and Parsers that assist in code generation and configuration.
• Utilities (like logging, plotting, HTML helpers, regex, etc.) under ai_data_science_team/utils.
• Orchestration (ai_data_science_team/orchestration.py) which likely coordinates and manages flows between agents.
• Applications Folder (apps): Contains concrete AI-powered data science apps based on the core library – for instance:
• Exploratory Data Copilot
• Pandas Data Analyst App
• SQL Database Agent App
Relationships and Interactions
• The central orchestration module manages the calling of various agents. This module acts as the “director” that invokes proper agents based on user instructions.
• Each agent in the Agents, ds_agents, ml_agents, and multiagents directories is specialized for a distinct task. They often communicate with supporting tools (found in the tools and utils directories), which in turn abstract lower-level functionality (for example, data formatting, plotting, and SQL connections).
• The apps layer interacts with the core library via a well-defined API, using the agents to perform end-to-end workflows (e.g., load data → clean data → engineer features → run models).
• External files (such as CSV, Excel files, databases, etc.) are inputs passed into the processing chain managed by the agents.
Architectural Patterns and Technologies
• Modular design: The code is organized into clear modules based on functionality, promoting separation of concerns.
• Agent-based architecture: Different agents accomplish focused data science tasks. This design is extensible (with additional agents such as the future “Supervisor” or “Interpretability Agent”).
• Integration and orchestration: The orchestration script ties individual components together in a pipeline style.
• Technology stack includes Python with support for machine learning libraries (H2O, MLflow), data visualization (Plotly, Matplotlib), and data manipulation (pandas).