# digai-co/datagpt architecture

> Architecture diagram of the GitHub repository digai-co/datagpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/digai-co/datagpt
- Repository: https://github.com/digai-co/datagpt

## Overview

##### Project type and purpose
- DataGPT is a full-stack AI Agent web application.
- Purpose: Accept natural-language queries via a browser UI, translate them into SQL via LLM, fetch data from a relational database, generate charts, and display results.

##### File structure
- Top-level:
- server/app.py – entrypoint for the web server and HTTP routes.
- datagpt/ – core Python package implementing the AI Agent.
- config/config.yaml – runtime configuration for OpenAI, database, and memory.
- examples/ – sample database schemas, scripts to spin up MySQL/Postgres.
- Dockerfile, Makefile – containerization and build automation.
- requirements.txt – Python dependencies.
- datagpt/ subpackages:
- action/ – “skills” the agent can perform: text2sql, fetch_data, data2chart.
- tool/db/ – database adapters (MySQL, Postgres).
- tool/llm_openai.py – wrapper around the OpenAI API.
- memory/faiss.py – vector store for schema and session memory.
- util.py, config.py, log.py – utilities, configuration loader, logging.
- datagpt.py – orchestrator gluing LLM, tools, actions into an agent loop.

##### README for additional insights
- Confirms architecture: web front end → backend application → LLM calls → DB queries → chart rendering.
- Mentions design principles (simplicity, AI Agent pattern) and key features (NL queries, visualization, web UI).

a. Main components
1. User Interface (browser) – web page for input and chart display.
2. Web Server (server/app.py + Flask/FastAPI) – HTTP endpoints.
3. Agent Orchestrator (datagpt/datagpt.py) – core loop interpreting user intent.
4. LLM Service (OpenAI API) – hosted AI model for text2sql and planning.
5. Database (PostgreSQL/MySQL) – stores domain data.
6. Memory Store (Faiss vector DB) – caches schema, previous interactions.
7. Chart Generator (matplotlib/plotly inside action/data2chart.py).
8. Configuration & Logging – config/config.yaml and datagpt/log.py.
9. Examples & DB Setup Scripts – for quickstart environments (examples/db).
b. Relationships and data flow
- User → HTTP Request → Web Server
- Server loads configuration → passes query to Agent
- Agent invokes text2sql → calls OpenAI API → generates SQL
- Agent calls fetch_data → DB adapter → executes SQL → returns rows
- Agent calls data2chart → generates chart image or JSON
- Server returns chart to UI
- Agent writes/reads schema & session embeddings in Faiss
c. Architectural patterns and principles
- AI Agent pattern: decoupled “actions” invoked dynamically.
- Modular design: clear separation between LLM interface, DB tools, and actions.
- Config-driven: supports multiple DBs and LLM endpoints.
- Simple monolith: single codebase, no microservices but clear logical modules.
d. Key technologies
- Python, Flask (or similar) for server
- OpenAI API for LLM calls
- SQLAlchemy or raw connectors in datagpt/tool/db for DB
- Faiss for vector memory
- Matplotlib/Plotly for charts
- Docker for containerization

##### Detail
- Every logical component: config loader, logging, memory, each action module.
- Sequence: user input → planning → execution → response.
- Avoid over-splitting: group related files under one component (e.g., all datagpt/tool/db files under “DB Adapter”).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend
    subgraph "Frontend Layer"
        direction TB
        Browser["User Browser UI"]:::frontend
    end

    %% Backend
    subgraph "Backend Application" 
        direction TB
        Server["Web Server\n(server/app.py)"]:::backend
        Agent["Agent Orchestrator\n(datagpt/datagpt.py)"]:::backend

        subgraph "AI Agent Actions"
            direction TB
            Text2SQL["Text-to-SQL Action\n(datagpt/action/text2sql.py)"]:::backend
            FetchData["Fetch Data Action\n(datagpt/action/fetch_data.py)"]:::backend
            Data2Chart["Data-to-Chart Action\n(datagpt/action/data2chart.py)"]:::backend
        end

        subgraph "DB Adapters"
            direction TB
            BaseAdapter["Base Adapter\n(datagpt/tool/db/base.py)"]:::backend
            MySQLAdapter["MySQL Adapter\n(datagpt/tool/db/mysql.py)"]:::backend
            PostgresAdapter["Postgres Adapter\n(datagpt/tool/db/postgres.py)"]:::backend
        end

        LLMWrapper["LLM Service Wrapper\n(datagpt/tool/llm_openai.py)"]:::external
        MemoryStore["Memory Store\n(datagpt/memory/faiss.py)"]:::datastore

        ConfigLoader["Config Loader\n(datagpt/config.py)"]:::util
        ConfigFile["YAML Config\n(config/config.yaml)"]:::datastore
        Logger["Logging Utility\n(datagpt/log.py)"]:::util
        Util["General Utilities\n(datagpt/util.py)"]:::util

        %% dependencies within backend
        Server -->|loads config| ConfigLoader
        Server -->|reads config| ConfigFile
        Server -->|handles POST /query| Agent
        Agent -->|invoke| Text2SQL
        Agent -->|invoke| FetchData
        Agent -->|invoke| Data2Chart
        Text2SQL -->|"calls LLM"| LLMWrapper
        FetchData -->|uses| BaseAdapter
        BaseAdapter -->|implements| MySQLAdapter
        BaseAdapter -->|implements| PostgresAdapter
        Agent -->|read/write| MemoryStore
        Agent -->|log actions| Logger
        Agent -->|use utilities| Util
    end

    %% External Services
    subgraph "External Services"
        direction TB
        OpenAI["OpenAI API"]:::external
    end

    %% Data Stores
    subgraph "Databases"
        direction TB
        RelationalDB["Postgres/MySQL DB"]:::datastore
    end

    %% Examples & Scripts
    subgraph "Examples & Scripts"
        direction TB
        RunMySQL["Run MySQL Script\n(examples/db/run_mysql.sh)"]:::util
        RunPG["Run Postgres Script\n(examples/db/run_pg.sh)"]:::util
        SampleSQL["Sample SQL Schemas\n(examples/db/sql/)"]:::util
    end

    %% Dev & Build Files
    subgraph "Dev & Build"
        direction TB
        Requirements["requirements.txt"]:::util
        Dockerfile["Dockerfile"]:::util
        Makefile["Makefile"]:::util
    end

    %% Connections to externals
    LLMWrapper -->|"API call"| OpenAI
    BaseAdapter -->|executes SQL| RelationalDB
    FetchData -->|SQL SELECT| RelationalDB
    Server -->|returns chart| Browser
    Data2Chart -->|generate image/JSON| Server

    %% Click Events
    click Server "https://github.com/digai-co/datagpt/blob/main/server/app.py"
    click Agent "https://github.com/digai-co/datagpt/blob/main/datagpt/datagpt.py"
    click LLMWrapper "https://github.com/digai-co/datagpt/blob/main/datagpt/tool/llm_openai.py"
    click BaseAdapter "https://github.com/digai-co/datagpt/blob/main/datagpt/tool/db/base.py"
    click MySQLAdapter "https://github.com/digai-co/datagpt/blob/main/datagpt/tool/db/mysql.py"
    click PostgresAdapter "https://github.com/digai-co/datagpt/blob/main/datagpt/tool/db/postgres.py"
    click Text2SQL "https://github.com/digai-co/datagpt/blob/main/datagpt/action/text2sql.py"
    click FetchData "https://github.com/digai-co/datagpt/blob/main/datagpt/action/fetch_data.py"
    click Data2Chart "https://github.com/digai-co/datagpt/blob/main/datagpt/action/data2chart.py"
    click MemoryStore "https://github.com/digai-co/datagpt/blob/main/datagpt/memory/faiss.py"
    click ConfigFile "https://github.com/digai-co/datagpt/blob/main/config/config.yaml"
    click ConfigLoader "https://github.com/digai-co/datagpt/blob/main/datagpt/config.py"
    click Logger "https://github.com/digai-co/datagpt/blob/main/datagpt/log.py"
    click Util "https://github.com/digai-co/datagpt/blob/main/datagpt/util.py"
    click RunMySQL "https://github.com/digai-co/datagpt/blob/main/examples/db/run_mysql.sh"
    click RunPG "https://github.com/digai-co/datagpt/blob/main/examples/db/run_pg.sh"
    click SampleSQL "https://github.com/digai-co/datagpt/tree/main/examples/db/sql/"
    click Requirements "https://github.com/digai-co/datagpt/blob/main/requirements.txt"
    click Dockerfile "https://github.com/digai-co/datagpt/tree/main/Dockerfile"
    click Makefile "https://github.com/digai-co/datagpt/tree/main/Makefile"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#70A7D1,stroke-width:2px
    classDef backend fill:#D5F5E3,stroke:#27AE60,stroke-width:2px
    classDef external fill:#FDEBD0,stroke:#E67E22,stroke-width:2px
    classDef datastore fill:#F9E79F,stroke:#B7950B,stroke-width:2px
    classDef util fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px
```
