# sinaptik-ai/pandas-ai architecture

> Architecture diagram of the GitHub repository sinaptik-ai/pandas-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sinaptik-ai/pandas-ai
- Repository: https://github.com/sinaptik-ai/pandas-ai

## Overview

1. **What this repo is**
   PandasAI is a Python library for natural-language-to-data analysis. The primary runtime is a library API, not a server: users import `pandasai`, configure an LLM, load data into PandasAI dataframe abstractions, and ask questions through `chat()`.

2. **Top-level architecture**
   The repository is split into a **core library** (`pandasai/`) plus **optional extension packages** under `extensions/`. The core handles orchestration, prompt generation, code/SQL synthesis, execution, and response shaping. Extensions add pluggable integrations for LLMs, databases, vector stores, and sandboxing.

3. **User-facing entry points**
   The main entry points are `pandasai.DataFrame`, `pandasai.chat(...)`, `pandasai.config`, and the CLI under `pandasai/cli/main.py`. The library supports both single-dataframe and multi-dataframe conversations, and also has a “smart datalake” abstraction for broader data access.

4. **Core request flow**
   A typical request starts with a user query and one or more data objects. The agent layer (`pandasai/agent`) coordinates state, wraps the query, and routes it into prompt construction. The prompt layer uses schema/data context, output-type expectations, and SQL-specific templates when needed.

5. **LLM boundary**
   LLM integration is intentionally abstracted through `pandasai/llm/base.py`. The repo itself includes a fake LLM for tests, while real provider adapters live in extensions such as `pandasai-openai` and `pandasai-litellm`. This keeps the core independent of any single model vendor.

6. **Code generation and validation**
   Generated output is not executed blindly. The `pandasai/core/code_generation` subsystem cleans and validates model output before execution. This is the main safety boundary between untrusted LLM text and runtime execution.

7. **Execution boundary**
   The execution path is handled by `pandasai/core/code_execution` and can run either locally or inside a sandbox. The core sandbox abstraction is complemented by the Docker-based sandbox extension in `extensions/sandbox/docker`, which isolates generated code in a separate containerized environment.

8. **Data access layer**
   Data ingestion is split by source type. `pandasai/data_loader` includes local loading, SQL loading, DuckDB connection management, view loading, and semantic-layer schema support. The query builders in `pandasai/query_builders` adapt those sources into executable local queries, SQL queries, or view queries, with pagination and SQL parsing/transformation support where needed.

9. **Response shaping**
   After execution, results are converted into typed response objects under `pandasai/core/response` such as dataframe, number, string, chart, or error. This gives the library a stable output contract regardless of whether the model produced tabular data, text, or a visualization.

10. **Safety and support subsystems**
   The `pandasai/helpers` package provides cross-cutting concerns such as logging, session/memory, file handling, telemetry, JSON encoding, environment handling, and SQL sanitization. These utilities are important architectural boundaries because they support reproducibility, observability, and risk reduction around generated queries/code.

11. **Extensions ecosystem**
   The `extensions/` tree contains independently packaged integrations. There are connectors for SQL sources, Yahoo Finance, and enterprise-grade databases like BigQuery, Databricks, Oracle, and Snowflake; vector stores such as ChromaDB, LanceDB, Milvus, Pinecone, and Qdrant; and LLM adapters for OpenAI and LiteLLM. These packages extend the core without embedding all dependencies into the base install.

12. **Enterprise and skills**
   The `pandasai/ee` area and `extensions/ee` packages indicate a separate enterprise feature surface, including a skills manager and enterprise connectors/vector stores. This suggests a product boundary between the open-source core and paid or restricted functionality.

13. **Documentation and compatibility layers**
   The `docs/v2` and `docs/v3` directories show two major API generations, including migration and compatibility guidance. That implies the architecture must preserve backward compatibility while evolving the semantic-layer, agent, and chat APIs.

14. **Validation and release workflow**
   The repository has separate core and extension CI workflows, plus packaging metadata for multiple distributable packages. Architecturally, this reinforces that the system is a modular Python ecosystem: the core library is tested and released alongside optional integration packages with their own test suites.

Main entry points: [`pandasai/data_loader`](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/data_loader) (Data loaders), [`pandasai/agent/base.py`](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/agent/base.py) (Agent), [`pandasai/core/prompts`](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/prompts) (Prompts), [`pandasai/__init__.py`](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/__init__.py) (PandasAI API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core library"]
  node_pandasai_init["PandasAI API<br/>library entry<br/>[__init__.py]"]
  node_config["Config<br/>settings<br/>[config.py]"]
  node_cli["CLI<br/>command line<br/>[main.py]"]
  node_agent{{"Agent<br/>orchestration<br/>[base.py]"}}
  node_agent_state["State<br/>[state.py]"]
  node_prompts["Prompts<br/>prompting"]
  node_codegen{{"Code gen<br/>code synthesis"}}
  node_executor{{"Executor<br/>runtime execution"}}
  node_responses["Responses<br/>result model"]
  node_dataframe["Dataframe<br/>data abstraction"]
  node_datalake["Datalake<br/>multi-source data"]
  node_loaders[("Data loaders<br/>ingestion")]
  node_query_builders{{"Query builders<br/>query planning"}}
  node_llm_base["LLM interface<br/>model boundary<br/>[base.py]"]
  node_helpers["Helpers<br/>cross-cutting"]
  node_sandbox_core{{"Sandbox API<br/>execution boundary<br/>[sandbox.py]"}}
  node_ee_skills["Skills<br/>enterprise features<br/>[manager.py]"]
end

subgraph group_ext["Extensions"]
  node_openai_ext["OpenAI<br/>LLM adapter"]
  node_litellm_ext["LiteLLM<br/>LLM adapter"]
  node_sql_ext[("SQL connector<br/>data connector")]
  node_yfinance_ext[("YFinance<br/>data connector")]
  node_docker_sandbox{{"Docker sandbox<br/>sandbox adapter<br/>[docker_sandbox.py]"}}
  node_ee_connectors[("EE connectors<br/>enterprise connectors")]
  node_ee_vectorstores[("EE vectorstores<br/>enterprise vector")]
end

subgraph group_docs["Docs"]
  node_docs_v2["Docs v2"]
  node_docs_v3["Docs v3"]
end

node_pandasai_init -->|"configures"| node_config
node_pandasai_init -->|"chat flow"| node_agent
node_pandasai_init -->|"wraps data"| node_dataframe
node_pandasai_init -->|"multi-data"| node_datalake
node_cli -->|"invokes"| node_agent
node_agent -->|"tracks"| node_agent_state
node_agent -->|"builds prompts"| node_prompts
node_prompts -->|"requests model"| node_llm_base
node_llm_base -->|"returns text"| node_codegen
node_codegen -->|"executes"| node_executor
node_executor -->|"formats"| node_responses
node_dataframe -->|"loads from"| node_loaders
node_datalake -->|"ingests"| node_loaders
node_loaders -->|"plans queries"| node_query_builders
node_query_builders -->|"context"| node_prompts
node_helpers -->|"supports"| node_agent
node_helpers -->|"sanitizes"| node_codegen
node_helpers -->|"supports"| node_executor
node_sandbox_core -->|"isolates"| node_executor
node_docker_sandbox -->|"implements"| node_sandbox_core
node_llm_base -->|"adapter"| node_openai_ext
node_llm_base -->|"adapter"| node_litellm_ext
node_loaders -->|"extension path"| node_sql_ext
node_loaders -->|"extension path"| node_yfinance_ext
node_ee_skills -->|"extends"| node_prompts
node_ee_connectors -->|"extends"| node_loaders
node_ee_vectorstores -->|"augments"| node_prompts
node_docs_v2 -->|"migration"| node_docs_v3

click node_pandasai_init "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/__init__.py"
click node_config "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/config.py"
click node_cli "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/cli/main.py"
click node_agent "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/agent/base.py"
click node_agent_state "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/agent/state.py"
click node_prompts "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/prompts"
click node_codegen "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/code_generation"
click node_executor "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/code_execution"
click node_responses "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/response"
click node_dataframe "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/dataframe"
click node_datalake "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/smart_datalake"
click node_loaders "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/data_loader"
click node_query_builders "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/query_builders"
click node_llm_base "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/llm/base.py"
click node_helpers "https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/helpers"
click node_sandbox_core "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/sandbox/sandbox.py"
click node_ee_skills "https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/ee/skills/manager.py"
click node_openai_ext "https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/llms/openai/pandasai_openai"
click node_litellm_ext "https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/llms/litellm/pandasai_litellm"
click node_sql_ext "https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/connectors/sql/pandasai_sql"
click node_yfinance_ext "https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/connectors/yfinance/pandasai_yfinance"
click node_docker_sandbox "https://github.com/sinaptik-ai/pandas-ai/blob/main/extensions/sandbox/docker/pandasai_docker/docker_sandbox.py"
click node_ee_connectors "https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/ee/connectors"
click node_ee_vectorstores "https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/ee/vectorstores"
click node_docs_v2 "https://github.com/sinaptik-ai/pandas-ai/tree/main/docs/v2"
click node_docs_v3 "https://github.com/sinaptik-ai/pandas-ai/tree/main/docs/v3"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pandasai_init,node_config,node_cli,node_agent,node_agent_state,node_prompts,node_codegen,node_executor,node_responses,node_dataframe,node_datalake,node_loaders,node_query_builders,node_llm_base,node_helpers,node_sandbox_core,node_ee_skills toneBlue
class node_openai_ext,node_litellm_ext,node_sql_ext,node_yfinance_ext,node_docker_sandbox,node_ee_connectors,node_ee_vectorstores toneAmber
class node_docs_v2,node_docs_v3 toneMint
```

## Components

### Core library

Primary runtime for natural-language data analysis.

- [PandasAI API](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/__init__.py): library entry · Top-level package surface for importing the library and common entry points. · `pandasai/__init__.py`
- [Config](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/config.py): settings · Library configuration and runtime options. · `pandasai/config.py`
- [CLI](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/cli/main.py): command line · Command-line entry point for running the library from the shell. · `pandasai/cli/main.py`
- [Agent](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/agent/base.py): orchestration · Coordinates user queries, state, and the analysis flow. · `pandasai/agent/base.py`
- [State](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/agent/state.py): conversation state · Session and turn state used by the agent during a chat. · `pandasai/agent/state.py`
- [Prompts](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/prompts): prompting · Templates and prompt builders for system, SQL, and error correction prompts. · `pandasai/core/prompts`
- [Code gen](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/code_generation): code synthesis · Cleans and validates model output before execution. · `pandasai/core/code_generation`
- [Executor](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/code_execution): runtime execution · Runs generated code either locally or through a sandbox boundary. · `pandasai/core/code_execution`
- [Responses](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/core/response): result model · Typed response objects for dataframes, numbers, strings, charts, and errors. · `pandasai/core/response`
- [Dataframe](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/dataframe): data abstraction · User-facing dataframe abstractions, including virtual and smart dataframe behavior. · `pandasai/dataframe`
- [Datalake](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/smart_datalake): multi-source data · Broader data access abstraction for conversations over multiple data sources. · `pandasai/smart_datalake`
- [Data loaders](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/data_loader): ingestion · Loads local files, SQL sources, views, DuckDB connections, and semantic schemas. · `pandasai/data_loader`
- [Query builders](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/query_builders): query planning · Builds local, SQL, and view queries and handles pagination and SQL transforms. · `pandasai/query_builders`
- [LLM interface](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/llm/base.py): model boundary · Abstract LLM contract that isolates the core from provider-specific implementations. · `pandasai/llm/base.py`
- [Helpers](https://github.com/sinaptik-ai/pandas-ai/tree/main/pandasai/helpers): cross-cutting · Logging, memory, sessions, telemetry, sanitization, file handling, and encoding support. · `pandasai/helpers`
- [Sandbox API](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/sandbox/sandbox.py): execution boundary · Core sandbox abstraction for isolating generated code. · `pandasai/sandbox/sandbox.py`
- [Skills](https://github.com/sinaptik-ai/pandas-ai/blob/main/pandasai/ee/skills/manager.py): enterprise features · Enterprise skills management and related restricted surface area. · `pandasai/ee/skills/manager.py`

### Extensions

Optional packaged integrations and sandboxes.

- [OpenAI](https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/llms/openai/pandasai_openai): LLM adapter · Provider adapter package for OpenAI and Azure OpenAI. · `extensions/llms/openai/pandasai_openai`
- [LiteLLM](https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/llms/litellm/pandasai_litellm): LLM adapter · Provider adapter package for LiteLLM-backed model access. · `extensions/llms/litellm/pandasai_litellm`
- [SQL connector](https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/connectors/sql/pandasai_sql): data connector · Optional SQL source integration packaged separately from core. · `extensions/connectors/sql/pandasai_sql`
- [YFinance](https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/connectors/yfinance/pandasai_yfinance): data connector · Optional market-data connector packaged as an extension. · `extensions/connectors/yfinance/pandasai_yfinance`
- [Docker sandbox](https://github.com/sinaptik-ai/pandas-ai/blob/main/extensions/sandbox/docker/pandasai_docker/docker_sandbox.py): sandbox adapter · Containerized sandbox implementation for isolated execution. · `extensions/sandbox/docker/pandasai_docker/docker_sandbox.py`
- [EE connectors](https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/ee/connectors): enterprise connectors · Enterprise database connectors such as BigQuery, Databricks, Oracle, and Snowflake. · `extensions/ee/connectors`
- [EE vectorstores](https://github.com/sinaptik-ai/pandas-ai/tree/main/extensions/ee/vectorstores): enterprise vector · Enterprise vector store integrations such as ChromaDB, LanceDB, Milvus, Pinecone, and Qdrant. · `extensions/ee/vectorstores`

### Docs

Versioned API and migration documentation.

- [Docs v2](https://github.com/sinaptik-ai/pandas-ai/tree/main/docs/v2): docs · Older API generation with migration-relevant guidance and examples. · `docs/v2`
- [Docs v3](https://github.com/sinaptik-ai/pandas-ai/tree/main/docs/v3): docs · Current API generation covering chat, semantic layer, and migration guidance. · `docs/v3`

## Connections

- PandasAI API → Config: configures · Library entry uses runtime configuration.
- PandasAI API → Agent: chat flow · Public chat and dataframe APIs route into the agent orchestration layer.
- PandasAI API → Dataframe: wraps data · Top-level API exposes dataframe abstractions to users.
- PandasAI API → Datalake: multi-data · Top-level API also exposes the datalake abstraction.
- CLI → Agent: invokes · CLI delegates work to the same core orchestration path.
- Agent → State: tracks · Agent maintains conversational and execution state.
- Agent → Prompts: builds prompts · Agent prepares prompt context and output expectations.
- Prompts → LLM interface: requests model · Prompts are sent through the abstract LLM boundary.
- LLM interface → Code gen: returns text · Model output is passed to code generation and validation.
- Code gen → Executor: executes · Validated generated code flows into execution.
- Executor → Responses: formats · Execution results are normalized into typed responses.
- Dataframe → Data loaders: loads from · Dataframe abstractions are backed by source-specific loaders.
- Datalake → Data loaders: ingests · Datalake composes multiple data sources through loaders.
- Data loaders → Query builders: plans queries · Loaders rely on query builders for source-specific query generation.
- Query builders → Prompts: context · Query builders supply source context used in prompt construction.
- Helpers → Agent: supports · Shared utilities support orchestration, logging, memory, and telemetry.
- Helpers → Code gen: sanitizes · Helpers provide SQL and environment safety utilities used before execution.
- Helpers → Executor: supports · Execution depends on file, session, and serialization helpers.
- Sandbox API → Executor: isolates · Core execution can delegate to a sandbox boundary.
- Docker sandbox → Sandbox API: implements · Docker extension provides a concrete sandbox implementation.
- LLM interface → OpenAI: adapter · OpenAI extension implements the model boundary.
- LLM interface → LiteLLM: adapter · LiteLLM extension implements the model boundary.
- Data loaders → SQL connector: extension path · SQL connector is an optional packaged source integration.
- Data loaders → YFinance: extension path · YFinance connector is an optional packaged source integration.
- Skills → Prompts: extends · Enterprise skills surface affects prompt and tool behavior.
- EE connectors → Data loaders: extends · Enterprise connectors add additional data source backends.
- EE vectorstores → Prompts: augments · Enterprise vector stores extend retrieval context for prompts.
- Docs v2 → Docs v3: migration · Documentation shows the transition between API generations.
