# frappe/insights architecture

> Architecture diagram of the GitHub repository frappe/insights, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/frappe/insights
- Repository: https://github.com/frappe/insights

## Overview

Based on our review of the file tree and README, Frappe Insights is an open source business intelligence (BI) tool with a full‐stack architecture. It leverages the Frappe Framework on the backend and a Vue-based frontend (with components, dashboards, query builders, chart renderers, etc.) on the client side. It is designed to integrate multiple databases and data sources and to provide a guided user experience for building queries and dashboards.

##### Main Components
- Backend API & Business Logic:
– The “insights” folder contains Python code (using Frappe Framework) that defines API endpoints (e.g., endpoints for dashboards, queries, data sources, user permissions, and setup).
– It includes modules to interact with various databases using connectors (MariaDB, PostgreSQL, DuckDB, BigQuery) and to manage data processing, caching, and multi-source query composition (powered by Ibis).
- Frontend Application:
– The “frontend” directory (and the “frontend/src2” variant) comprises a Vue-based client application. This part includes components for chart building, query building, dashboards (drag-and-drop UI), and notebooks.
– It also includes development tooling (Vite, ESLint, Prettier) and public assets.
- UI Library:
– The “frappe-ui” folder indicates the use of a dedicated UI component library that provides reusable components for the project.
- Deployment and DevOps:
– The “docker” directory contains Docker Compose configurations and initialization scripts, which are used to run the application in production or for development/testing.
- Integration with External Libraries:
– Technologies like Ibis (for SQL query composition), eCharts (for interactive visualizations), and Frappe Framework itself form vital building blocks of the system.

##### Data Sources and External Interactions
- The system supports multiple databases and file data sources.
- The backend interfaces through the Frappe Framework to manage data operations, while the frontend consumes these APIs to render query builder interfaces and visual dashboards.
- The Docker container setup (via docker-compose) illustrates containerized deployment and the separation between the backend (likely “bench” commands) and frontend (development servers or compiled Vue assets).

##### Understand Architectural Patterns and Relationships
- Full-Stack Separation: There is a clear separation between the presentation layer (frontend) and the data/business logic layer (backend running on Frappe).
- API-Driven Communication: The frontend makes HTTP API calls (via endpoints in “insights/api”) to perform actions such as fetching dashboards, running queries, and retrieving chart data.
- Modularity: The well-organized directory structure (for queries, dashboards, notebooks, and settings) demonstrates a modular arrangement likely adhering to MVC or component-driven design principles.
- Use of Docker: Provides a clear infrastructure layer that encapsulates the backend environment and supports easy deployment.

##### Essential Elements
- Annotations: Optionally, include notes for integration flow (e.g., “Query Builder sends JSON payload to backend API”, or “Docker Compose orchestrates multi-container deployment”).

##### Putting It All Together
- A Frontend block that encompasses:
- Vue based application (subdivided into modules like Query Builder, Dashboard, Notebook)
- Frappe UI library components.
- A Backend block depicting:
- Frappe Framework (with its API modules: insights/api/*)
- Business logic for alerts, dashboards, queries, subscriptions, etc.
- Integration components (Ibis, database connectors).
- A Data Layer:
- External databases (MySQL, PostgreSQL, DuckDB, BigQuery)
- Docker Infrastructure Layer:
- Docker Compose controlling the overall deployment (mapping to a container that runs the backend, frontend, etc.)
- External Integrations:
- External libraries like eCharts for data visualization.

##### Final Advice
- Be very detailed. Do not hesitate to break down the system into many individual components for clarity.
- Essential architectural elements (frontend, backend API, database connections, container orchestration, and external libraries).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Deployment and DevOps
    D["Docker Compose Orchestration"]:::deployment

    %% Backend Service
    subgraph "Backend Service"
        B1["Frappe Framework API"]:::backend
        subgraph "API Endpoints"
            B2["Alerts API"]:::backend
            B3["Dashboards API"]:::backend
            B4["Queries API"]:::backend
        end
        B5["Ibis & DB Connectors"]:::backend
    end

    %% Frontend Service
    subgraph "Frontend Service"
        F1["Vue Application"]:::frontend
        F2["Query Builder"]:::frontend
        F3["Dashboard Editor"]:::frontend
        F4["Chart Renderer"]:::frontend
        F5["Notebooks"]:::frontend
        F6["Frappe UI Components"]:::frontend
    end

    %% Data Sources
    subgraph "Data Sources"
        D1["MySQL/MariaDB"]:::datasource
        D2["PostgreSQL"]:::datasource
        D3["DuckDB"]:::datasource
        D4["BigQuery"]:::datasource
    end

    %% External Integration
    E1["eCharts (Visualization)"]:::external

    %% Relationships
    D -->|"orchestrates"| B1
    D -->|"orchestrates"| F1

    F1 -->|"HTTPcalls"| B1

    %% Linking API endpoints as sub-services of the main API
    B1 -->|"includes"| B2
    B1 -->|"includes"| B3
    B1 -->|"includes"| B4

    %% Backend to Data Sources relationships via Ibis connectors
    B5 -->|"connects"| D1
    B5 -->|"connects"| D2
    B5 -->|"connects"| D3
    B5 -->|"connects"| D4

    %% Frontend module interactions
    F1 -->|"composes"| F2
    F1 -->|"composes"| F3
    F1 -->|"composes"| F4
    F1 -->|"composes"| F5
    F1 -->|"integrates"| F6

    %% External library integration with Chart Renderer
    F4 -->|"rendersusing"| E1

    %% Click Events
    click D "https://github.com/frappe/insights/tree/develop/docker"
    click B1 "https://github.com/frappe/insights/tree/develop/insights/api"
    click F1 "https://github.com/frappe/insights/tree/develop/frontend"
    click F6 "https://github.com/frappe/insights/tree/develop/frappe-ui"

    %% Styles
    classDef deployment fill:#fef2cc,stroke:#e2b007,stroke-width:2px;
    classDef backend fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef frontend fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef datasource fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef external fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
```
