# firebird-technologies/auto-analyst architecture

> Architecture diagram of the GitHub repository firebird-technologies/auto-analyst, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/firebird-technologies/auto-analyst
- Repository: https://github.com/firebird-technologies/auto-analyst

## Overview

##### Project Type & Purpose
- This is a full-stack, modular AI data-science platform.
- The **backend** (auto-analyst-backend) is a Python/FastAPI (or Flask) service that orchestrates LLM‐based “agents” using DSPy, stores sessions, code, feedback in a SQL database, and runs scheduled reports.
- The **frontend** (auto-analyst-frontend) is a Next.js 13 app with:
- A React UI for chat, code editing, analytics dashboards
- Serverless API routes (Next.js API) handling auth (NextAuth), billing (Stripe), credits & trials, webhooks, and proxying chat requests.
- **Infrastructure** is managed via Terraform, CI pipelines (.github/workflows), and Docker.

##### Top‐Level Containers

A. Frontend Container (Next.js App)
- Client Browser (React UI)
- Next.js Server (Node.js runtime)
- API Routes for auth, subscription, credits, webhooks, analytics hub
- Proxy Module (app/api/chat? calls the Python backend)
- State Stores (Zustand in client / Redis on server for session & credit data)

B. Backend Container (Python Service)
- HTTP Layer (FastAPI or Flask in app.py behind Uvicorn/Gunicorn)
- Agent Orchestrator (DSPy planner, multi-agent workflows)
- Agents Module (preprocessing, statistical, ML, viz)
- Managers & Routes (chat_manager, session_manager, REST endpoints)
- Database Layer (SQLAlchemy models → PostgreSQL or similar)
- Utils (logger, model_registry, report generator)
- Scripts & Schedulers (daily reports, DB init, agent template populators)

C. Data Stores & External Services
- SQL Database (users, sessions, feedback, templates)
- Redis Cache (sessions, credit counts, pub/sub for real-time)
- Object Storage (for uploaded CSVs/XLSX files)
- LLM Providers (OpenAI, Anthropic, Deepseek)
- Stripe API (payments, subscriptions, webhooks)
- OAuth Providers (Google, GitHub via NextAuth)
- Terraform-provisioned Infrastructure (AWS RDS, ElastiCache, S3, Vercel, Hugging Face Spaces)

##### Relationships & Data Flow
- Client → Next.js UI: user interactions (chat, uploads, settings)
- Next.js UI ↔ Next.js Server: REST/Server Components for auth, billing, analytics
- Next.js Server → Stripe/OAuth/Redis: subscription checks, credit updates
- Next.js Server → Python Backend API: chat queries, dataset uploads, agent routing
- Python Backend → LLM Providers: send prompt & code, receive results
- Python Backend → SQL DB: persist sessions, feedback, user data
- Python Backend → Object Storage: store & retrieve datasets
- Python Backend → Redis (optional): shared dataframes, intermediate caches
- Python Backend → Next.js (via API response): returns chat messages, code, charts
- Terraform & CI/CD → Provisioning & Deployment: Vercel for frontend, Docker Image → HF Spaces or Kubernetes for backend

##### Architectural Patterns & Principles
- **Containerized micro-service style**: clear separation of frontend and backend containers
- **Serverless API layer** in frontend: isolates auth/billing logic from UI
- **Modular agent architecture**: each agent is a DSPy Signature class, pluggable
- **Shared Planner Pattern**: central planner orchestrates multi-agent workflows
- **Dataflow pipelines**: user uploads → metadata extraction → agent processing → result

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Container
    subgraph "Auto-Analyst System"
        subgraph "Frontend Container (Next.js App)"
            direction TB
            FB["Client Browser (React UI)"]:::frontend
            FS["Next.js Server (Node.js runtime)"]:::frontend
            subgraph "Serverless API Routes"
                direction TB
                Auth["Auth, Billing, Webhooks, Analytics API Routes"]:::frontend
                ChatProxy["Chat Proxy Module"]:::frontend
            end
            State["State Stores (Zustand / Redis)"]:::frontend
            Static["Static Assets (public/, CSS/Tailwind)"]:::frontend
            Config["Frontend Config & Middleware"]:::frontend
        end

        %% Backend Container
        subgraph "Backend Container (Python Service)"
            direction TB
            HTTP["HTTP Layer (FastAPI/Flask)"]:::backend
            Orchestrator["Agent Orchestrator (DSPy Planner)"]:::backend
            Agents["Agents Module (preprocessing, ML, viz)"]:::backend
            Managers["Managers & Routes"]:::backend
            DBLayer["Database Layer (SQLAlchemy → PostgreSQL)"]:::backend
            Utils["Utility Modules (Logging, Model Registry, Reports)"]:::backend
            Scripts["Scripts & Schedulers"]:::backend
            Docker["Docker Configuration"]:::backend
        end

        %% Data Stores & External Services
        subgraph "Data Stores & External Services"
            direction TB
            SQLDB["SQL Database"]:::datastore
            Redis["Redis Cache"]:::datastore
            Storage["Object Storage (CSV/XLSX)"]:::datastore
            LLM["LLM Providers"]:::external
            Stripe["Stripe API"]:::external
            OAuth["OAuth Providers"]:::external
            Infra["Terraform-provisioned Infra"]:::infra
        end
    end

    %% Connections
    FB -->|"interactions"| FS
    FS -->|"calls auth/billing"| Auth
    FS -->|"proxies chat"| ChatProxy
    Auth -->|"uses"| Stripe
    Auth -->|"uses"| OAuth
    FS -->|"reads/writes"| State
    FS -->|"serves assets"| Static
    FS -->|"loads config"| Config

    ChatProxy -->|"HTTP/REST"| HTTP
    HTTP -->|"orchestrates"| Orchestrator
    Orchestrator -->|"dispatches to"| Agents
    Agents -->|"uses managers"| Managers
    Managers -->|"CRUD"| DBLayer
    HTTP -->|"stores/retrieves"| Storage
    Orchestrator -->|"caches"| Redis
    Orchestrator -->|"queries"| LLM
    Managers -->|"queues tasks"| Scripts
    HTTP -->|"logs"| Utils

    Infra -->|"provisions"| SQLDB
    Infra -->|"provisions"| Redis
    Infra -->|"provisions"| Storage

    %% Click Events - Frontend
    click FB "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/app/"
    click FB "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/components/"
    click FS "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/"
    click Auth "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/app/api/"
    click ChatProxy "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/app/api/chat"
    click State "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/lib/store/"
    click State "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-frontend/lib/redis.ts"
    click Static "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-frontend/public/"
    click Static "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-frontend/app/globals.css"
    click Config "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-frontend/next.config.js"
    click Config "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-frontend/middleware.ts"
    click Config "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-frontend/tailwind.config.js"
    click Config "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-frontend/postcss.config.js"

    %% Click Events - Backend
    click HTTP "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-backend/app.py"
    click Docker "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-backend/Dockerfile"
    click Docker "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-backend/.dockerignore"
    click Docker "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-backend/entrypoint_local.sh"
    click Orchestrator "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-backend/src/agents/"
    click Managers "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-backend/src/managers/"
    click Managers "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-backend/src/routes/"
    click DBLayer "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-backend/src/db/schemas/models.py"
    click DBLayer "https://github.com/firebird-technologies/auto-analyst/blob/main/auto-analyst-backend/src/db/init_db.py"
    click Utils "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-backend/src/utils/"
    click Scripts "https://github.com/firebird-technologies/auto-analyst/tree/main/auto-analyst-backend/scripts/"
    click Infra "https://github.com/firebird-technologies/auto-analyst/tree/main/terraform/"
    click .github "https://github.com/firebird-technologies/auto-analyst/blob/main/ .github/workflows/deploy-to-hf.yml"
    click .github "https://github.com/firebird-technologies/auto-analyst/blob/main/ .github/workflows/frontend-build-check.yml"

    %% Styles
    classDef frontend fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef backend fill:#90EE90,stroke:#333,stroke-width:1px
    classDef datastore fill:#FFFACD,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef infra fill:#FFB6C1,stroke:#333,stroke-width:1px
```
