# huggingface/aisheets architecture

> Architecture diagram of the GitHub repository huggingface/aisheets, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huggingface/aisheets
- Repository: https://github.com/huggingface/aisheets

## Overview

##### Project type and purpose
- This is a full-stack, server-side rendered web application built with Qwik + QwikCity, using an Express adapter.
- It provides a “no-code” AI-driven spreadsheet interface (AI Sheets) where users build, enrich, and transform datasets via Hugging Face and custom LLMs.
- It also includes standalone data-generation scripts for batch dataset enrichment via HF Jobs or vLLM.

##### File structure
- Top-level adapter: adapters/express (Express server entry point).
- src/routes: QwikCity directory-based routing, including page components and API endpoints (/routes/api, /routes/auth, /routes/oauth2).
- src/components & src/features: UI layer – from pure UI primitives to feature-rich components with business logic.
- src/usecases + src/state: Application/domain logic and state management (Clean Architecture style).
- src/services: Infrastructure services – auth/session, caching, database (DuckDB), repositories (CRUD over DB and Hub), inference (HF providers or custom), websearch scraping & embedding.
- scripts/extend_dataset: Batch scripts to run dataset-enrichment pipelines via HF Jobs or vLLM.
- Configuration & deployment: Dockerfile, .github workflows, pnpm scripts, environment variables.

##### Key README insights
- Frontend is Qwik + Vite, SSR in dev and prod, Express serves the production build.
- Authentication via HF OAuth or token, inference via HF API or custom endpoints (OpenAI API spec).
- Data dir (./data) holds DuckDB files.
- Advanced config via env vars for default models, concurrency, telemetry, external API keys.

##### Main components
a. Browser/UI Layer
– Qwik client modules, hydration, interactive UI components (src/components, src/features).
b. Server/SSR Layer
– Qwik SSR runtime invoked by Express (entry.express.tsx → Express routes).
– QwikCity router handling page and API requests.
c. API Layer / Adapter
– Express server with:
- Static asset serving (public/)
- SSR endpoints (SSR middleware)
- API endpoints under /api, /auth, /oauth2
d. Domain & Application Layer
– Use cases (src/usecases) orchestrating business logic.
– State management (src/state).
e. Infrastructure Services
– Auth/session (OAuth2, HF_TOKEN)
– Cache (in-memory or Redis-style abstraction)
– Database (DuckDB via services/db/duckdb.ts)
– Repositories (services/repository) for CRUD on tables, datasets, columns, cells
– Inference service (services/inference) calling:
- Hugging Face Inference Providers API
- Custom LLM endpoint (MODEL_ENDPOINT_URL / NAME)
– Web search (services/websearch) for sourcing prompts or context
f. Persistence
– Local DuckDB files (DATA_DIR)
– Optionally, dataset files cached from Hugging Face Hub
g. External Systems
– Hugging Face Hub for models & datasets
– Custom LLM server (e.g., Ollama)
– Serper web search API
– HF Jobs for batch data generation
h. Offline scripts
– Batch dataset enrichment pipelines under scripts/extend_dataset

##### Relationships and interactions
- Browser → SSR (HTTP request, initial HTML + data)
- Browser → API endpoints (XHR/fetch to /api/upload, /api/… for file operations)
- SSR → Use cases → Services → Repositories → DuckDB → Services → SSR → Browser
- Inference use case → Inference service → External LLM API → Use case → UI
- Web search use case → Websearch service → External Serper / scraping → Use case → UI
- Scripts → HF Jobs or vLLM endpoints → generate rows → push back to Hub or local DB

##### Architectural patterns and design principles
- Layered / Onion / Clean Architecture:
– UI → Application (usecases) → Infrastructure (services & repositories) → Persistence
- Adapter pattern: Express adapter for Qwik SSR
- Directory-based routing (QwikCity) for clear separation of pages vs API
- Domain-driven modules: features/ for cohesive UI logic, services/ for infra, usecases/ for business flows
- Configurable via environment variables (12-factor apps)

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% Browser/UI Layer
  subgraph "Browser" 
    direction TB
    Browser["User Browser"]:::external
    UIComponents["UI Components (src/components)"]:::ui
    Features["Feature Components (src/features)"]:::ui
  end

  %% Server Process
  subgraph "Server Process" 
    direction TB
    ExpressAdapter["Express Adapter (adapters/express)"]:::server
    EntryExpress["Entry Express (src/entry.express.tsx)"]:::server
    EntrySSR["Entry SSR (src/entry.ssr.tsx)"]:::server
    subgraph "QwikCity Routing"
      direction TB
      Routes["Routes Directory (src/routes)"]:::server
      ApiRoutes["API Endpoints (src/routes/api)"]:::server
      AuthRoutes["Auth Endpoints (src/routes/auth)"]:::server
      OauthRoutes["OAuth2 Endpoints (src/routes/oauth2)"]:::server
    end
  end

  %% Domain & Application Layer
  subgraph "Domain & Application" 
    direction TB
    Usecases["Use Cases (src/usecases)"]:::domain
    State["State Management (src/state)"]:::domain
  end

  %% Infrastructure Services
  subgraph "Infrastructure Services"
    direction TB
    AuthService["Auth/Session Service (src/services/auth/session.ts)"]:::infra
    CacheService["Cache Service (src/services/cache/index.ts)"]:::infra
    subgraph "Database Service"
      direction TB
      DuckDB1["DuckDB Connector (src/services/db/duckdb.ts)"]:::infra
      DuckDB2["DB Utils (src/services/db/db.ts)"]:::infra
      Models["DB Models (src/services/db/models)"]:::infra
    end
    Repos["Repositories (src/services/repository)"]:::infra
    Inference["Inference Service (src/services/inference)"]:::infra
    Websearch["WebSearch Service (src/services/websearch)"]:::infra
    ViteConfig["Vite Config (adapters/express/vite.config.ts)"]:::server
  end

  %% Persistence
  subgraph "Persistence"
    direction TB
    DataDir["DuckDB File Store (./data)"]:::persistence
  end

  %% External Systems
  subgraph "External Systems"
    direction TB
    HFHub["Hugging Face Hub"]:::external
    CustomLLM["Custom LLM Endpoint"]:::external
    Serper["Serper Web Search API"]:::external
    HFJobs["HF Jobs"]:::external
    VLLM["vLLM Endpoint"]:::external
  end

  %% Offline Scripts
  subgraph "Batch Scripts"
    direction TB
    Scripts["Enrichment Scripts (scripts/extend_dataset)"]:::script
    ScriptInf["with_inference_client.py"]:::script
    ScriptVLLM["with_vllm.py"]:::script
  end

  %% Main Flows
  Browser -->|"HTTP Request"| ExpressAdapter
  ExpressAdapter --> EntryExpress
  EntryExpress --> EntrySSR
  EntrySSR --> Routes
  Routes --> ApiRoutes
  Routes --> AuthRoutes
  Routes --> OauthRoutes
  ApiRoutes --> Usecases
  AuthRoutes --> Usecases
  OauthRoutes --> Usecases
  Usecases --> State
  Usecases --> AuthService
  Usecases --> CacheService
  Usecases --> Repos
  Repos --> DuckDB1
  DuckDB1 --> DuckDB2
  DuckDB2 --> Models
  Usecases --> Inference
  Usecases --> Websearch
  Usecases -->|reads/writes| DataDir
  Inference -.->|"Inference API Calls"| HFHub
  Inference -.->|"Custom LLM API"| CustomLLM
  Websearch -.->|"Search API"| Serper

  %% Batch Script Flows
  Scripts --> ScriptInf
  Scripts --> ScriptVLLM
  ScriptInf -.->|"HF Jobs"| HFJobs
  ScriptVLLM -.->|"vLLM"| VLLM
  Scripts --> DataDir

  %% UI Interaction
  Browser --> UIComponents
  UIComponents --> Features
  Features -->|XHR/Fetch| ApiRoutes

  %% Click Events
  click UIComponents "https://github.com/huggingface/aisheets/tree/main/src/components"
  click Features "https://github.com/huggingface/aisheets/tree/main/src/features"
  click ExpressAdapter "https://github.com/huggingface/aisheets/tree/main/adapters/express"
  click EntryExpress "https://github.com/huggingface/aisheets/blob/main/src/entry.express.tsx"
  click EntrySSR "https://github.com/huggingface/aisheets/blob/main/src/entry.ssr.tsx"
  click Routes "https://github.com/huggingface/aisheets/tree/main/src/routes"
  click ApiRoutes "https://github.com/huggingface/aisheets/tree/main/src/routes/api"
  click AuthRoutes "https://github.com/huggingface/aisheets/tree/main/src/routes/auth"
  click OauthRoutes "https://github.com/huggingface/aisheets/tree/main/src/routes/oauth2"
  click ViteConfig "https://github.com/huggingface/aisheets/blob/main/adapters/express/vite.config.ts"
  click Usecases "https://github.com/huggingface/aisheets/tree/main/src/usecases"
  click State "https://github.com/huggingface/aisheets/tree/main/src/state"
  click AuthService "https://github.com/huggingface/aisheets/blob/main/src/services/auth/session.ts"
  click CacheService "https://github.com/huggingface/aisheets/blob/main/src/services/cache/index.ts"
  click DuckDB1 "https://github.com/huggingface/aisheets/blob/main/src/services/db/duckdb.ts"
  click DuckDB2 "https://github.com/huggingface/aisheets/blob/main/src/services/db/db.ts"
  click Models "https://github.com/huggingface/aisheets/tree/main/src/services/db/models"
  click Repos "https://github.com/huggingface/aisheets/tree/main/src/services/repository"
  click Inference "https://github.com/huggingface/aisheets/tree/main/src/services/inference"
  click Websearch "https://github.com/huggingface/aisheets/tree/main/src/services/websearch"
  click DataDir "https://github.com/huggingface/aisheets/tree/main/data"
  click Scripts "https://github.com/huggingface/aisheets/tree/main/scripts/extend_dataset"
  click ScriptInf "https://github.com/huggingface/aisheets/blob/main/scripts/extend_dataset/with_inference_client.py"
  click ScriptVLLM "https://github.com/huggingface/aisheets/blob/main/scripts/extend_dataset/with_vllm.py"

  %% Styles
  classDef ui fill:#D0E8FF,stroke:#0366D6;
  classDef server fill:#E0E0E0,stroke:#888888;
  classDef domain fill:#FFE0B2,stroke:#E65100;
  classDef infra fill:#E0F2F1,stroke:#00796B;
  classDef persistence fill:#F3E5F5,stroke:#8E24AA;
  classDef external fill:#FFEBEE,stroke:#C62828,stroke-dasharray: 5 5;
  classDef script fill:#FFF3E0,stroke:#FB8C00;
```
