How huggingface/aisheets works
Project type and purpose
• This is a full-stack, server-side rendered web application built with Qwik + QwikCity, using an Express adapter.
• It provides a “no-code” AI-driven spreadsheet interface (AI Sheets) where users build, enrich, and transform datasets via Hugging Face and custom LLMs.
• It also includes standalone data-generation scripts for batch dataset enrichment via HF Jobs or vLLM.
File structure
• Top-level adapter: adapters/express (Express server entry point).
• src/routes: QwikCity directory-based routing, including page components and API endpoints (/routes/api, /routes/auth, /routes/oauth2).
• src/components & src/features: UI layer – from pure UI primitives to feature-rich components with business logic.
• src/usecases + src/state: Application/domain logic and state management (Clean Architecture style).
• src/services: Infrastructure services – auth/session, caching, database (DuckDB), repositories (CRUD over DB and Hub), inference (HF providers or custom), websearch scraping & embedding.
• scripts/extend_dataset: Batch scripts to run dataset-enrichment pipelines via HF Jobs or vLLM.
• Configuration & deployment: Dockerfile, .github workflows, pnpm scripts, environment variables.
Key README insights
• Frontend is Qwik + Vite, SSR in dev and prod, Express serves the production build.
• Authentication via HF OAuth or token, inference via HF API or custom endpoints (OpenAI API spec).
• Data dir (./data) holds DuckDB files.
• Advanced config via env vars for default models, concurrency, telemetry, external API keys.
Main components
a. Browser/UI Layer
– Qwik client modules, hydration, interactive UI components (src/components, src/features).
b. Server/SSR Layer
– Qwik SSR runtime invoked by Express (entry.express.tsx → Express routes).
– QwikCity router handling page and API requests.
c. API Layer / Adapter
– Express server with:
• Static asset serving (public/)
• SSR endpoints (SSR middleware)
• API endpoints under /api, /auth, /oauth2
d. Domain & Application Layer
– Use cases (src/usecases) orchestrating business logic.
– State management (src/state).
e. Infrastructure Services
– Auth/session (OAuth2, HF_TOKEN)
– Cache (in-memory or Redis-style abstraction)
– Database (DuckDB via services/db/duckdb.ts)
– Repositories (services/repository) for CRUD on tables, datasets, columns, cells
– Inference service (services/inference) calling:
• Hugging Face Inference Providers API
• Custom LLM endpoint (MODEL_ENDPOINT_URL / NAME)
– Web search (services/websearch) for sourcing prompts or context
f. Persistence
– Local DuckDB files (DATA_DIR)
– Optionally, dataset files cached from Hugging Face Hub
g. External Systems
– Hugging Face Hub for models & datasets
– Custom LLM server (e.g., Ollama)
– Serper web search API
– HF Jobs for batch data generation
h. Offline scripts
– Batch dataset enrichment pipelines under scripts/extend_dataset
Relationships and interactions
• Browser → SSR (HTTP request, initial HTML + data)
• Browser → API endpoints (XHR/fetch to /api/upload, /api/… for file operations)
• SSR → Use cases → Services → Repositories → DuckDB → Services → SSR → Browser
• Inference use case → Inference service → External LLM API → Use case → UI
• Web search use case → Websearch service → External Serper / scraping → Use case → UI
• Scripts → HF Jobs or vLLM endpoints → generate rows → push back to Hub or local DB
Architectural patterns and design principles
• Layered / Onion / Clean Architecture:
– UI → Application (usecases) → Infrastructure (services & repositories) → Persistence
• Adapter pattern: Express adapter for Qwik SSR
• Directory-based routing (QwikCity) for clear separation of pages vs API
• Domain-driven modules: features/ for cohesive UI logic, services/ for infra, usecases/ for business flows
• Configurable via environment variables (12-factor apps)