# lilysu/schemata architecture

> Architecture diagram of the GitHub repository lilysu/schemata, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lilysu/schemata
- Repository: https://github.com/lilysu/schemata

## Overview

1. **Overall shape**
   - This is a hybrid monorepo centered on a Next.js 16 frontend, with an optional FastAPI backend in `backend/`.
   - The product’s core function is to take a GitHub repository URL/owner-repo pair, fetch repository structure, generate a diagram, and let users drill into subdirectories recursively.

2. **Frontend application boundary**
   - The main user-facing app lives under `src/app/` and `src/components/`.
   - Next.js route handlers under `src/app/api/...` can serve generation, cost, health, and prefetch requests.
   - The frontend can also be configured to call the external FastAPI backend instead of its local handlers via environment flags, which makes the architecture split between “embedded backend” and “legacy/external backend” modes.

3. **Diagram generation pipeline**
   - The diagram pipeline is intentionally hybrid:
     - deterministic repo analysis first,
     - then one LLM streaming pass for edges/relationships.
   - The deterministic layer lives in `src/server/generate/analyzer.ts` and produces nodes, groups, and inferred structure immediately from the repo file tree.
   - The LLM layer then adds semantic relationships and missing components, which reduces token usage and avoids multi-stage generation.

4. **Streaming request flow**
   - The main generation path is exposed by `src/app/api/generate/stream/route.ts` on the frontend side and `backend/app/routers/generate.py` on the FastAPI side.
   - The UI consumes streamed output through the diagram hooks in `src/hooks/diagram/` and `src/hooks/useDiagram.ts`.
   - This means the diagram can render progressively: nodes appear first, then edges and explanation stream in.

5. **Recursive background prefetch**
   - A distinctive subsystem is the prefetch pipeline in `src/app/api/generate/prefetch/route.ts`.
   - After a diagram loads, the frontend identifies directory nodes and pre-generates child diagrams in the background.
   - This recursion continues down the tree, so drill-down views are often already cached before the user clicks them.

6. **Caching and persistence**
   - Generated diagrams are cached in Supabase PostgreSQL via Drizzle ORM, with schema in `src/server/db/schema.ts`.
   - The README describes two main caches: one for top-level repo diagrams and one for sub-diagrams keyed by directory/file scope.
   - This cache is a major architectural boundary: once populated, it eliminates repeat LLM calls for the same repo scope.

7. **GitHub data access**
   - Repository structure and README content are fetched through GitHub-specific services in `src/server/generate/github.ts` and mirrored in backend `backend/app/services/github_service.py`.
   - GitHub authentication can use a PAT or GitHub App credentials, which matters because rate limits and access to private repos affect the fetch layer.
   - This layer is upstream of both deterministic analysis and LLM generation.

8. **AI provider abstraction**
   - The model/provider layer is centralized in `src/server/generate/provider.ts`, `model-config.ts`, and `prompts.ts`, with parallel backend equivalents under `backend/app/services/`.
   - The app supports direct provider calls to OpenAI, Anthropic, or Google, and optionally routes through Vercel AI Gateway.
   - This abstraction matters because the rest of the app treats the model as pluggable, while prompts and fallback model selection remain centralized.

9. **UI rendering and interaction**
   - Diagram visualization is handled by React Flow in `src/components/flow-diagram.tsx` and the node components under `src/components/flow-nodes/`.
   - The diagram is not just static output: nodes are interactive, clickable, and support drill-down navigation into directories and source files.
   - The UI layer also includes export, copy, breadcrumbs, dialogs, theme controls, and repo-specific actions that wrap the core graph.

10. **Domain logic in hooks and feature modules**
   - Diagram-specific domain logic is concentrated in `src/features/diagram/` and `src/hooks/diagram/`.
   - These modules handle GitHub URL parsing, SSE/event streaming, export formatting, diagram state transitions, and drill-down behavior.
   - This keeps the UI components relatively presentational and pushes orchestration into reusable client-side logic.

11. **Backend FastAPI service**
   - The Python backend mirrors the core generation responsibilities: routing, GitHub access, prompt definitions, model configuration, pricing, and observability.
   - It includes its own tests under `backend/tests/`, Nginx setup, and Docker entrypoints, suggesting it is deployable as a standalone API service.
   - In practice, this backend acts as the production-grade or legacy service path, while the Next.js app can also operate with local route handlers.

12. **Operational and deployment boundaries**
   - The repo includes GitHub Actions for CI and deployment, plus Docker Compose for local full-stack runs.
   - Supabase is the persistence layer, GitHub is the source-of-truth data source, and LLM providers are the external compute layer.
   - The architectural picture is therefore: browser UI → Next.js/FastAPI API → GitHub + LLM provider → Supabase cache → streamed diagram back to the client, with recursive prefetch expanding the cached graph in the background.

Main entry points: [`src/app/api/generate/stream/route.ts`](https://github.com/lilysu/schemata/blob/main/src/app/api/generate/stream/route.ts) (Stream API), [`backend/app/routers/generate.py`](https://github.com/lilysu/schemata/blob/main/backend/app/routers/generate.py) (Generate Route), [`src/app/[username]/[repo]/repo-page-client.tsx`](https://github.com/lilysu/schemata/blob/main/src/app/[username]/[repo]/repo-page-client.tsx) (Repo Client), [`src/hooks/diagram/useDiagramStream.ts`](https://github.com/lilysu/schemata/blob/main/src/hooks/diagram/useDiagramStream.ts) (Diagram Hooks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Next.js App"]
  node_landing["Landing<br/>pages<br/>[page.tsx]"]
  node_repo_page["Repo Page<br/>route<br/>[page.tsx]"]
  node_repo_client["Repo Client<br/>client page"]
  node_api_stream["Stream API<br/>route handler<br/>[route.ts]"]
  node_api_prefetch["Prefetch API<br/>route handler<br/>[route.ts]"]
  node_api_cost["Cost API<br/>route handler<br/>[route.ts]"]
  node_diagram_hooks["Diagram Hooks<br/>client orchestration"]
  node_use_diagram["Diagram State<br/>hook<br/>[useDiagram.ts]"]
  node_flow_diagram["Flow Diagram<br/>ui renderer<br/>[flow-diagram.tsx]"]
  node_flow_nodes["Node Views<br/>ui nodes<br/>[index.ts]"]
end

subgraph group_core["Shared Core"]
  node_diagram_api["Diagram API<br/>client API<br/>[api.ts]"]
  node_diagram_sse["SSE Parser<br/>stream utils<br/>[sse.ts]"]
  node_analyzer["Analyzer<br/>deterministic analysis<br/>[analyzer.ts]"]
  node_github_fetch["GitHub Fetch<br/>source access<br/>[github.ts]"]
  node_llm_provider["Model Layer<br/>provider abstraction<br/>[provider.ts]"]
  node_db_schema[("Cache Schema<br/>persistence schema<br/>[schema.ts]")]
end

subgraph group_backend["FastAPI Service"]
  node_backend_main["Backend App<br/>fastapi entry<br/>[main.py]"]
  node_backend_route["Generate Route<br/>api router<br/>[generate.py]"]
  node_backend_github["GitHub Service<br/>[github_service.py]"]
  node_backend_models["Model Config<br/>[model_config.py]"]
end

node_landing -->|"open repo"| node_repo_page
node_repo_page -->|"hydrate"| node_repo_client
node_repo_client -->|"init state"| node_use_diagram
node_use_diagram -->|"compose"| node_diagram_hooks
node_diagram_hooks -->|"request"| node_diagram_api
node_diagram_hooks -->|"parse stream"| node_diagram_sse
node_repo_client -->|"render"| node_flow_diagram
node_flow_diagram -->|"node types"| node_flow_nodes
node_api_stream -->|"analyze"| node_analyzer
node_api_stream -->|"fetch repo"| node_github_fetch
node_api_stream -->|"stream llm"| node_llm_provider
node_api_stream -->|"cache"| node_db_schema
node_api_prefetch -.->|"enqueue"| node_api_stream
node_api_cost -->|"estimate"| node_llm_provider
node_api_stream -->|"serve"| node_diagram_api
node_diagram_api -->|"events"| node_diagram_sse
node_api_stream -.->|"alt path"| node_backend_route
node_backend_main -->|"mount"| node_backend_route
node_backend_route -->|"fetch repo"| node_backend_github
node_backend_route -->|"model ops"| node_backend_models
node_backend_route -.->|"cache"| node_db_schema
node_db_schema -.->|"hydrate cache"| node_repo_client

click node_landing "https://github.com/lilysu/schemata/blob/main/src/app/page.tsx"
click node_repo_page "https://github.com/lilysu/schemata/blob/main/src/app/[username]/[repo]/page.tsx"
click node_repo_client "https://github.com/lilysu/schemata/blob/main/src/app/[username]/[repo]/repo-page-client.tsx"
click node_api_stream "https://github.com/lilysu/schemata/blob/main/src/app/api/generate/stream/route.ts"
click node_api_prefetch "https://github.com/lilysu/schemata/blob/main/src/app/api/generate/prefetch/route.ts"
click node_api_cost "https://github.com/lilysu/schemata/blob/main/src/app/api/generate/cost/route.ts"
click node_diagram_hooks "https://github.com/lilysu/schemata/blob/main/src/hooks/diagram/useDiagramStream.ts"
click node_use_diagram "https://github.com/lilysu/schemata/blob/main/src/hooks/useDiagram.ts"
click node_flow_diagram "https://github.com/lilysu/schemata/blob/main/src/components/flow-diagram.tsx"
click node_flow_nodes "https://github.com/lilysu/schemata/blob/main/src/components/flow-nodes/index.ts"
click node_diagram_api "https://github.com/lilysu/schemata/blob/main/src/features/diagram/api.ts"
click node_diagram_sse "https://github.com/lilysu/schemata/blob/main/src/features/diagram/sse.ts"
click node_analyzer "https://github.com/lilysu/schemata/blob/main/src/server/generate/analyzer.ts"
click node_github_fetch "https://github.com/lilysu/schemata/blob/main/src/server/generate/github.ts"
click node_llm_provider "https://github.com/lilysu/schemata/blob/main/src/server/generate/provider.ts"
click node_db_schema "https://github.com/lilysu/schemata/blob/main/src/server/db/schema.ts"
click node_backend_main "https://github.com/lilysu/schemata/blob/main/backend/app/main.py"
click node_backend_route "https://github.com/lilysu/schemata/blob/main/backend/app/routers/generate.py"
click node_backend_github "https://github.com/lilysu/schemata/blob/main/backend/app/services/github_service.py"
click node_backend_models "https://github.com/lilysu/schemata/blob/main/backend/app/services/model_config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_landing,node_repo_page,node_repo_client,node_api_stream,node_api_prefetch,node_api_cost,node_diagram_hooks,node_use_diagram,node_flow_diagram,node_flow_nodes toneBlue
class node_diagram_api,node_diagram_sse,node_analyzer,node_github_fetch,node_llm_provider,node_db_schema toneAmber
class node_backend_main,node_backend_route,node_backend_github,node_backend_models toneMint
```

## Components

### Next.js App

Browser-facing app, route handlers, UI, and client-side diagram orchestration.

- [Landing](https://github.com/lilysu/schemata/blob/main/src/app/page.tsx): pages · Home entry and top-level app shell. · `src/app/page.tsx`
- [Repo Page](https://github.com/lilysu/schemata/blob/main/src/app/[username]/[repo]/page.tsx): route · Owner/repo page that hosts the interactive diagram experience. · `src/app/[username]/[repo]/page.tsx`
- [Repo Client](https://github.com/lilysu/schemata/blob/main/src/app/[username]/[repo]/repo-page-client.tsx): client page · Client-side page wrapper for repo loading and interaction state. · `src/app/[username]/[repo]/repo-page-client.tsx`
- [Stream API](https://github.com/lilysu/schemata/blob/main/src/app/api/generate/stream/route.ts): route handler · Main streaming generation endpoint for diagrams. · `src/app/api/generate/stream/route.ts`
- [Prefetch API](https://github.com/lilysu/schemata/blob/main/src/app/api/generate/prefetch/route.ts): route handler · Background recursive pre-generation of child diagrams. · `src/app/api/generate/prefetch/route.ts`
- [Cost API](https://github.com/lilysu/schemata/blob/main/src/app/api/generate/cost/route.ts): route handler · Cost estimation and usage-related generation endpoint. · `src/app/api/generate/cost/route.ts`
- [Diagram Hooks](https://github.com/lilysu/schemata/blob/main/src/hooks/diagram/useDiagramStream.ts): client orchestration · State, streaming, drill-down, export, and diagram lifecycle hooks. · `src/hooks/diagram/useDiagramStream.ts`
- [Diagram State](https://github.com/lilysu/schemata/blob/main/src/hooks/useDiagram.ts): hook · Top-level diagram state coordinator used by the page. · `src/hooks/useDiagram.ts`
- [Flow Diagram](https://github.com/lilysu/schemata/blob/main/src/components/flow-diagram.tsx): ui renderer · React Flow diagram canvas and interaction surface. · `src/components/flow-diagram.tsx`
- [Node Views](https://github.com/lilysu/schemata/blob/main/src/components/flow-nodes/index.ts): ui nodes · Specialized rendered node types for files, services, APIs, and data. · `src/components/flow-nodes/index.ts`

### Shared Core

Repository analysis, streaming, GitHub access, provider abstraction, and caching used by the app.

- [Diagram API](https://github.com/lilysu/schemata/blob/main/src/features/diagram/api.ts): client API · Client helpers for generation, exports, and stream/event transport. · `src/features/diagram/api.ts`
- [SSE Parser](https://github.com/lilysu/schemata/blob/main/src/features/diagram/sse.ts): stream utils · Stream event parsing and incremental diagram update helpers. · `src/features/diagram/sse.ts`
- [Analyzer](https://github.com/lilysu/schemata/blob/main/src/server/generate/analyzer.ts): deterministic analysis · Repo-structure analysis that produces initial nodes, groups, and inferred structure. · `src/server/generate/analyzer.ts`
- [GitHub Fetch](https://github.com/lilysu/schemata/blob/main/src/server/generate/github.ts): source access · Fetches repository structure, metadata, and README content from GitHub. · `src/server/generate/github.ts`
- [Model Layer](https://github.com/lilysu/schemata/blob/main/src/server/generate/provider.ts): provider abstraction · Centralized model selection, prompts, provider routing, and fallback behavior. · `src/server/generate/provider.ts`
- [Cache Schema](https://github.com/lilysu/schemata/blob/main/src/server/db/schema.ts): persistence schema · Supabase/Drizzle schema for cached top-level and scoped sub-diagrams. · `src/server/db/schema.ts`

### FastAPI Service

Standalone Python API path that mirrors generation and infrastructure responsibilities.

- [Backend App](https://github.com/lilysu/schemata/blob/main/backend/app/main.py): fastapi entry · Standalone Python API application entrypoint. · `backend/app/main.py`
- [Generate Route](https://github.com/lilysu/schemata/blob/main/backend/app/routers/generate.py): api router · FastAPI generation endpoint mirroring the frontend stream path. · `backend/app/routers/generate.py`
- [GitHub Service](https://github.com/lilysu/schemata/blob/main/backend/app/services/github_service.py): service · Python GitHub access layer for repository fetching. · `backend/app/services/github_service.py`
- [Model Config](https://github.com/lilysu/schemata/blob/main/backend/app/services/model_config.py): service · Backend model/provider configuration and prompt wiring. · `backend/app/services/model_config.py`

## Connections

- Landing → Repo Page: open repo · Users navigate from the landing page into a repository view.
- Repo Page → Repo Client: hydrate · The route renders a client wrapper that manages repo interaction.
- Repo Client → Diagram State: init state · The page initializes diagram state and lifecycle from the hook.
- Diagram State → Diagram Hooks: compose · Top-level hook composes stream, export, and drill-down behavior.
- Diagram Hooks → Diagram API: request · Client orchestration calls the diagram API helpers.
- Diagram Hooks → SSE Parser: parse stream · Streaming events are normalized into incremental diagram updates.
- Repo Client → Flow Diagram: render · The interactive diagram canvas renders the current graph state.
- Flow Diagram → Node Views: node types · The canvas uses specialized node renderers for semantic display.
- Stream API → Analyzer: analyze · The streaming route runs deterministic repo analysis first.
- Stream API → GitHub Fetch: fetch repo · Repository structure and README content are fetched upstream.
- Stream API → Model Layer: stream llm · Semantic relationships and missing components are added by the model layer.
- Stream API → Cache Schema: cache · Generated diagrams are persisted for reuse by repo scope.
- Prefetch API → Stream API: enqueue · Background recursion reuses the main generator for child scopes.
- Cost API → Model Layer: estimate · Cost logic relies on model/provider pricing and selection.
- Stream API → Diagram API: serve · The frontend consumes the generation endpoints through client helpers.
- Diagram API → SSE Parser: events · Helpers wrap the streamed SSE/event protocol used by the UI.
- Stream API → Generate Route: alt path · When configured for external backend mode, requests go to FastAPI.
- Backend App → Generate Route: mount · The backend application exposes the generation router.
- Generate Route → GitHub Service: fetch repo · Backend generation also depends on GitHub access services.
- Generate Route → Model Config: model ops · Backend generation uses its own model and prompt configuration.
- Generate Route → Cache Schema: cache · Backend and frontend share the same diagram caching boundary.
- Cache Schema → Repo Client: hydrate cache · Cached diagrams reduce repeat generation and speed drill-down views.