# simonw/datasette architecture

> Architecture diagram of the GitHub repository simonw/datasette, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/simonw/datasette
- Repository: https://github.com/simonw/datasette

## Overview

##### Project Type and Purpose
- Datasette is an open-source, Python-based full-stack data publishing tool.
- It provides a CLI for ingesting SQLite databases and a web server (ASGI app) exposing an interactive UI and JSON API for exploring those databases.
- It also includes “publish” subcommands that build and deploy Docker images to services like Heroku and Cloud Run.

##### File-Structure Analysis
- Top-level executables and metadata:
- Dockerfile, Justfile (build/dev tasks), setup.py, README.md, GitHub workflows.
- Core package `datasette/`:
- `__main__.py` & `cli.py` – entry point and subcommand orchestration.
- `app.py` – constructs the ASGI application and mounts static assets.
- `views/` and `templates/` – URL handlers and Jinja2 templates for HTML rendering.
- `database.py` and `utils/sqlite.py` – manage SQLite connections and queries.
- `plugins.py` & `hookspecs.py` – plugin discovery and hook definitions.
- `publish/` – modules to build and push Docker images to cloud platforms.
- Static assets in `datasette/static/` and plugin/demo code in `demos/`.
- Tests, docs, CI configuration underscore a modular, test-driven, plugin-centric architecture.

##### README Insights
- Confirms ASGI app with CLI interface (“serve”, “publish”).
- Highlights support for metadata.json, plugin hooks, and multi-platform deployment.
- Emphasizes interactive web UI + JSON API, plus cloud publishing.

a. Main Components
1. User Interface Layer
– Web browser or API client.
2. Ingress/Hosting Layer
– HTTP server/ASGI server (e.g., uvicorn, Hypercorn).
3. Core Application Layer
– `datasette/app.py`: ASGI app builder.
– Middleware (tracing, events).
– Router → view handlers (`datasette/views/*`).
4. Data Access Layer
– `datasette/database.py` + SQLite driver → SQLite file(s).
5. Rendering Layer
– JSON renderers, HTML templates (Jinja2), CSV export.
6. Plugin System
– Hook manager, hook specifications, plugin discovery.
7. CLI & Deployment Subsystem
– `datasette/cli.py` + `datasette/publish/*`.
– Dockerfile, build scripts, push to Heroku/Cloud Run.
8. External Services
– Cloud platforms (Heroku, Cloud Run), Docker registry.

b. Relationships and Data Flows
- Browser/API client → HTTP request → ASGI server → Datasette app.
- Request passes through middleware → dispatched to view → database layer executes SQL → returns results.
- View hands data to renderer → HTTP response back to client.
- Plugins can intercept at events hooks (startup, before-render, after-query).
- CLI “publish” command invokes Docker build context → image pushed to external registry → deployed on cloud.

c. Architectural Patterns and Principles
- Layered architecture (Ingress → Core App → Data → Rendering).
- Plugin-hook architecture for extensibility.
- Declarative routing + MVC-like separation of views/templates.
- ASGI standard for concurrency and middleware.
- Docker-centric deployment pipeline.

d. Key Technologies and Frameworks
- Python 3.8+, ASGI (Starlette-style), Jinja2, SQLite.
- Docker, Heroku, Google Cloud Run.
- GitHub Actions for CI, pytest for testing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    Browser["Web Browser / API Client"]:::frontend
    Browser -->|"HTTP Request"| ASGIServer

    %% Ingress Layer
    ASGIServer["ASGI Server (uvicorn, Hypercorn)"]:::frontend
    ASGIServer -->|"mount static"| StaticAssets
    ASGIServer -->|"invoke app builder"| AppBuilder

    %% Core Application Layer
    subgraph "Core Application Layer"
        AppBuilder["datasette/app.py"]:::core
        click AppBuilder "https://github.com/simonw/datasette/blob/main/datasette/app.py"

        subgraph Middleware
            Events["datasette/events.py"]:::core
            click Events "https://github.com/simonw/datasette/blob/main/datasette/events.py"
            Tracer["datasette/tracer.py"]:::core
            click Tracer "https://github.com/simonw/datasette/blob/main/datasette/tracer.py"
            ExceptionHandler["datasette/handle_exception.py"]:::core
            click ExceptionHandler "https://github.com/simonw/datasette/blob/main/datasette/handle_exception.py"
        end

        AppBuilder --> Middleware
        Middleware --> Router

        subgraph Routing & Views
            Router["Router"]:::core
            BaseView["base.py"]:::core
            click BaseView "https://github.com/simonw/datasette/blob/main/datasette/views/base.py"
            IndexView["index.py"]:::core
            click IndexView "https://github.com/simonw/datasette/blob/main/datasette/views/index.py"
            DatabaseView["database.py"]:::core
            click DatabaseView "https://github.com/simonw/datasette/blob/main/datasette/views/database.py"
            TableView["table.py"]:::core
            click TableView "https://github.com/simonw/datasette/blob/main/datasette/views/table.py"
            RowView["row.py"]:::core
            click RowView "https://github.com/simonw/datasette/blob/main/datasette/views/row.py"
            SpecialView["special.py"]:::core
            click SpecialView "https://github.com/simonw/datasette/blob/main/datasette/views/special.py"
            Router --> BaseView
            Router --> IndexView
            Router --> DatabaseView
            Router --> TableView
            Router --> RowView
            Router --> SpecialView
        end
    end

    %% Plugin System
    subgraph "Plugin System"
        Plugins["datasette/plugins.py"]:::plugin
        click Plugins "https://github.com/simonw/datasette/blob/main/datasette/plugins.py"
        HookSpecs["datasette/hookspecs.py"]:::plugin
        click HookSpecs "https://github.com/simonw/datasette/blob/main/datasette/hookspecs.py"
    end
    Middleware -.->|"hook events"| Plugins
    BaseView -.->|"before-render hook"| Plugins
    DatabaseView -.->|"after-query hook"| Plugins

    %% Data Access Layer
    subgraph "Data Access Layer"
        DatabaseModule["datasette/database.py"]:::data
        click DatabaseModule "https://github.com/simonw/datasette/blob/main/datasette/database.py"
        SQLiteUtil["datasette/utils/sqlite.py"]:::data
        click SQLiteUtil "https://github.com/simonw/datasette/blob/main/datasette/utils/sqlite.py"
        DatabaseModule --> SQLiteUtil
    end
    Router --> DatabaseModule

    %% Rendering Layer
    subgraph "Rendering Layer"
        JSONRenderer["datasette/renderer.py"]:::render
        click JSONRenderer "https://github.com/simonw/datasette/blob/main/datasette/renderer.py"
        BlobRenderer["datasette/blob_renderer.py"]:::render
        click BlobRenderer "https://github.com/simonw/datasette/blob/main/datasette/blob_renderer.py"
        Templates["datasette/templates/*.html"]:::render
        click Templates "https://github.com/simonw/datasette/tree/main/datasette/templates/"
        StaticAssets["datasette/static/*.js & app.css"]:::render
        click StaticAssets "https://github.com/simonw/datasette/tree/main/datasette/static/"
        BaseView --> Templates
        BaseView --> JSONRenderer
        BaseView --> BlobRenderer
    end

    %% CLI & Deployment Subsystem
    subgraph "CLI & Publish Subsystem"
        CLIEntry["datasette/__main__.py & datasette/cli.py"]:::cli
        click CLIEntry "https://github.com/simonw/datasette/blob/main/datasette/__main__.py"
        click CLIEntry "https://github.com/simonw/datasette/blob/main/datasette/cli.py"
        subgraph "Publish Commands"
            PublishCommon["datasette/publish/common.py"]:::cli
            click PublishCommon "https://github.com/simonw/datasette/blob/main/datasette/publish/common.py"
            HerokuPub["datasette/publish/heroku.py"]:::cli
            click HerokuPub "https://github.com/simonw/datasette/blob/main/datasette/publish/heroku.py"
            CloudRunPub["datasette/publish/cloudrun.py"]:::cli
            click CloudRunPub "https://github.com/simonw/datasette/blob/main/datasette/publish/cloudrun.py"
        end
        CLIEntry -->|"publish command"| PublishCommon
        PublishCommon --> HerokuPub
        PublishCommon --> CloudRunPub
    end

    %% Project Metadata & Deployment Scripts
    subgraph "Project Metadata & Scripts"
        Dockerfile["Dockerfile"]:::external
        Justfile["Justfile"]:::external
        SetupPy["setup.py"]:::external
        CI[".github/workflows/*.yml"]:::external
    end
    PublishCommon --> Dockerfile
    PublishCommon --> Justfile
    PublishCommon --> SetupPy
    PublishCommon --> CI

    %% External Services
    subgraph "External Services"
        DockerRegistry["Docker Registry"]:::external
        HerokuService["Heroku"]:::external
        CloudRunService["Google Cloud Run"]:::external
    end
    HerokuPub -->|"push image"| DockerRegistry
    CloudRunPub -->|"push image"| DockerRegistry
    DockerRegistry --> HerokuService
    DockerRegistry --> CloudRunService

    %% Styles
    classDef frontend fill:#D0E4FD,stroke:#1A73E8,color:#1A73E8;
    classDef core fill:#DFF7D9,stroke:#34A853,color:#34A853;
    classDef data fill:#FFF4CC,stroke:#FBBC05,color:#FBBC05;
    classDef render fill:#F3E5F5,stroke:#A142F4,color:#A142F4;
    classDef plugin fill:#FFE8D1,stroke:#FB8C00,color:#FB8C00;
    classDef cli fill:#E8EAED,stroke:#5F6368,color:#5F6368;
    classDef external fill:#FCE8E6,stroke:#EA4335,color:#EA4335;
```
