# superlinear-ai/substrate architecture

> Architecture diagram of the GitHub repository superlinear-ai/substrate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/superlinear-ai/substrate
- Repository: https://github.com/superlinear-ai/substrate

## Overview

##### Project Type and Purpose
- The project is a template repository (using Copier) that scaffolds new Python projects. It abstracts away repetitive setup by generating a project structure with integrated CI/CD, dependency management, development containers, linting/formatting, and even options for FastAPI API endpoints or Typer CLI support.

##### File Structure
- Top-level files and directories include configuration files (.devcontainer, Dockerfile, pyproject.toml), CI/CD configurations (GitHub workflows, GitLab CI), and project metadata (cookiecutter.json, copier.yml).
- The template/ folder holds the actual Jinja templated files that form the base of every new project. Within this folder, subdirectories for development environments, Docker setups, source code (src/), testing (tests/), and even CI/CD configurations are specified.
- There is also a generated folder structure (using cookiecutter variables) that indicates how a new project will be produced, complete with its own src, tests, and configuration files. This shows that the architecture is layered: the template definition and the output project.

##### README for Additional Insights
- The README clarifies that this project supports both developing the template and generating new Python projects.
- It highlights features like development with Dev Containers, Codespaces integration, pre-commit hooks, continuous integration pipelines, and more.
- The instructions for “migrating from Cookiecutter” further reinforce that this template encapsulates best practices in Python project structures.

a. Main Components:
- Template Engine Layer: Represents the Copier template configuration (copier.yml, cookiecutter.json) and the templates (files under template/).
- Project Generation Engine: The process that executes the Copier template to generate a new Python project.
- Generated Project Structure:
- Source Code Component: Project source files under src/{{ project_name_snake_case }} (and variants for API/CLI functionalities).
- Testing Suite: Tests directory with various test files (e.g., test_api.py, test_cli.py, test_import.py).
- Configuration and Environment: Files for development containers (.devcontainer), Docker configurations, pyproject.toml for dependency management, and pre-commit configurations.
- CI/CD Integration: GitHub Actions workflows, GitLab CI, Dependabot, and Release/Versioning processes integrated via Commitizen.
- Development Tools & Environments: Dev Containers setups, VS Code and Codespaces integration, as well as tools like uv, throw-poet, and Starship for shell experience.

b. Relationships and Interactions:
- That the generated project itself has internal boundaries:
- A distinct division between the application’s core logic (src/) and tests.
- Separate blocks for development tooling (Dev Containers, Docker, .gitignore) and continuous integration (GitHub workflows, GitLab CI files).
- Communication between the CI/CD systems and the repository (i.e., automated testing, dependency checks, and deployment workflows are triggered by changes in the project).

c. Architectural Patterns and Design Principles:
- Separation of concerns:
- Template versus Generated Project (build time versus runtime structure).
- Development environment configuration is isolated (dev container, Docker, VS Code settings).
- Use of modern DevOps practices – automated testing, linting, and dependency management integrated through pre-commit hooks and CI pipelines.
- Extensibility points where optional features (e.g., FastAPI API, Typer CLI) branch out from the main project structure.

d. Relevant Technologies and Frameworks:
- Tools: Copier for templating; uv for dependency management; Poe the Poet for task running; Pre-commit, Ruff, and Mypy for code quality.
- Environments: Docker, VS Code Dev Containers, GitHub Codespaces.
- CI/CD: GitHub Actions workflows, GitLab CI configurations, Dependabot, and Commitizen for semantic versioning.

##### Elements
- Annotations for specific integrations (e.g., “Dev Container (VS Code)” or “GitHub Actions Workflow”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    UI["User Input (Copier Parameters)"]:::input

    %% Template Engine Layer
    subgraph "Template Engine Layer"
        T1["copier.yml"]:::template
        T2["cookiecutter.json"]:::template
        T3["template/ (Jinja Templates)"]:::template
    end

    %% Project Generation Engine
    subgraph "Project Generation Engine"
        PGE["hooks/post_gen_project.py"]:::engine
    end

    %% Generated Project Structure
    subgraph "Generated Project Structure"
        subgraph "Application Core"
            AC["Source Code: {{ cookiecutter.__project_name_kebab_case }}/src/{{ cookiecutter.__project_name_snake_case }}"]:::core
        end
        subgraph "Testing Suite"
            TS["Tests: {{ cookiecutter.__project_name_kebab_case }}/tests"]:::test
        end
        subgraph "Configuration and Environment"
            DEV1["Dev Container: .devcontainer (Top-level)"]:::config
            DEV2["Dev Container: {{ cookiecutter.__project_name_kebab_case }}/.devcontainer (Generated)"]:::config
            DOCK1["Dockerfile (Top-level)"]:::config
            DOCK2["Dockerfile.jinja (Template)"]:::config
            DOCK3["docker-compose.yml.jinja (Template)"]:::config
            PY1["pyproject.toml (Top-level)"]:::config
            PY2["pyproject.toml.jinja (Template)"]:::config
            PRE[".pre-commit-config.yaml"]:::config
        end
    end

    %% CI/CD Integration
    subgraph "CI/CD Integration"
        GHA1[".github/workflows/test.yml (Top-level)"]:::ci
        GHA2["{{ cookiecutter.__project_name_kebab_case }}/.github/workflows (Generated)"]:::ci
        GHA3["template/.github/workflows (Template)"]:::ci
        GL[".gitlab-ci.yml"]:::ci
    end

    %% Connections from User Input to Template Engine Layer
    UI -->|"drives"| T1
    UI -->|"drives"| T2
    UI -->|"drives"| T3

    %% Template Engine feeds into Project Generation
    T1 -->|"feeds"| PGE
    T2 -->|"feeds"| PGE
    T3 -->|"feeds"| PGE

    %% Project Generation creates Generated Project Structure
    PGE -->|"generates"| AC
    PGE -->|"generates"| TS
    PGE -->|"generates"| DEV1
    PGE -->|"generates"| DEV2
    PGE -->|"generates"| DOCK1
    PGE -->|"generates"| DOCK2
    PGE -->|"generates"| DOCK3
    PGE -->|"generates"| PY1
    PGE -->|"generates"| PY2
    PGE -->|"generates"| PRE

    %% Generated Project triggers CI/CD Integration
    AC -->|"triggers"| GHA1
    TS -->|"triggers"| GHA1
    DEV1 -->|"supports"| AC
    DOCK1 -->|"supports"| AC
    PY1 -->|"supports"| AC
    GHA1 -->|"integrates with"| GHA2
    GHA2 -->|"coordinates with"| GHA3
    GHA3 -->|"complements"| GL

    %% Click Events for Template Engine Layer
    click T1 "https://github.com/superlinear-ai/substrate/blob/main/copier.yml"
    click T2 "https://github.com/superlinear-ai/substrate/blob/main/cookiecutter.json"
    click T3 "https://github.com/superlinear-ai/substrate/tree/main/template/"

    %% Click Event for Project Generation Engine
    click PGE "https://github.com/superlinear-ai/substrate/blob/main/hooks/post_gen_project.py"

    %% Click Events for Generated Project Structure - Application Core and Testing Suite
    click AC "https://github.com/superlinear-ai/substrate/blob/main/{{ cookiecutter.__project_name_kebab_case}}/src/{{ cookiecutter.__project_name_snake_case }}"
    click TS "https://github.com/superlinear-ai/substrate/tree/main/{{ cookiecutter.__project_name_kebab_case}}/tests"

    %% Click Events for Configuration and Environment
    click DEV1 "https://github.com/superlinear-ai/substrate/blob/main/.devcontainer"
    click DEV2 "https://github.com/superlinear-ai/substrate/blob/main/{{ cookiecutter.__project_name_kebab_case}}/.devcontainer"
    click DOCK1 "https://github.com/superlinear-ai/substrate/tree/main/Dockerfile"
    click DOCK2 "https://github.com/superlinear-ai/substrate/blob/main/template/Dockerfile.jinja"
    click DOCK3 "https://github.com/superlinear-ai/substrate/blob/main/template/docker-compose.yml.jinja"
    click PY1 "https://github.com/superlinear-ai/substrate/blob/main/pyproject.toml"
    click PY2 "https://github.com/superlinear-ai/substrate/blob/main/template/pyproject.toml.jinja"
    click PRE "https://github.com/superlinear-ai/substrate/blob/main/.pre-commit-config.yaml"

    %% Click Events for CI/CD Integration
    click GHA1 "https://github.com/superlinear-ai/substrate/blob/main/.github/workflows/test.yml"
    click GHA2 "https://github.com/superlinear-ai/substrate/tree/main/{{ cookiecutter.__project_name_kebab_case}}/.github/workflows"
    click GHA3 "https://github.com/superlinear-ai/substrate/tree/main/template/.github/workflows"
    click GL "https://github.com/superlinear-ai/substrate/blob/main/.gitlab-ci.yml"

    %% Styling Definitions
    classDef input fill:#fff3cd,stroke:#d39e00,stroke-width:2px;
    classDef template fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef engine fill:#ffeeba,stroke:#856404,stroke-width:2px;
    classDef core fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef test fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef config fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef ci fill:#f5c6cb,stroke:#721c24,stroke-width:2px;
```
