How superlinear-ai/substrate works
Project Type and Purpose
• The project is a template repository (using Copier) that scaffolds new Python projects. It abstracts away repetitive setup by generating a project structure with integrated CI/CD, dependency management, development containers, linting/formatting, and even options for FastAPI API endpoints or Typer CLI support.
File Structure
• Top-level files and directories include configuration files (.devcontainer, Dockerfile, pyproject.toml), CI/CD configurations (GitHub workflows, GitLab CI), and project metadata (cookiecutter.json, copier.yml).
• The template/ folder holds the actual Jinja templated files that form the base of every new project. Within this folder, subdirectories for development environments, Docker setups, source code (src/), testing (tests/), and even CI/CD configurations are specified.
• There is also a generated folder structure (using cookiecutter variables) that indicates how a new project will be produced, complete with its own src, tests, and configuration files. This shows that the architecture is layered: the template definition and the output project.
README for Additional Insights
• The README clarifies that this project supports both developing the template and generating new Python projects.
• It highlights features like development with Dev Containers, Codespaces integration, pre-commit hooks, continuous integration pipelines, and more.
• The instructions for “migrating from Cookiecutter” further reinforce that this template encapsulates best practices in Python project structures.
a. Main Components:
• Template Engine Layer: Represents the Copier template configuration (copier.yml, cookiecutter.json) and the templates (files under template/).
• Project Generation Engine: The process that executes the Copier template to generate a new Python project.
• Generated Project Structure:
• Source Code Component: Project source files under src/{{ project_name_snake_case }} (and variants for API/CLI functionalities).
• Testing Suite: Tests directory with various test files (e.g., test_api.py, test_cli.py, test_import.py).
• Configuration and Environment: Files for development containers (.devcontainer), Docker configurations, pyproject.toml for dependency management, and pre-commit configurations.
• CI/CD Integration: GitHub Actions workflows, GitLab CI, Dependabot, and Release/Versioning processes integrated via Commitizen.
• Development Tools & Environments: Dev Containers setups, VS Code and Codespaces integration, as well as tools like uv, throw-poet, and Starship for shell experience.
b. Relationships and Interactions:
• That the generated project itself has internal boundaries:
• A distinct division between the application’s core logic (src/) and tests.
• Separate blocks for development tooling (Dev Containers, Docker, .gitignore) and continuous integration (GitHub workflows, GitLab CI files).
• Communication between the CI/CD systems and the repository (i.e., automated testing, dependency checks, and deployment workflows are triggered by changes in the project).
c. Architectural Patterns and Design Principles:
• Separation of concerns:
• Template versus Generated Project (build time versus runtime structure).
• Development environment configuration is isolated (dev container, Docker, VS Code settings).
• Use of modern DevOps practices – automated testing, linting, and dependency management integrated through pre-commit hooks and CI pipelines.
• Extensibility points where optional features (e.g., FastAPI API, Typer CLI) branch out from the main project structure.
d. Relevant Technologies and Frameworks:
• Tools: Copier for templating; uv for dependency management; Poe the Poet for task running; Pre-commit, Ruff, and Mypy for code quality.
• Environments: Docker, VS Code Dev Containers, GitHub Codespaces.
• CI/CD: GitHub Actions workflows, GitLab CI configurations, Dependabot, and Commitizen for semantic versioning.
Elements
• Annotations for specific integrations (e.g., “Dev Container (VS Code)” or “GitHub Actions Workflow”).