# hfwittmann/annotated-transformer architecture

> Architecture diagram of the GitHub repository hfwittmann/annotated-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hfwittmann/annotated-transformer
- Repository: https://github.com/hfwittmann/annotated-transformer

## Overview

##### Main Components
- Core Processing Module:
- Represented by files such as the_annotated_transformer.py and Positional_encodings.py.
- This is where the main Transformer implementation logic resides.
- Notebook and Interactive Artifacts:
- The project uses jupytext to convert between Python script and Jupyter Notebook (.ipynb).
- A module (or process) that handles notebook generation (triggered via Makefile targets, such as “make notebook” and “make html”).
- Build & Automation Tools:
- The Makefile encapsulates commands for generating notebooks and HTML.
- GitHub Actions in the .github/workflows folder (e.g., checks.yml) are used for continuous integration, enforcing formatting and other tests.
- Dependency & Environment Management:
- Files like pyproject.toml, poetry.toml, requirements.txt, and .python-version show that dependencies are managed via Poetry and pip.
- Documentation & Writeup:
- The docs folder contains HTML, CSS (e.g., tufte.css), and YAML includes that form the project’s documentation site.
- The writeup folder with LaTeX-related files (acl2018.sty, acl_natbib.bst) indicate support for generating an academic writeup.
- Developer Tooling & Configuration:
- Files & folders like .vscode and .github contribute to developer experience with recommended settings and CI/CD.
- The README includes guidelines for formatting with black and flake8, plus setting up jupytext.

##### Relationships and Interactions
- Data Flow & Execution:
- The core processing modules produce transformer model outputs that can be viewed interactively via the notebook.
- The Makefile acts as a command layer to convert source code into the notebook (using jupytext) and then subsequently into an HTML artifact.
- Build and Testing Pipeline:
- Commits trigger GitHub actions (through .github/workflows) which invoke code formatting tools (black, flake8) and may run automated tests.
- Development and Deployment:
- The dependency management ensures that environment (via Poetry/pip) and code formatting (via black, flake8) remain consistent.
- Documentation files (in docs) and academic writeup files (in writeup) are built or updated as part of project maintenance and dissemination.

##### Architectural Patterns and Design Principles
- Modular Separation – There is a clear distinction between core computation (calculating transformer outputs), build/automation processes (Makefile and CI), and documentation.
- Automated Testing and Continuous Integration – The presence of GitHub Actions and explicit formatting commands highlights a CI-driven development process.
- Reproducibility and Developer Experience – Through Poetry, jupytext, and VSCode configuration, the project emphasizes reproducible builds and an efficient interactive development environment.

##### Relevant Technologies/Frameworks
- Python – The main language for the transformer implementation.
- jupytext & Jupyter Notebook – For synchronizing active code and interactive documents.
- Make – Provides a simplified command-line interface to trigger notebook and HTML generation.
- GitHub Actions – Facilitates automated testing and code style verification.
- Black, flake8, isort – Tools for formatting and linting.
- Poetry (as well as pyproject.toml and poetry.toml files) – Handles dependency management.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Dependency & Environment Management Cluster
    subgraph "Dependency & Environment Management"
        pythonVer[".python-version"]:::dep
        poetry["poetry.toml"]:::dep
        pyproject["pyproject.toml"]:::dep
        req["requirements.txt"]:::dep
    end

    %% Core Transformer Module Cluster
    subgraph "Core Transformer Module"
        transformer["the_annotated_transformer.py"]:::core
        positional["Positional_encodings.py"]:::core
    end

    %% Notebook Conversion System Cluster
    subgraph "Notebook Conversion System"
        makefileConv["Makefile (Conversion)"]:::conversion
        notebook["Jupyter Notebook (.ipynb)"]:::notebook
    end

    %% CI/CD & Build Automation Cluster
    subgraph "CI/CD & Build Automation"
        makefileCI["Makefile (CI)"]:::ci
        checks[".github/workflows/checks.yml"]:::ci
    end

    %% Documentation & Writeup Cluster
    subgraph "Documentation & Writeup"
        docs["docs"]:::doc
        writeup["writeup"]:::doc
    end

    %% Developer Tooling & Configuration Cluster
    subgraph "Developer Tooling & Configuration"
        vscode[".vscode"]:::dev
        devGithub[".github"]:::dev
        readme["README.md"]:::dev
    end

    %% Connections
    %% Dependency Management provides the environment for Core Transformer Module
    pyproject -->|"provides_environment"| transformer

    %% Core Transformer Module outputs source for conversion and triggers CI tests
    transformer -->|"jupytext_conversion"| makefileConv
    transformer -->|"trigger_CI"| checks

    %% Notebook Conversion System produces the Notebook artifact
    makefileConv -->|"conversion_output"| notebook

    %% CI/CD & Build Automation relates to Developer Tooling for configuration and guidelines
    checks -->|"config_guidelines"| readme

    %% Optional: Documentation & Writeup can be updated based on CI/CD results and Core output
    transformer -->|"update_docs"| docs
    transformer -->|"update_writeup"| writeup

    %% Styles
    classDef dep       fill:#FFEEAD,stroke:#D4A017,stroke-width:1px;
    classDef core      fill:#AEDFF7,stroke:#5B8FD7,stroke-width:1px;
    classDef conversion fill:#C7F9CC,stroke:#66B032,stroke-width:1px;
    classDef notebook  fill:#EEE,stroke:#999,stroke-dasharray: 5 5,stroke-width:1px;
    classDef ci        fill:#F7D6AE,stroke:#D75A4A,stroke-width:1px;
    classDef doc       fill:#FFF4B8,stroke:#E3B505,stroke-width:1px;
    classDef dev       fill:#E0BBE4,stroke:#9D4EDD,stroke-width:1px;

    %% Click Events
    click transformer "https://github.com/hfwittmann/annotated-transformer/blob/master/the_annotated_transformer.py"
    click positional "https://github.com/hfwittmann/annotated-transformer/blob/master/Positional_encodings.py"
    click makefileConv "https://github.com/hfwittmann/annotated-transformer/tree/master/Makefile"
    click makefileCI "https://github.com/hfwittmann/annotated-transformer/tree/master/Makefile"
    click checks "https://github.com/hfwittmann/annotated-transformer/blob/master/.github/workflows/checks.yml"
    click pythonVer "https://github.com/hfwittmann/annotated-transformer/blob/master/.python-version"
    click poetry "https://github.com/hfwittmann/annotated-transformer/blob/master/poetry.toml"
    click pyproject "https://github.com/hfwittmann/annotated-transformer/blob/master/pyproject.toml"
    click req "https://github.com/hfwittmann/annotated-transformer/blob/master/requirements.txt"
    click docs "https://github.com/hfwittmann/annotated-transformer/tree/master/docs"
    click writeup "https://github.com/hfwittmann/annotated-transformer/tree/master/writeup"
    click vscode "https://github.com/hfwittmann/annotated-transformer/blob/master/.vscode"
    click devGithub "https://github.com/hfwittmann/annotated-transformer/blob/master/.github"
    click readme "https://github.com/hfwittmann/annotated-transformer/blob/master/README.md"
```
