# danielgtaylor/python-betterproto architecture

> Architecture diagram of the GitHub repository danielgtaylor/python-betterproto, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/danielgtaylor/python-betterproto
- Repository: https://github.com/danielgtaylor/python-betterproto

## Overview

##### Project type and purpose
- This is an open-source Python library plus a protoc plugin (“betterproto”) that generates idiomatic Python code for Protobuf3 and gRPC.
- It consists of two main deliverables:
– A runtime library (`src/betterproto/...`) that provides base classes, serialization, gRPC stubs, casing, utilities, and bundled well-known types.
– A compiler plugin (`src/betterproto/plugin/...` and `src/betterproto/compile/...`) that protoc (or grpc_tools.protoc) invokes to parse `.proto` files and emit Python code.

##### File structure
- Top-level: configuration (`pyproject.toml`, `pytest.ini`), CI workflows, docs, benchmarks, tests, the library source under `src/betterproto`, and the plugin under `src/betterproto/plugin` and `src/betterproto/compile`.
- `src/betterproto/compile`: parsing, naming, importing modules—turn `.proto` AST into an internal representation.
- `src/betterproto/plugin`: plugin entrypoint (`main.py`, `compiler.py`), model validation, templating (`.j2` files in `templates`), typing compiler, command-line interface.
- `src/betterproto/grpc`: grpclib-based client/server stubs generator and utilities.
- `src/betterproto/lib/google` and `/std`: pre-compiled well-known types for performance.
- `src/betterproto/_types.py`, `casing.py`, `utils.py`: shared helpers for code generation and runtime.
- `tests` and `benchmarks`: verify code generation output and runtime behavior.

##### README for additional insights
- High-level architecture: protoc → plugin → generated code → user code + runtime library.
- Supports JSON & binary serialization, async gRPC (via grpclib), Pydantic dataclasses as an optional codegen mode.
- Injection points: `--python_betterproto_opt` flags to modify templates (e.g. pydantic, typing imports).
- Tested via both “standard tests” (data-driven proto/json directories) and “custom tests” (pytest).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Tools
    PROTOC["protoc / grpc_tools.protoc"]:::external

    %% Plugin Layer
    subgraph "Plugin Layer"
        PLUG_MAIN["__main__.py"]:::codegen
        PLUG_ENTRY["main.py"]:::codegen
        PLUG_COMP["compiler.py"]:::codegen
    end

    %% Codegen Pipeline
    subgraph "Codegen Pipeline"
        PARSER["Parser\n(plugin/parser.py)"]:::codegen
        VALID["Validation\n(module_validation.py)"]:::codegen
        IMPORT["Import Resolver\n(compile/importing.py)"]:::codegen
        NAMING["Naming Resolver\n(compile/naming.py)"]:::codegen
        TYPING["Typing Compiler\n(typing_compiler.py)"]:::codegen
    end

    %% Templating
    subgraph "Templating"
        HEADER["Header Template\n(templates/header.py.j2)"]:::codegen
        MODULET["Module Template\n(templates/template.py.j2)"]:::codegen
    end

    %% Generated Code
    GEN_CODE["Generated Python Modules\n(dataclasses, enums, stubs)"]:::codegen

    %% Runtime Library
    subgraph "Runtime Library"
        Types["_types.py"]:::runtime
        Casing["casing.py"]:::runtime
        Utils["utils.py"]:::runtime
        Version["_version.py"]:::runtime
        Enum["enum.py"]:::runtime
        WK_STD["std/"]:::runtime
        WK_GOOGLE["google/"]:::runtime
        WK_PYD["pydantic/"]:::runtime
    end

    %% gRPC Support
    subgraph "gRPC Support"
        CLIENT["grpclib_client.py"]:::runtime
        SERVER["grpclib_server.py"]:::runtime
        CHANNEL["async_channel.py"]:::runtime
    end

    %% External Dependencies
    subgraph "External Dependencies"
        PYTHON["Python 3.7+"]:::external
        GRPCLIB["grpclib"]:::external
        Pydantic["pydantic (optional)"]:::external
        Jinja2["Jinja2"]:::external
    end

    %% User Application
    USER_APP["User Application"]:::external

    %% Tests & CI
    subgraph "Tests & CI"
        CompileTests["Data-driven compile tests"]:::tests
        GRPCTests["GRPC-specific tests"]:::tests
        UnitTests["Unit tests"]:::tests
        Benchmarks["Benchmarks Suite"]:::tests
    end

    %% Flow Connections
    PROTOC --> PLUG_MAIN
    PLUG_MAIN --> PLUG_ENTRY --> PLUG_COMP
    PLUG_COMP --> PARSER --> VALID --> IMPORT --> NAMING --> TYPING
    TYPING --> HEADER --> MODULET --> GEN_CODE
    GEN_CODE --> Types & Casing & Utils & Version & Enum & WK_STD & WK_GOOGLE & WK_PYD
    GEN_CODE --> CLIENT --> CHANNEL --> SERVER
    GEN_CODE --> USER_APP

    Jinja2 --> HEADER & MODULET
    PYTHON --> GEN_CODE & Types & CLIENT
    GRPCLIB --> CLIENT & SERVER
    Pydantic --> TYPING

    CompileTests --> PARSER
    GRPCTests --> CLIENT
    UnitTests --> VALID & TYPING
    Benchmarks --> PARSER & Types

    %% Click Events
    click Types "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/_types.py"
    click Casing "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/casing.py"
    click Utils "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/utils.py"
    click Version "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/_version.py"
    click Enum "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/enum.py"
    click IMPORT "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/compile/importing.py"
    click NAMING "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/compile/naming.py"
    click PLUG_MAIN "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/plugin/__main__.py"
    click PLUG_ENTRY "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/plugin/main.py"
    click PLUG_COMP "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/plugin/compiler.py"
    click VALID "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/plugin/module_validation.py"
    click PARSER "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/plugin/parser.py"
    click TYPING "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/plugin/typing_compiler.py"
    click HEADER "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/templates/header.py.j2"
    click MODULET "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/templates/template.py.j2"
    click CLIENT "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/grpc/grpclib_client.py"
    click SERVER "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/grpc/grpclib_server.py"
    click CHANNEL "https://github.com/danielgtaylor/python-betterproto/blob/master/src/betterproto/grpc/util/async_channel.py"
    click WK_STD "https://github.com/danielgtaylor/python-betterproto/tree/master/src/betterproto/lib/std/"
    click WK_GOOGLE "https://github.com/danielgtaylor/python-betterproto/tree/master/src/betterproto/lib/google/"
    click WK_PYD "https://github.com/danielgtaylor/python-betterproto/tree/master/src/betterproto/lib/pydantic/"

    %% Styles
    classDef codegen fill:#f9f,stroke:#333,stroke-width:2px
    classDef runtime fill:#9f9,stroke:#333,stroke-width:2px
    classDef external fill:#99f,stroke:#333,stroke-width:2px
    classDef tests fill:#ff9,stroke:#333,stroke-width:2px
```
