# pantsbuild/pants architecture

> Architecture diagram of the GitHub repository pantsbuild/pants, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pantsbuild/pants
- Repository: https://github.com/pantsbuild/pants

## Overview

##### Project type and purpose
- This is “Pants,” an open-source, polyglot build system written in Python (src/python/pants) and a Rust “engine” core (src/rust).
- Purpose: orchestrate builds, tests, packaging, remote caching/execution, and plugins across multiple languages in a monorepo.

##### High-level file structure
- Top-level:
- .devcontainer: VS Code/devcontainer setup
- build-support: CI and release helper scripts (Python, shell, JS)
- src/python/pants: the Python implementation of Pants CLI, plugins, backends
- src/rust: the Rust “engine” that underpins fast, concurrent execution
- pants-plugins: example/internal plugins
- docs & testprojects: documentation & sample repos
- CONTRIBUTING, workflows, and .github indicate CI/CD and governance.

##### README insights
- Emphasizes monorepo support, explicit dependency modeling, remote caching/execution, ext-ensibility via plugin API.

a. Main components
1. User Interface (CLI): `pants` Python script in src/python/pants/bin
2. Python Layer:
- Option parsing, goal dispatch, build graph construction (pants.base, pants.engine, pants.build_graph, pants.option, pants.core, etc.)
- Plugin/backends loader (pants.init → loads built-in & user plugins under pants-plugins)
3. FFI Boundary: `pyo3` or CPython bindings into Rust engine
4. Rust Engine Core:
- Scheduler (concurrent task graph execution)
- fs_store (file system / cache)
- process_execution (local / remote execution)
- internals: nodes, intrinsics, tasks
5. Backends & Plugins:
- In Python: src/python/pants/backend/* (Java, Python, Go, Docker, etc.)
- In pants-plugins: optional 3rd-party or internal plugins
6. External Services / Tools:
- Remote cache/execution services (gRPC remote API)
- External tool binaries (javac, pytest, Docker, etc.)
- GraphQL server (pants_explorer plugin under pants-plugins/pants_explorer/server)
7. Build-support & CI: scripts invoking Pants, packaging releases, docs generation

b. Relationships & interactions
- User runs `pants <goal> …` → Python CLI
→ Option parsing → Load plugins/backends → Construct BuildGraph → Translate goals into rule requests
→ FFI call → Rust engine returns snapshots, digests, spawn ProcessRequests
→ Engine schedules tasks in parallel, calls back into Python for rule implementations
→ Engine uses fs_store to read/write caches, optionally invoking remote execution via gRPC
→ Engine returns results to Python → Python prints output, writes artifacts, triggers post-actions
- Plugins register goals/rules via Python’s rules API; engine discovers them at startup.

c. Architectural patterns / design principles
- Layered architecture:
- Presentation / CLI (Python)
- Application logic & plugin API (Python)
- Execution engine (Rust)
- Plugin hook points: plugin resolver loads Python backends and optional GraphQL server
- FFI boundary isolates performance-critical code in Rust
- Remote Execution via Bazel RP remote-exec API
- Concurrency: Rust scheduler executes independent tasks in parallel

d. Key technologies
- Python 3.8+, Pants codebase and plugin API
- Rust (Cargo) for engine, fs, process_execution, remote execution
- gRPC / Protobuf for remote caching/execution (under src/rust/protos and process_execution)
- pyo3 or CPython FFI shim under Rust engine
- Uvicorn / GraphQL (for pants_explorer web UI)
- Various external tool integrations: javac, pytest, Docker, npm, gofmt, etc.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layers
    subgraph "Pants CLI & API"
        direction TB
        CLI["Pants CLI & API"]:::python
    end

    subgraph "Python Layer"
        direction TB
        Opt["Option Parsing & Configuration"]:::python
        BG["Build Graph Construction"]:::python
        EA["Engine API & Rule Dispatch"]:::python
        PR["Plugin Resolver"]:::python
        EL["Extension Loader"]:::python
    end

    subgraph "Python Backends & Plugins"
        direction TB
        BB["Builtin Backends (Java, Python, Go, Docker)"]:::plugin
        EP["External Plugins"]:::plugin
    end

    PYFFI["FFI Boundary (PyO3 glue)"]:::rust

    subgraph "Rust Engine Core"
        direction TB
        Sched["Scheduler"]:::rust
        Nodes["Task Graph Nodes & Execution"]:::rust
        FS["File System & Cache Store"]:::rust
        Proc["Process Execution (local & remote)"]:::rust
        Protos["Remote Cache/Execution Protos (gRPC API)"]:::rust
    end

    subgraph "External Services & Tools"
        direction TB
        RC["Remote Cache/Execution Service"]:::external
        ET["External Tools (javac, pytest, Docker, etc.)"]:::external
        GQL["GraphQL Server Plugin"]:::external
    end

    subgraph "Supporting Modules"
        direction TB
        CS["CI & Release Helpers"]:::support
        DOC["Documentation & Examples"]:::support
    end

    %% Connections
    CLI -->|runs| Opt
    CLI -->|loads at startup| PR
    Opt --> BG
    BG --> EA
    PR -->|resolves| BB
    PR -->|resolves| EP
    PR -->|loads GraphQL plugin| GQL
    EA -->|calls| PYFFI
    PYFFI -->|invokes| Sched
    Sched -->|manages| Nodes
    Nodes --> FS
    Nodes --> Proc
    Proc -->|uses| ET
    Proc -->|calls remote| RC
    Protos -->|defined by| Proc
    Sched -->|returns results| EA
    EA -->|outputs| CLI

    CS & DOC

    %% Click Events
    click CLI "https://github.com/pantsbuild/pants/tree/main/src/python/pants/bin"
    click Opt "https://github.com/pantsbuild/pants/tree/main/src/python/pants/option"
    click BG "https://github.com/pantsbuild/pants/tree/main/src/python/pants/build_graph"
    click EA "https://github.com/pantsbuild/pants/tree/main/src/python/pants/engine"
    click PR "https://github.com/pantsbuild/pants/blob/main/src/python/pants/init/plugin_resolver.py"
    click EL "https://github.com/pantsbuild/pants/blob/main/src/python/pants/init/extension_loader.py"
    click BB "https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend"
    click EP "https://github.com/pantsbuild/pants/tree/main/pants-plugins"
    click PYFFI "https://github.com/pantsbuild/pants/blob/main/src/rust/engine/src/python.rs"
    click Sched "https://github.com/pantsbuild/pants/blob/main/src/rust/engine/src/scheduler.rs"
    click Nodes "https://github.com/pantsbuild/pants/tree/main/src/rust/engine/src/nodes"
    click FS "https://github.com/pantsbuild/pants/tree/main/src/rust/fs/store"
    click Proc "https://github.com/pantsbuild/pants/tree/main/src/rust/process_execution"
    click Protos "https://github.com/pantsbuild/pants/tree/main/src/rust/protos"
    click GQL "https://github.com/pantsbuild/pants/tree/main/pants-plugins/pants_explorer/server"

    %% Styles
    classDef python fill:#AED6F1,stroke:#1B4F72,color:#1B2631
    classDef rust fill:#F5B041,stroke:#B9770E,color:#4D2600
    classDef external fill:#A9DFBF,stroke:#196F3D,color:#145A32
    classDef plugin fill:#FCF3CF,stroke:#B7950B,color:#7D6608
    classDef support fill:#D5DBDB,stroke:#616A6B,color:#1B2631
```
