# pytest-dev/pytest-xdist architecture

> Architecture diagram of the GitHub repository pytest-dev/pytest-xdist, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pytest-dev/pytest-xdist
- Repository: https://github.com/pytest-dev/pytest-xdist

## Overview

##### Main Components Identification
The system has several key components that should be represented:

a) Core Components:
- Plugin Manager (plugin.py)
- Distributed Session Manager (dsession.py)
- Worker Manager (workermanage.py)
- Remote Execution Handler (remote.py)
- Various Schedulers (scheduler/ directory)

b) Scheduler Types:
- Load-based (load.py)
- Load-scope based (loadscope.py)
- Load-file based (loadfile.py)
- Load-group based (loadgroup.py)
- Work-stealing based (worksteal.py)

##### Component Relationships

**[pytest core] → [pytest-xdist plugin]**
- ↓
- [Distributed Session Manager]
- ↓
- ┌────────[Worker Manager]────────┐
- ↓            ↓           ↓       ↓
**[Worker 1]   [Worker 2]   [Worker 3] ...**

##### Specific Elements

a) Control Flow:
- Test collection flow
- Test distribution flow
- Result aggregation flow

b) Components:
- Use rectangles for main components
- Use diamonds for decision points (scheduler selection)

##### Important Details to Highlight

a) Plugin Integration:
- How it hooks into pytest
- Newhooks.py integration points

b) Scheduler Selection:
- How different schedulers can be selected
- Default load balancing strategy

c) Worker Management:
- Worker spawning
- Test distribution
- Result collection

**┌─ Plugin Integration ─┐  ┌─ Test Distribution ─┐  ┌─ Execution ─┐**
- Plugin Manager   │  │  - Session Manager  │  │  - Workers  │
- Hooks           │  │  - Schedulers       │  │  - Results  │
- ┘  └───────────────────--┘  └────────────-┘

##### Additional Notes

- Version compatibility information
- Important configuration options

##### The result aggregation process

Keep the abstraction level focused on the architectural components while providing enough detail to understand the distributed test execution flow. This will help developers understand both the high-level structure and the interaction between components.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Integration
        PytestCore[Pytest Core]
        PluginMgr[Plugin Manager]:::core
        Hooks[Pytest Hooks]:::core
    end

    subgraph Distribution
        DSManager[Distributed Session Manager]:::core
        WorkerMgr[Worker Manager]:::core
        Remote[Remote Execution Handler]:::core
    end

    subgraph Schedulers
        Protocol[Scheduler Protocol]:::scheduler
        LoadSched[Load Scheduler]:::scheduler
        LoadScope[Load-Scope Scheduler]:::scheduler
        LoadFile[Load-File Scheduler]:::scheduler
        LoadGroup[Load-Group Scheduler]:::scheduler
        WorkSteal[Work-Stealing Scheduler]:::scheduler
    end

    subgraph Execution
        Worker1[Worker 1]:::worker
        Worker2[Worker 2]:::worker
        Worker3[Worker 3]:::worker
        Report[Report Handler]:::core
    end

    %% Relationships
    PytestCore --> PluginMgr
    PluginMgr --> Hooks
    Hooks --> DSManager
    DSManager --> WorkerMgr
    WorkerMgr --> Remote
    Remote --> Worker1
    Remote --> Worker2
    Remote --> Worker3
    
    DSManager --> Protocol
    Protocol --> LoadSched
    Protocol --> LoadScope
    Protocol --> LoadFile
    Protocol --> LoadGroup
    Protocol --> WorkSteal
    
    Worker1 --> Report
    Worker2 --> Report
    Worker3 --> Report
    Report --> DSManager

    %% Styling
    classDef core fill:#2374ab
    classDef worker fill:#57b894
    classDef scheduler fill:#ffd166

    %% Click events
    click PluginMgr "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/plugin.py"
    click DSManager "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/dsession.py"
    click WorkerMgr "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/workermanage.py"
    click Remote "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/remote.py"
    click Protocol "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/protocol.py"
    click LoadSched "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/load.py"
    click LoadScope "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/loadscope.py"
    click LoadFile "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/loadfile.py"
    click LoadGroup "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/loadgroup.py"
    click WorkSteal "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/worksteal.py"
    click Hooks "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/newhooks.py"
    click Report "https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/report.py"
```
