# thetorproject/bwscanner architecture

> Architecture diagram of the GitHub repository thetorproject/bwscanner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thetorproject/bwscanner
- Repository: https://github.com/thetorproject/bwscanner

## Overview

##### Project type and purpose
- BWScanner is a command‐line, Python2 network measurement tool that scans Tor relays for bandwidth and aggregates the results. It is neither a web app nor a microservice suite but a modular CLI application with distinct phases: scanning (bwscan scan), writing measurements, and aggregating results.

##### File structure
- Top‐level:
- bwscanner/            ← Python package containing core modules
- scripts/              ← auxiliary scripts (e.g., aggregate.py)
- test/                 ← unit tests and test scripts
- docs/                 ← Sphinx documentation
- setup.py, requirements.txt, tox.ini, .travis.yml ← build, CI and dependency config
- Inside bwscanner/:
- config.py             ← global settings loader
- logger.py             ← logging facility
- fetcher.py            ← download Tor consensus & descriptor data
- circuit.py            ← build/attach circuits to relays
- measurement.py        ← orchestrate bandwidth tests over circuits
- writer.py             ← persist raw measurement files
- aggregate.py          ← combine multiple scan result files
- attacher.py           ← helper to attach streams to circuits
- scanner.py            ← CLI “scan” command driver
- __init__.py

##### README
- Describes two primary workflows:
a. “bwscan scan” for collecting bandwidth measurements
b. “bwscan aggregate” or scripts/aggregate.py for merging results
- External dependency: Tor control port via txsocksx

##### Main components
a. CLI Layer
- bwscan entrypoint (console_script defined in setup.py)
- Commands: scan, aggregate
b. Core Modules
- Config module
- Logger
- Fetcher
- Circuit & Attacher
- Measurement
- Writer
- Aggregator
c. External Services
- Tor instances (ControlPort)
- Tor network / Relays
d. Storage
- Local filesystem directories for raw scans
- Aggregated output files
e. CI/Test/Docs (optional to show ancillary workflows)

##### Relationships & interactions
- User invokes “bwscan scan” → scanner.py
- scanner loads config & logger
- scanner calls fetcher to get the consensus/descriptor
- fetcher writes raw consensus data (cache)
- scanner iterates relays:
– for each relay: circuit.py sets up a Tor circuit
– attacher.py attaches test streams to the circuit
– measurement.py runs throughput tests over the circuit
– writer.py persists measurement records to disk
- End of scan: measurement files stored under ~/.config/bwscanner/measurements/<timestamp>
- User invokes “bwscan aggregate” or scripts/aggregate.py → aggregate.py
– aggregate loads multiple measurement directories
– merges and summarizes into final CSV/json for authorities

##### Architectural patterns / design principles
- Modular separation of concerns (fetcher vs. circuit vs. measurement vs. writer)
- Single responsibility per module
- CLI→Core library→External (Tor)→Storage
- Test suite mirrors modules under test/

##### Additional details
- Tests invoking modules in parallel
- Docs/ CI badges as side annotations if desired
- Config file injection into modules

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI Layer"
        CLI["bwscan CLI"]:::internal
        Scanner["scanner.py"]:::internal
        AggCmd["aggregate.py"]:::internal
        scriptsAgg["scripts/aggregate.py"]:::aux
    end

    %% Core Modules
    subgraph "Core Modules"
        Config["config.py"]:::internal
        Logger["logger.py"]:::internal
        Fetcher["fetcher.py"]:::internal
        Circuit["circuit.py"]:::internal
        Attacher["attacher.py"]:::internal
        Measurement["measurement.py"]:::internal
        Writer["writer.py"]:::internal
    end

    %% External Systems
    subgraph "External Systems"
        TorInstance["Tor ControlPort"]:::external
        TorNetwork["Tor Network/Relays"]:::external
    end

    %% Storage
    subgraph "Storage"
        RawStore["Raw Measurements Store"]:::storage
        AggregatedStore["Aggregated Output Store"]:::storage
    end

    %% CI/Test/Docs
    subgraph "CI/Test/Docs"
        testCircuit["test/test_circuit.py"]:::aux
        testMeasurement["test/test_measurement.py"]:::aux
        testWriter["test/test_writer.py"]:::aux
        Docs["docs/source/bwscanner.rst"]:::aux
        CI[".travis.yml & tox.ini"]:::aux
    end

    %% Workflow
    User["User Terminal"]:::external
    User -->|"invoke scan/aggregate"| CLI
    CLI -->|"scan command"| Scanner
    CLI -->|"aggregate command"| AggCmd

    Scanner -->|"load config"| Config
    Scanner -->|"initialize logger"| Logger
    Scanner -->|"fetch consensus"| Fetcher
    Fetcher -->|"write raw consensus"| RawStore

    Scanner -->|"build circuit"| Circuit
    Circuit -->|"connect"| TorInstance
    TorInstance -->|"relay info"| TorNetwork

    Scanner -->|"attach streams"| Attacher
    Scanner -->|"run tests"| Measurement
    Measurement -->|"persist records"| Writer
    Writer -->|"store measurements"| RawStore

    AggCmd -->|"read measurements"| RawStore
    AggCmd -->|"merge results"| AggCmd
    AggCmd -->|"write summary"| AggregatedStore

    User -->|"run script"| scriptsAgg
    scriptsAgg -->|"invoke aggregation"| AggCmd

    %% Click Events
    click CLI "https://github.com/thetorproject/bwscanner/blob/develop/setup.py"
    click Scanner "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/scanner.py"
    click AggCmd "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/aggregate.py"
    click scriptsAgg "https://github.com/thetorproject/bwscanner/blob/develop/scripts/aggregate.py"
    click Config "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/config.py"
    click Logger "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/logger.py"
    click Fetcher "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/fetcher.py"
    click Circuit "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/circuit.py"
    click Attacher "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/attacher.py"
    click Measurement "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/measurement.py"
    click Writer "https://github.com/thetorproject/bwscanner/blob/develop/bwscanner/writer.py"
    click testCircuit "https://github.com/thetorproject/bwscanner/blob/develop/test/test_circuit.py"
    click testMeasurement "https://github.com/thetorproject/bwscanner/blob/develop/test/test_measurement.py"
    click testWriter "https://github.com/thetorproject/bwscanner/blob/develop/test/test_writer.py"
    click Docs "https://github.com/thetorproject/bwscanner/blob/develop/docs/source/bwscanner.rst"
    click CI "https://github.com/thetorproject/bwscanner/blob/develop/.travis.yml, tox.ini"

    %% Styles
    classDef internal fill:#cce5ff,stroke:#1f78b4;
    classDef external fill:#d4edda,stroke:#28a745;
    classDef storage fill:#fff3cd,stroke:#ff9800;
    classDef aux fill:#e2e3e5,stroke:#6c757d;
```
