How thetorproject/bwscanner works
Project type and purpose
• BWScanner is a command‐line, Python2 network measurement tool that scans Tor relays for bandwidth and aggregates the results. It is neither a web app nor a microservice suite but a modular CLI application with distinct phases: scanning (bwscan scan), writing measurements, and aggregating results.
File structure
• Top‐level:
• bwscanner/ ← Python package containing core modules
• scripts/ ← auxiliary scripts (e.g., aggregate.py)
• test/ ← unit tests and test scripts
• docs/ ← Sphinx documentation
• setup.py, requirements.txt, tox.ini, .travis.yml ← build, CI and dependency config
• Inside bwscanner/:
• config.py ← global settings loader
• logger.py ← logging facility
• fetcher.py ← download Tor consensus & descriptor data
• circuit.py ← build/attach circuits to relays
• measurement.py ← orchestrate bandwidth tests over circuits
• writer.py ← persist raw measurement files
• aggregate.py ← combine multiple scan result files
• attacher.py ← helper to attach streams to circuits
• scanner.py ← CLI “scan” command driver
• __init__.py
README
• Describes two primary workflows:
a. “bwscan scan” for collecting bandwidth measurements
b. “bwscan aggregate” or scripts/aggregate.py for merging results
• External dependency: Tor control port via txsocksx
Main components
a. CLI Layer
• bwscan entrypoint (console_script defined in setup.py)
• Commands: scan, aggregate
b. Core Modules
• Config module
• Logger
• Fetcher
• Circuit & Attacher
• Measurement
• Writer
• Aggregator
c. External Services
• Tor instances (ControlPort)
• Tor network / Relays
d. Storage
• Local filesystem directories for raw scans
• Aggregated output files
e. CI/Test/Docs (optional to show ancillary workflows)
Relationships & interactions
• User invokes “bwscan scan” → scanner.py
• scanner loads config & logger
• scanner calls fetcher to get the consensus/descriptor
• fetcher writes raw consensus data (cache)
• scanner iterates relays:
– for each relay: circuit.py sets up a Tor circuit
– attacher.py attaches test streams to the circuit
– measurement.py runs throughput tests over the circuit
– writer.py persists measurement records to disk
• End of scan: measurement files stored under ~/.config/bwscanner/measurements/<timestamp>
• User invokes “bwscan aggregate” or scripts/aggregate.py → aggregate.py
– aggregate loads multiple measurement directories
– merges and summarizes into final CSV/json for authorities
Architectural patterns / design principles
• Modular separation of concerns (fetcher vs. circuit vs. measurement vs. writer)
• Single responsibility per module
• CLI→Core library→External (Tor)→Storage
• Test suite mirrors modules under test/
Additional details
• Tests invoking modules in parallel
• Docs/ CI badges as side annotations if desired
• Config file injection into modules