# s-rah/onionscan architecture

> Architecture diagram of the GitHub repository s-rah/onionscan, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/s-rah/onionscan
- Repository: https://github.com/s-rah/onionscan

## Overview

OnionScan investigates Tor hidden services for operators and researchers. The main workflow accepts onion targets, runs configured protocol checks and web crawling, extracts identity correlations, then emits human-readable or JSON reports. Researchers can also use the correlation-lab web interface to search and tag stored crawl relationships. The graph covers those documented capabilities; some subsystem links rely on imports or README descriptions because their implementation excerpts were not supplied.

Main entry points: [`main.go`](https://github.com/s-rah/onionscan/blob/master/main.go) (CLI entry), [`crawldb/crawldb.go`](https://github.com/s-rah/onionscan/blob/master/crawldb/crawldb.go) (Crawl database), [`onionscan/pipeline.go`](https://github.com/s-rah/onionscan/blob/master/onionscan/pipeline.go) (Scan pipeline), [`spider/onionspider.go`](https://github.com/s-rah/onionscan/blob/master/spider/onionspider.go) (Web crawler)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_scan["Scanning"]
  node_main["CLI entry<br/>[main.go]"]
  node_config["Scan configuration"]
  node_pipeline["Scan pipeline<br/>[pipeline.go]"]
  node_protocols["Protocol scanners"]
  node_webscan["Web protocol scan<br/>[http_scanner.go]"]
  node_spider["Web crawler<br/>[onionspider.go]"]
  node_parser["Page parser<br/>[pageparser.go]"]
end

subgraph group_analysis["Analysis"]
  node_deanon["Identity extraction"]
  node_bitcoin_extract["Bitcoin address check"]
  node_reports["Scan report model<br/>[onionscanreport.go]"]
  node_anonymity["Anonymity findings"]
  node_simple_report["Risk summary<br/>[simple_report.go]"]
end

subgraph group_storage["Persistence"]
  node_crawl_db[("Crawl database<br/>[crawldb.go]")]
  node_page_model["Page records<br/>[page.go]"]
end

subgraph group_interface["Outputs"]
  node_webui["Correlation lab<br/>[webui.go]"]
  node_json_writer["JSON report writer"]
  node_simple_writer["Simple report writer"]
end

node_operator(("Operator / researcher"))

node_operator -->|"starts scan"| node_main
node_main -->|"configures"| node_config
node_main -->|"runs targets"| node_pipeline
node_main -->|"adds analysis"| node_deanon
node_main -->|"starts interface"| node_webui
node_pipeline -->|"runs scan step"| node_protocols
node_pipeline -->|"creates report"| node_reports
node_protocols -->|"dispatches web scan"| node_webscan
node_webscan -->|"uses crawler"| node_spider
node_spider -->|"parses pages"| node_parser
node_spider -->|"stores crawls"| node_crawl_db
node_parser -->|"builds page"| node_page_model
node_deanon -.->|"includes check"| node_bitcoin_extract
node_bitcoin_extract -->|"reads crawls"| node_crawl_db
node_bitcoin_extract -->|"records findings"| node_anonymity
node_config -->|"opens database"| node_crawl_db
node_webui -->|"searches relationships"| node_crawl_db
node_reports -.->|"provides summary"| node_simple_report
node_main -->|"selects output"| node_json_writer
node_main -->|"selects output"| node_simple_writer
node_simple_writer -->|"formats report"| node_reports
node_operator -->|"searches and tags"| node_webui

click node_main "https://github.com/s-rah/onionscan/blob/master/main.go"
click node_config "https://github.com/s-rah/onionscan/blob/master/config/onionscan_config.go"
click node_pipeline "https://github.com/s-rah/onionscan/blob/master/onionscan/pipeline.go"
click node_protocols "https://github.com/s-rah/onionscan/tree/master/protocol"
click node_webscan "https://github.com/s-rah/onionscan/blob/master/protocol/http_scanner.go"
click node_spider "https://github.com/s-rah/onionscan/blob/master/spider/onionspider.go"
click node_parser "https://github.com/s-rah/onionscan/blob/master/spider/pageparser.go"
click node_deanon "https://github.com/s-rah/onionscan/tree/master/deanonymization"
click node_bitcoin_extract "https://github.com/s-rah/onionscan/blob/master/deanonymization/check_bitcoin_addresses.go"
click node_reports "https://github.com/s-rah/onionscan/blob/master/report/onionscanreport.go"
click node_anonymity "https://github.com/s-rah/onionscan/blob/master/report/anonymity_report.go"
click node_simple_report "https://github.com/s-rah/onionscan/blob/master/report/simple_report.go"
click node_crawl_db "https://github.com/s-rah/onionscan/blob/master/crawldb/crawldb.go"
click node_page_model "https://github.com/s-rah/onionscan/blob/master/model/page.go"
click node_webui "https://github.com/s-rah/onionscan/blob/master/webui/webui.go"
click node_json_writer "https://github.com/s-rah/onionscan/blob/master/onionscan/steps/jsonreportwriter.go"
click node_simple_writer "https://github.com/s-rah/onionscan/blob/master/onionscan/steps/simplereportwriter.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_config,node_pipeline,node_protocols,node_webscan,node_spider,node_parser toneBlue
class node_deanon,node_bitcoin_extract,node_reports,node_anonymity,node_simple_report toneAmber
class node_crawl_db,node_page_model toneMint
class node_webui,node_json_writer,node_simple_writer toneRose
class node_operator toneIndigo
```

## Components

### Scanning

- [CLI entry](https://github.com/s-rah/onionscan/blob/master/main.go): `main.go`
- [Scan configuration](https://github.com/s-rah/onionscan/blob/master/config/onionscan_config.go): `config/onionscan_config.go`
- [Scan pipeline](https://github.com/s-rah/onionscan/blob/master/onionscan/pipeline.go): `onionscan/pipeline.go`
- [Protocol scanners](https://github.com/s-rah/onionscan/tree/master/protocol): `protocol`
- [Web protocol scan](https://github.com/s-rah/onionscan/blob/master/protocol/http_scanner.go): `protocol/http_scanner.go`
- [Web crawler](https://github.com/s-rah/onionscan/blob/master/spider/onionspider.go): `spider/onionspider.go`
- [Page parser](https://github.com/s-rah/onionscan/blob/master/spider/pageparser.go): `spider/pageparser.go`

### Analysis

- [Identity extraction](https://github.com/s-rah/onionscan/tree/master/deanonymization): `deanonymization`
- [Bitcoin address check](https://github.com/s-rah/onionscan/blob/master/deanonymization/check_bitcoin_addresses.go): `deanonymization/check_bitcoin_addresses.go`
- [Scan report model](https://github.com/s-rah/onionscan/blob/master/report/onionscanreport.go): `report/onionscanreport.go`
- [Anonymity findings](https://github.com/s-rah/onionscan/blob/master/report/anonymity_report.go): `report/anonymity_report.go`
- [Risk summary](https://github.com/s-rah/onionscan/blob/master/report/simple_report.go): `report/simple_report.go`

### Persistence

- [Crawl database](https://github.com/s-rah/onionscan/blob/master/crawldb/crawldb.go): `crawldb/crawldb.go`
- [Page records](https://github.com/s-rah/onionscan/blob/master/model/page.go): `model/page.go`

### Outputs

- [Correlation lab](https://github.com/s-rah/onionscan/blob/master/webui/webui.go): `webui/webui.go`
- [JSON report writer](https://github.com/s-rah/onionscan/blob/master/onionscan/steps/jsonreportwriter.go): `onionscan/steps/jsonreportwriter.go`
- [Simple report writer](https://github.com/s-rah/onionscan/blob/master/onionscan/steps/simplereportwriter.go): `onionscan/steps/simplereportwriter.go`

### Other components

- **Operator / researcher**

## Connections

- Operator / researcher → CLI entry: starts scan
- CLI entry → Scan configuration: configures (evidence: `config/onionscan_config.go`)
- CLI entry → Scan pipeline: runs targets (evidence: `main.go`)
- CLI entry → Identity extraction: adds analysis (evidence: `main.go`)
- CLI entry → Correlation lab: starts interface (evidence: `main.go`)
- Scan pipeline → Protocol scanners: runs scan step
- Scan pipeline → Scan report model: creates report (evidence: `onionscan/pipeline.go`)
- Protocol scanners → Web protocol scan: dispatches web scan
- Web protocol scan → Web crawler: uses crawler (evidence: `protocol/http_scanner.go`)
- Web crawler → Page parser: parses pages (evidence: `spider/onionspider.go`)
- Web crawler → Crawl database: stores crawls (evidence: `spider/onionspider.go`)
- Page parser → Page records: builds page (evidence: `spider/pageparser.go`)
- Identity extraction → Bitcoin address check: includes check
- Bitcoin address check → Crawl database: reads crawls (evidence: `deanonymization/check_bitcoin_addresses.go`)
- Bitcoin address check → Anonymity findings: records findings (evidence: `deanonymization/check_bitcoin_addresses.go`)
- Scan configuration → Crawl database: opens database (evidence: `config/onionscan_config.go`)
- Correlation lab → Crawl database: searches relationships (evidence: `webui/webui.go`)
- Scan report model → Risk summary: provides summary
- CLI entry → JSON report writer: selects output (evidence: `main.go`)
- CLI entry → Simple report writer: selects output (evidence: `main.go`)
- Simple report writer → Scan report model: formats report (evidence: `onionscan/steps/simplereportwriter.go`)
- Operator / researcher → Correlation lab: searches and tags
