# genezera/ozymandias architecture

> Architecture diagram of the GitHub repository genezera/ozymandias, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/genezera/ozymandias
- Repository: https://github.com/genezera/ozymandias

## Overview

Ozymandias is a Tor-oriented OSINT console: users submit a query through the dashboard or CLI, the crawler searches engines, validates and enriches onion results, and stores intelligence and reports for review. Discovery and endpoint reputation extend that workflow; the dashboard also provides operational monitoring and a separate engine probe. The excerpts show crawler enrichment and persistence calls and dashboard subprocess launches, but not all crawler internals, so some documented subsystem links cite the README rather than unsampled implementation.

Main entry points: [`crawler.py`](https://github.com/genezera/ozymandias/blob/main/crawler.py) (Crawler CLI), [`dashboard.py`](https://github.com/genezera/ozymandias/blob/main/dashboard.py) (Streamlit Dashboard), [`probe_engines.py`](https://github.com/genezera/ozymandias/blob/main/probe_engines.py) (Engine Probe)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_operations["Operations UI"]
  node_dashboard["Streamlit Dashboard<br/>[dashboard.py]"]
  node_hud["HUD Analytics<br/>[dashboard.py]"]
  node_data_logs["Data and Logs<br/>[dashboard.py]"]
  node_probe_ui["Probe Launcher<br/>[dashboard.py]"]
end

subgraph group_collection["Search and Collection"]
  node_crawler["Crawler CLI<br/>[crawler.py]"]
  node_engine_search["Engine Search<br/>[crawler.py]"]
  node_endpoint_check["Endpoint Validation<br/>[crawler.py]"]
  node_enrichment["Result Enrichment<br/>[crawler.py]"]
  node_relevance["Relevance Analysis<br/>[crawler.py]"]
end

subgraph group_intelligence["Engine Intelligence"]
  node_discovery["Engine Discovery<br/>[crawler.py]"]
  node_reputation["Engine Reputation<br/>[crawler.py]"]
  node_probe["Engine Probe<br/>[probe_engines.py]"]
end

subgraph group_outputs["Persistence and Reports"]
  node_knowledge[("Knowledge Database<br/>[crawler.py]")]
  node_results_file["Search Results Workbook"]
  node_reports["Markdown Reports"]
  node_logs["Operation Logs"]
  node_html_captures["Probe HTML Captures"]
end

node_user(("Researcher"))
node_tor{{"Tor SOCKS Proxy"}}
node_onion_engines{{"Onion Search Engines"}}
node_onion_sites{{"Onion Services"}}

node_user -->|"starts scan"| node_dashboard
node_user -->|"runs query"| node_crawler
node_dashboard -->|"launches"| node_crawler
node_dashboard -->|"launches"| node_probe
node_dashboard -->|"loads workbook"| node_results_file
node_dashboard -->|"reads logs"| node_logs
node_dashboard -->|"opens captures"| node_html_captures
node_dashboard -->|"renders"| node_hud
node_dashboard -->|"renders"| node_data_logs
node_crawler -->|"orchestrates"| node_engine_search
node_crawler -->|"may invoke"| node_discovery
node_engine_search -->|"queries"| node_onion_engines
node_engine_search -->|"routes requests"| node_tor
node_endpoint_check -->|"tests through"| node_tor
node_endpoint_check -->|"validates"| node_onion_engines
node_crawler -->|"validates endpoints"| node_endpoint_check
node_crawler -->|"enriches results"| node_enrichment
node_enrichment -->|"fetches pages"| node_onion_sites
node_enrichment -->|"analyzes pages"| node_relevance
node_crawler -->|"exports results"| node_results_file
node_crawler -->|"generates reports"| node_reports
node_crawler -->|"records operations"| node_logs
node_crawler -->|"persists intelligence"| node_knowledge
node_discovery -->|"finds candidates"| node_onion_engines
node_discovery -->|"validates candidates"| node_endpoint_check
node_discovery -->|"tracks outcomes"| node_reputation
node_reputation -->|"stores engine data"| node_knowledge
node_probe -->|"routes probes"| node_tor
node_probe -->|"tests engines"| node_onion_engines
node_probe -->|"saves responses"| node_html_captures

click node_dashboard "https://github.com/genezera/ozymandias/blob/main/dashboard.py"
click node_hud "https://github.com/genezera/ozymandias/blob/main/dashboard.py"
click node_data_logs "https://github.com/genezera/ozymandias/blob/main/dashboard.py"
click node_probe_ui "https://github.com/genezera/ozymandias/blob/main/dashboard.py"
click node_crawler "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_engine_search "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_endpoint_check "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_enrichment "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_relevance "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_discovery "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_reputation "https://github.com/genezera/ozymandias/blob/main/crawler.py"
click node_probe "https://github.com/genezera/ozymandias/blob/main/probe_engines.py"
click node_knowledge "https://github.com/genezera/ozymandias/blob/main/crawler.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_hud,node_data_logs,node_probe_ui toneBlue
class node_crawler,node_engine_search,node_endpoint_check,node_enrichment,node_relevance toneAmber
class node_discovery,node_reputation,node_probe toneMint
class node_knowledge,node_results_file,node_reports,node_logs,node_html_captures toneRose
class node_user,node_tor,node_onion_engines,node_onion_sites toneIndigo
```

## Components

### Operations UI

- [Streamlit Dashboard](https://github.com/genezera/ozymandias/blob/main/dashboard.py): `dashboard.py`
- [HUD Analytics](https://github.com/genezera/ozymandias/blob/main/dashboard.py): `dashboard.py`
- [Data and Logs](https://github.com/genezera/ozymandias/blob/main/dashboard.py): `dashboard.py`
- [Probe Launcher](https://github.com/genezera/ozymandias/blob/main/dashboard.py): `dashboard.py`

### Search and Collection

- [Crawler CLI](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- [Engine Search](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- [Endpoint Validation](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- [Result Enrichment](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- [Relevance Analysis](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`

### Engine Intelligence

- [Engine Discovery](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- [Engine Reputation](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- [Engine Probe](https://github.com/genezera/ozymandias/blob/main/probe_engines.py): `probe_engines.py`

### Persistence and Reports

- [Knowledge Database](https://github.com/genezera/ozymandias/blob/main/crawler.py): `crawler.py`
- **Search Results Workbook**
- **Markdown Reports**
- **Operation Logs**
- **Probe HTML Captures**

### Other components

- **Researcher**
- **Tor SOCKS Proxy**
- **Onion Search Engines**
- **Onion Services**

## Connections

- Researcher → Streamlit Dashboard: starts scan (evidence: `README.md`)
- Researcher → Crawler CLI: runs query (evidence: `README.md`)
- Streamlit Dashboard → Crawler CLI: launches (evidence: `dashboard.py`)
- Streamlit Dashboard → Engine Probe: launches (evidence: `dashboard.py`)
- Streamlit Dashboard → Search Results Workbook: loads workbook (evidence: `dashboard.py`)
- Streamlit Dashboard → Operation Logs: reads logs (evidence: `README.md`)
- Streamlit Dashboard → Probe HTML Captures: opens captures (evidence: `dashboard.py`)
- Streamlit Dashboard → HUD Analytics: renders (evidence: `dashboard.py`)
- Streamlit Dashboard → Data and Logs: renders (evidence: `dashboard.py`)
- Crawler CLI → Engine Search: orchestrates (evidence: `README.md`)
- Crawler CLI → Engine Discovery: may invoke (evidence: `crawler.py`)
- Engine Search → Onion Search Engines: queries (evidence: `README.md`)
- Engine Search → Tor SOCKS Proxy: routes requests (evidence: `README.md`)
- Endpoint Validation → Tor SOCKS Proxy: tests through (evidence: `crawler.py`)
- Endpoint Validation → Onion Search Engines: validates (evidence: `crawler.py`)
- Crawler CLI → Endpoint Validation: validates endpoints (evidence: `README.md`)
- Crawler CLI → Result Enrichment: enriches results (evidence: `README.md`)
- Result Enrichment → Onion Services: fetches pages (evidence: `crawler.py`)
- Result Enrichment → Relevance Analysis: analyzes pages (evidence: `crawler.py`)
- Crawler CLI → Search Results Workbook: exports results (evidence: `README.md`)
- Crawler CLI → Markdown Reports: generates reports (evidence: `README.md`)
- Crawler CLI → Operation Logs: records operations (evidence: `README.md`)
- Crawler CLI → Knowledge Database: persists intelligence (evidence: `README.md`)
- Engine Discovery → Onion Search Engines: finds candidates (evidence: `crawler.py`)
- Engine Discovery → Endpoint Validation: validates candidates (evidence: `README.md`)
- Engine Discovery → Engine Reputation: tracks outcomes (evidence: `README.md`)
- Engine Reputation → Knowledge Database: stores engine data (evidence: `README.md`)
- Engine Probe → Tor SOCKS Proxy: routes probes (evidence: `probe_engines.py`)
- Engine Probe → Onion Search Engines: tests engines (evidence: `probe_engines.py`)
- Engine Probe → Probe HTML Captures: saves responses (evidence: `probe_engines.py`)