# anonghosty/looking-glass architecture

> Architecture diagram of the GitHub repository anonghosty/looking-glass, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anonghosty/looking-glass
- Repository: https://github.com/anonghosty/looking-glass
- Video tour (about a minute): https://gitdiagram.com/anonghosty/looking-glass/video

## Overview

Looking Glass turns daily Shadowserver reports into a replayable national threat picture and tools for CSIRT follow-up. Its principal workflow is report delivery, validation and loading into a shared MongoDB, followed by read-only web analysis and response by analysts and executives. The graph preserves the documented map, services, findings, intelligence, advisory, audit and privacy capabilities. Some subsystem links are documented in the README; sampled code and the source index support the more specific pipeline and services relationships. Unsampled implementations are represented only at paths present in the tree.

Main entry points: [`platform/app.py`](https://github.com/anonghosty/looking-glass/blob/main/platform/app.py) (Web application), [`database/shadowserver_pipeline.py`](https://github.com/anonghosty/looking-glass/blob/main/database/shadowserver_pipeline.py) (Report pipeline), [`platform/services_api.py`](https://github.com/anonghosty/looking-glass/blob/main/platform/services_api.py) (Services API), [`database/db_connect.py`](https://github.com/anonghosty/looking-glass/blob/main/database/db_connect.py) (Shared reports DB)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Report ingestion"]
  node_pipeline["Report pipeline"]
  node_sendercheck["Sender verification<br/>[mail_auth.py]"]
  node_reportapi["Report API<br/>[report_api.py]"]
  node_workerconfig["Pipeline settings<br/>[worker_settings.py]"]
  node_mongo[("Shared reports DB<br/>[db_connect.py]")]
end

subgraph group_platform["Web platform"]
  node_webapp["Web application<br/>[app.py]"]
  node_servicesapi["Services API<br/>[services_api.py]"]
  node_cache["Shared result cache<br/>[respcache.py]"]
end

subgraph group_analysis["Analysis and intelligence"]
  node_servicesdata["Services analysis<br/>[services.py]"]
  node_map["Communications map<br/>[attack_map.py]"]
  node_combos["Combination analysis<br/>[combos.py]"]
  node_findings["Findings<br/>[findings.py]"]
  node_intelligence["Threat intelligence<br/>[intel.py]"]
  node_executive["Executive reports<br/>[quarterly.py]"]
  node_addressblocks["Address analysis<br/>[address_blocks.py]"]
end

subgraph group_response["CSIRT response"]
  node_advisories["Advisories<br/>[advisory.py]"]
  node_audit["Audit trail<br/>[audit_chain.py]"]
  node_notes["Research notes<br/>[notes.py]"]
end

subgraph group_security["Trust and privacy"]
  node_auth["Authentication<br/>[auth.py]"]
  node_privacy["Server-side masking<br/>[privacy.py]"]
end

node_shadowserver(("Shadowserver reports"))
node_user(("CSIRT users"))

node_shadowserver -->|"sends reports"| node_pipeline
node_pipeline -->|"checks sender"| node_sendercheck
node_pipeline -->|"fetches reports"| node_reportapi
node_pipeline -->|"loads reports"| node_mongo
node_workerconfig -->|"configures"| node_pipeline
node_user -->|"uses platform"| node_webapp
node_webapp -->|"reads reports"| node_mongo
node_webapp -->|"registers API"| node_servicesapi
node_servicesapi -->|"queries services"| node_servicesdata
node_servicesapi -->|"masks output"| node_privacy
node_webapp -->|"serves replay map"| node_map
node_webapp -->|"supports analysis"| node_combos
node_webapp -->|"shows findings"| node_findings
node_webapp -->|"shows intelligence"| node_intelligence
node_webapp -->|"provides portal"| node_executive
node_webapp -->|"supports advisories"| node_advisories
node_webapp -->|"records activity"| node_audit
node_webapp -->|"shares notes"| node_notes
node_webapp -->|"authenticates users"| node_auth
node_webapp -->|"caches results"| node_cache
node_webapp -->|"supports network analysis"| node_addressblocks

click node_pipeline "https://github.com/anonghosty/looking-glass/blob/main/database/shadowserver_pipeline.py"
click node_sendercheck "https://github.com/anonghosty/looking-glass/blob/main/database/mail_auth.py"
click node_reportapi "https://github.com/anonghosty/looking-glass/blob/main/database/report_api.py"
click node_workerconfig "https://github.com/anonghosty/looking-glass/blob/main/database/worker_settings.py"
click node_mongo "https://github.com/anonghosty/looking-glass/blob/main/database/db_connect.py"
click node_webapp "https://github.com/anonghosty/looking-glass/blob/main/platform/app.py"
click node_servicesapi "https://github.com/anonghosty/looking-glass/blob/main/platform/services_api.py"
click node_servicesdata "https://github.com/anonghosty/looking-glass/blob/main/platform/services.py"
click node_map "https://github.com/anonghosty/looking-glass/blob/main/platform/attack_map.py"
click node_combos "https://github.com/anonghosty/looking-glass/blob/main/platform/combos.py"
click node_findings "https://github.com/anonghosty/looking-glass/blob/main/platform/findings.py"
click node_intelligence "https://github.com/anonghosty/looking-glass/blob/main/platform/intel.py"
click node_executive "https://github.com/anonghosty/looking-glass/blob/main/platform/quarterly.py"
click node_advisories "https://github.com/anonghosty/looking-glass/blob/main/platform/advisory.py"
click node_audit "https://github.com/anonghosty/looking-glass/blob/main/platform/audit_chain.py"
click node_notes "https://github.com/anonghosty/looking-glass/blob/main/platform/notes.py"
click node_auth "https://github.com/anonghosty/looking-glass/blob/main/platform/auth.py"
click node_privacy "https://github.com/anonghosty/looking-glass/blob/main/platform/privacy.py"
click node_cache "https://github.com/anonghosty/looking-glass/blob/main/platform/respcache.py"
click node_addressblocks "https://github.com/anonghosty/looking-glass/blob/main/platform/address_blocks.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_sendercheck,node_reportapi,node_workerconfig,node_mongo,node_user toneBlue
class node_webapp,node_servicesapi,node_cache toneAmber
class node_servicesdata,node_map,node_combos,node_findings,node_intelligence,node_executive,node_addressblocks,node_shadowserver toneMint
class node_advisories,node_audit,node_notes toneRose
class node_auth,node_privacy toneIndigo
```

## Components

### Report ingestion

- [Report pipeline](https://github.com/anonghosty/looking-glass/blob/main/database/shadowserver_pipeline.py): `database/shadowserver_pipeline.py`
- [Sender verification](https://github.com/anonghosty/looking-glass/blob/main/database/mail_auth.py): `database/mail_auth.py`
- [Report API](https://github.com/anonghosty/looking-glass/blob/main/database/report_api.py): `database/report_api.py`
- [Pipeline settings](https://github.com/anonghosty/looking-glass/blob/main/database/worker_settings.py): `database/worker_settings.py`
- [Shared reports DB](https://github.com/anonghosty/looking-glass/blob/main/database/db_connect.py): `database/db_connect.py`

### Web platform

- [Web application](https://github.com/anonghosty/looking-glass/blob/main/platform/app.py): `platform/app.py`
- [Services API](https://github.com/anonghosty/looking-glass/blob/main/platform/services_api.py): `platform/services_api.py`
- [Shared result cache](https://github.com/anonghosty/looking-glass/blob/main/platform/respcache.py): `platform/respcache.py`

### Analysis and intelligence

- [Services analysis](https://github.com/anonghosty/looking-glass/blob/main/platform/services.py): `platform/services.py`
- [Communications map](https://github.com/anonghosty/looking-glass/blob/main/platform/attack_map.py): `platform/attack_map.py`
- [Combination analysis](https://github.com/anonghosty/looking-glass/blob/main/platform/combos.py): `platform/combos.py`
- [Findings](https://github.com/anonghosty/looking-glass/blob/main/platform/findings.py): `platform/findings.py`
- [Threat intelligence](https://github.com/anonghosty/looking-glass/blob/main/platform/intel.py): `platform/intel.py`
- [Executive reports](https://github.com/anonghosty/looking-glass/blob/main/platform/quarterly.py): `platform/quarterly.py`
- [Address analysis](https://github.com/anonghosty/looking-glass/blob/main/platform/address_blocks.py): `platform/address_blocks.py`

### CSIRT response

- [Advisories](https://github.com/anonghosty/looking-glass/blob/main/platform/advisory.py): `platform/advisory.py`
- [Audit trail](https://github.com/anonghosty/looking-glass/blob/main/platform/audit_chain.py): `platform/audit_chain.py`
- [Research notes](https://github.com/anonghosty/looking-glass/blob/main/platform/notes.py): `platform/notes.py`

### Trust and privacy

- [Authentication](https://github.com/anonghosty/looking-glass/blob/main/platform/auth.py): `platform/auth.py`
- [Server-side masking](https://github.com/anonghosty/looking-glass/blob/main/platform/privacy.py): `platform/privacy.py`

### Other components

- **Shadowserver reports**
- **CSIRT users**

## Connections

- Shadowserver reports → Report pipeline: sends reports
- Report pipeline → Sender verification: checks sender (evidence: `database/shadowserver_pipeline.py`)
- Report pipeline → Report API: fetches reports (evidence: `database/shadowserver_pipeline.py`)
- Report pipeline → Shared reports DB: loads reports (evidence: `database/shadowserver_pipeline.py`)
- Pipeline settings → Report pipeline: configures (evidence: `database/worker_settings.py`)
- CSIRT users → Web application: uses platform
- Web application → Shared reports DB: reads reports (evidence: `platform/app.py`)
- Web application → Services API: registers API (evidence: `platform/services_api.py`)
- Services API → Services analysis: queries services (evidence: `platform/services_api.py`)
- Services API → Server-side masking: masks output (evidence: `platform/services_api.py`)
- Web application → Communications map: serves replay map (evidence: `platform/app.py`)
- Web application → Combination analysis: supports analysis (evidence: `platform/app.py`)
- Web application → Findings: shows findings (evidence: `platform/app.py`)
- Web application → Threat intelligence: shows intelligence (evidence: `platform/app.py`)
- Web application → Executive reports: provides portal
- Web application → Advisories: supports advisories (evidence: `platform/app.py`)
- Web application → Audit trail: records activity
- Web application → Research notes: shares notes (evidence: `platform/app.py`)
- Web application → Authentication: authenticates users (evidence: `platform/app.py`)
- Web application → Shared result cache: caches results (evidence: `platform/app.py`)
- Web application → Address analysis: supports network analysis (evidence: `platform/app.py`)
