# aibhuyan/watchphish architecture

> Architecture diagram of the GitHub repository aibhuyan/watchphish, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aibhuyan/watchphish
- Repository: https://github.com/aibhuyan/watchphish

## Overview

1. **High-level shape**
   WatchPhish is a monorepo with two deployable applications and three shared libraries. The runtime architecture is split between a **React frontend** (`artifacts/phishwatch`), an **Express API server** (`artifacts/api-server`), and a **PostgreSQL-backed data layer** (`lib/db`). The shared libraries provide the API contract and generated client/types.

2. **Frontend boundary**
   `artifacts/phishwatch` is a Vite-based React app that serves the operator-facing UI: dashboard, brand monitor, attack library, emerging threats, and phishing simulation experiences. It is mostly a presentation layer and does not own the threat ingestion logic. It consumes the backend through generated API client code from `lib/api-client-react`.

3. **API server boundary**
   `artifacts/api-server` is the central orchestration service. It exposes the HTTP API under `/api/*`, handles collection/enrichment triggers, serves dashboard/feed/brand data, and persists or reads data via the database layer. It is also where scheduled/background threat collection logic lives.

4. **Threat ingestion pipeline**
   The backend `collectors/` directory shows the main external-data ingestion subsystem. It pulls from multiple phishing and threat feeds: **OpenPhish, URLhaus, PhishTank, ThreatFox, VirusTotal, RDAP, and crt.sh/CT logs**. This is the data acquisition boundary from external services into WatchPhish.

5. **Enrichment and classification**
   After raw collection, the server uses utility modules in `src/lib/` such as `enricher`, `classifier`, `typosquat`, and `certstream` to normalize and enrich items. This is where domain intelligence, brand lookalike detection, sector tagging, and threat categorization happen before the data is exposed to clients or stored.

6. **Scheduling and automation**
   `src/lib/scheduler.ts` indicates a background job layer inside the API server. This subsystem likely drives periodic feed collection, enrichment runs, and CT scanning without requiring a user request. The `POST /api/collect`, `POST /api/enrich`, and `POST /api/ct-scan` endpoints appear to be manual triggers for the same automated workflows.

7. **Data model and persistence**
   The `lib/db` package is the shared database schema definition using **Drizzle ORM** for **PostgreSQL**. Its tables model the core persistent entities: feed entries, brand watchlists, certificate alerts, attack types, stats, and API call logs. This library is the persistence contract used by the server and referenced by the broader monorepo.

8. **API contract and generated clients**
   `lib/api-spec/openapi.yaml` defines the REST contract, and `lib/api-zod` plus `lib/api-client-react` are generated artifacts from that spec. This creates a clear boundary: the backend implements the OpenAPI-described endpoints, while the frontend consumes generated, typed request/response helpers. The repo is set up so API shape, validation, and client types stay aligned.

9. **Primary data flows**
   The main flow is: **external phishing/threat sources → API collectors → enrichment/classification → PostgreSQL → API read endpoints → frontend dashboard and detail views**. A second flow exists for the brand monitor: **brand watchlist input → CT log scan / typosquat analysis → cert alerts → dismiss/inspect in UI**. A third flow is the simulation library, which is mostly client-side content and interaction rather than backend-driven threat data.

10. **Backend read vs write paths**
   Most GET endpoints are read paths over persisted or derived data: stats, feed pages, threat details, attack types, emerging threats, brands, and cert alerts. POST/DELETE endpoints mutate the system state: adding/removing brands, dismissing alerts, and kicking off collection/enrichment jobs. That separation is useful for diagramming request flow and storage dependencies.

11. **Frontend UI subsystems**
   The React app is organized into feature sections such as `DashboardSection`, `BrandMonitorSection`, `LibrarySection`, `NewTechniquesSection`, `ThreatDetailPanel`, and `SimulationModal`. These are presentation and interaction modules layered over the API-backed data. The simulation components represent a distinct educational subsystem that does not appear to depend on the ingestion pipeline.

12. **Operational and deployment boundaries**
   The README shows a three-part deployment model: **PostgreSQL on Supabase**, **API server on Render**, and **frontend on Vercel**. That means the system’s runtime dependency chain is frontend → API server → PostgreSQL/external threat services, with the API server also acting as the only component that talks to third-party intelligence sources and scans.

13. **What to emphasize in an architecture diagram**
   The most important boxes are: the **frontend UI**, the **Express API server**, the **collector/enrichment job subsystem**, the **PostgreSQL database**, and the **external threat/intelligence providers**. Around them, show the **shared API spec/generated client layer** as a contract boundary, because it is how the monorepo keeps frontend and backend synchronized.

Main entry points: [`artifacts/api-server/src/app.ts`](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/app.ts) (API Server), [`artifacts/phishwatch/src/App.tsx`](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/App.tsx) (PhishWatch UI), [`lib/db/src/schema/index.ts`](https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/index.ts) (DB Schema), [`artifacts/api-server/src/routes/index.ts`](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/routes/index.ts) (HTTP Routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_phishwatch_app["PhishWatch UI<br/>react app<br/>[App.tsx]"]
  node_dashboard_section["Dashboard<br/>ui section"]
  node_brand_section["Brand Monitor<br/>ui section"]
  node_library_section["Attack Library<br/>ui section<br/>[LibrarySection.tsx]"]
  node_threat_panel["Threat Detail<br/>ui panel"]
  node_simulation_modal["Simulation<br/>ui modal"]
end

subgraph group_backend["API Server"]
  node_api_server["API Server<br/>express service<br/>[app.ts]"]
  node_routes["HTTP Routes<br/>api routes<br/>[index.ts]"]
  node_collectors["Collectors<br/>ingestion pipeline<br/>[openphish.ts]"]
  node_enrichment["Enrichment<br/>analysis pipeline<br/>[enricher.ts]"]
  node_scheduler["Scheduler<br/>job runner<br/>[scheduler.ts]"]
end

subgraph group_shared["Shared Libraries"]
  node_db_schema[("DB Schema<br/>postgres schema<br/>[index.ts]")]
  node_api_spec["API Spec<br/>openapi contract<br/>[openapi.yaml]"]
  node_api_zod["Zod Types<br/>generated validators<br/>[index.ts]"]
  node_api_client["React Client<br/>generated client<br/>[index.ts]"]
  node_phish_entries[("Feed Data<br/>table set<br/>[phishEntries.ts]")]
  node_brand_watchlist[("Brands<br/>table set<br/>[brandWatchlist.ts]")]
  node_cert_alerts[("Cert Alerts<br/>table set<br/>[certAlerts.ts]")]
end

subgraph group_external["External Sources"]
  node_external_feeds(("Threat Feeds<br/>external sources"))
end

node_phishwatch_app -->|"renders"| node_dashboard_section
node_phishwatch_app -->|"renders"| node_brand_section
node_phishwatch_app -->|"renders"| node_library_section
node_phishwatch_app -->|"renders"| node_threat_panel
node_phishwatch_app -->|"renders"| node_simulation_modal
node_phishwatch_app -->|"calls"| node_api_client
node_api_client -.->|"generated from"| node_api_spec
node_api_zod -.->|"generated from"| node_api_spec
node_api_server -->|"hosts"| node_routes
node_routes -->|"implements"| node_api_spec
node_api_server -->|"runs"| node_collectors
node_api_server -->|"runs"| node_enrichment
node_api_server -->|"schedules"| node_scheduler
node_collectors -->|"pulls from"| node_external_feeds
node_enrichment -->|"enriches with"| node_external_feeds
node_routes -->|"reads/writes"| node_db_schema
node_collectors -->|"stores"| node_db_schema
node_enrichment -->|"updates"| node_db_schema
node_db_schema -->|"includes"| node_phish_entries
node_db_schema -->|"includes"| node_brand_watchlist
node_db_schema -->|"includes"| node_cert_alerts
node_routes -->|"served by"| node_api_server
node_api_server -->|"validates"| node_api_zod
node_api_server -.->|"documents"| node_api_spec

click node_phishwatch_app "https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/App.tsx"
click node_dashboard_section "https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/DashboardSection.tsx"
click node_brand_section "https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/BrandMonitorSection.tsx"
click node_library_section "https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/LibrarySection.tsx"
click node_threat_panel "https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/ThreatDetailPanel.tsx"
click node_simulation_modal "https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/SimulationModal.tsx"
click node_api_server "https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/app.ts"
click node_routes "https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/routes/index.ts"
click node_collectors "https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/collectors/openphish.ts"
click node_enrichment "https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/collectors/enricher.ts"
click node_scheduler "https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/lib/scheduler.ts"
click node_db_schema "https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/index.ts"
click node_api_spec "https://github.com/aibhuyan/watchphish/blob/main/lib/api-spec/openapi.yaml"
click node_api_zod "https://github.com/aibhuyan/watchphish/blob/main/lib/api-zod/src/index.ts"
click node_api_client "https://github.com/aibhuyan/watchphish/blob/main/lib/api-client-react/src/index.ts"
click node_phish_entries "https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/phishEntries.ts"
click node_brand_watchlist "https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/brandWatchlist.ts"
click node_cert_alerts "https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/certAlerts.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_phishwatch_app,node_dashboard_section,node_brand_section,node_library_section,node_threat_panel,node_simulation_modal toneBlue
class node_api_server,node_routes,node_collectors,node_enrichment,node_scheduler toneAmber
class node_db_schema,node_api_spec,node_api_zod,node_api_client,node_phish_entries,node_brand_watchlist,node_cert_alerts toneMint
class node_external_feeds toneRose
```

## Components

### Frontend

Operator UI and simulation experience

- [PhishWatch UI](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/App.tsx): react app · Operator-facing dashboard and simulation frontend · `artifacts/phishwatch/src/App.tsx`
- [Dashboard](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/DashboardSection.tsx): ui section · Summary metrics and recent activity view · `artifacts/phishwatch/src/components/DashboardSection.tsx`
- [Brand Monitor](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/BrandMonitorSection.tsx): ui section · Brand watchlist and certificate alert interface · `artifacts/phishwatch/src/components/BrandMonitorSection.tsx`
- [Attack Library](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/LibrarySection.tsx): ui section · Threat library and feed browsing experience · `artifacts/phishwatch/src/components/LibrarySection.tsx`
- [Threat Detail](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/ThreatDetailPanel.tsx): ui panel · Detailed threat inspection and context panel · `artifacts/phishwatch/src/components/ThreatDetailPanel.tsx`
- [Simulation](https://github.com/aibhuyan/watchphish/blob/main/artifacts/phishwatch/src/components/SimulationModal.tsx): ui modal · Phishing simulation interaction layer · `artifacts/phishwatch/src/components/SimulationModal.tsx`

### API Server

HTTP API, collectors, and background jobs

- [API Server](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/app.ts): express service · Central orchestration service for API, collection, and enrichment · `artifacts/api-server/src/app.ts`
- [HTTP Routes](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/routes/index.ts): api routes · REST endpoints for health, brands, phishing, and actions · `artifacts/api-server/src/routes/index.ts`
- [Collectors](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/collectors/openphish.ts): ingestion pipeline · Feed ingestion from external phishing and threat sources · `artifacts/api-server/src/collectors/openphish.ts`
- [Enrichment](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/collectors/enricher.ts): analysis pipeline · Normalization, classification, typosquat, and certstream logic · `artifacts/api-server/src/collectors/enricher.ts`
- [Scheduler](https://github.com/aibhuyan/watchphish/blob/main/artifacts/api-server/src/lib/scheduler.ts): job runner · Background automation for collection and enrichment runs · `artifacts/api-server/src/lib/scheduler.ts`

### Shared Libraries

Contract, client, and persistence packages

- [DB Schema](https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/index.ts): postgres schema · Shared persistence model for feeds, brands, alerts, stats, and logs · `lib/db/src/schema/index.ts`
- [API Spec](https://github.com/aibhuyan/watchphish/blob/main/lib/api-spec/openapi.yaml): openapi contract · Canonical REST contract and request/response shapes · `lib/api-spec/openapi.yaml`
- [Zod Types](https://github.com/aibhuyan/watchphish/blob/main/lib/api-zod/src/index.ts): generated validators · Generated runtime types and validation helpers · `lib/api-zod/src/index.ts`
- [React Client](https://github.com/aibhuyan/watchphish/blob/main/lib/api-client-react/src/index.ts): generated client · Typed API client used by the frontend · `lib/api-client-react/src/index.ts`
- [Feed Data](https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/phishEntries.ts): table set · Persistent phishing feed entries and related records · `lib/db/src/schema/phishEntries.ts`
- [Brands](https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/brandWatchlist.ts): table set · Brand watchlist and operator-managed brand records · `lib/db/src/schema/brandWatchlist.ts`
- [Cert Alerts](https://github.com/aibhuyan/watchphish/blob/main/lib/db/src/schema/certAlerts.ts): table set · Certificate and CT-scan alert records · `lib/db/src/schema/certAlerts.ts`

### External Sources

Third-party threat feeds and intelligence providers

- **Threat Feeds**: external sources · OpenPhish, URLhaus, PhishTank, ThreatFox, VirusTotal, RDAP, crt.sh/CT logs

## Connections

- PhishWatch UI → Dashboard: renders · App composes the dashboard experience
- PhishWatch UI → Brand Monitor: renders · App composes brand monitoring
- PhishWatch UI → Attack Library: renders · App composes the library view
- PhishWatch UI → Threat Detail: renders · App opens detailed threat inspection
- PhishWatch UI → Simulation: renders · App hosts simulation interactions
- PhishWatch UI → React Client: calls · Frontend uses generated typed API helpers
- React Client → API Spec: generated from · Client is derived from the OpenAPI contract
- Zod Types → API Spec: generated from · Validation types are derived from the same contract
- API Server → HTTP Routes: hosts · Express app mounts API route handlers
- HTTP Routes → API Spec: implements · Route surface follows the shared contract
- API Server → Collectors: runs · Server triggers and coordinates ingestion
- API Server → Enrichment: runs · Server normalizes and classifies collected data
- API Server → Scheduler: schedules · Background jobs drive recurring workflows
- Collectors → Threat Feeds: pulls from · Collectors ingest external phishing and threat sources
- Enrichment → Threat Feeds: enriches with · Enrichment may consult external intel and CT data
- HTTP Routes → DB Schema: reads/writes · API endpoints persist and query core entities
- Collectors → DB Schema: stores · Ingested items are written through the shared schema
- Enrichment → DB Schema: updates · Enrichment/classification updates persisted records
- DB Schema → Feed Data: includes · Feed entry storage model
- DB Schema → Brands: includes · Brand monitoring storage model
- DB Schema → Cert Alerts: includes · Certificate alert storage model
- HTTP Routes → API Server: served by · HTTP API is exposed by the server runtime
- API Server → Zod Types: validates · Runtime request/response validation uses generated types
- API Server → API Spec: documents · Spec defines the public API surface