# ethereum/discv4-dns-lists architecture

> Architecture diagram of the GitHub repository ethereum/discv4-dns-lists, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ethereum/discv4-dns-lists
- Repository: https://github.com/ethereum/discv4-dns-lists

## Overview

##### Project type and purpose
- This is an open-source tooling project (“discv4-dns-lists”) whose sole purpose is to crawl the Ethereum discovery V4 DHT, produce EIP-1459-compliant node lists in JSON form, and publish those lists under ethdisco.net DNS names.

##### File structure
- Top-level files:
- LICENSE, README.md – documentation/legal
- all.json – the raw crawl output of every discovered node
- Per-zone directories (e.g., all.mainnet.ethdisco.net, snap.sepolia.ethdisco.net, les.hoodi.ethdisco.net, etc.) each contain:
- enrtree-info.json – metadata about the ENR tree for that network/prefix
- nodes.json – the filtered list of node ENRs matching that network and prefix
- These directory names map one-to-one to DNS zones under ethdisco.net (e.g., all.mainnet.ethdisco.net)

##### Key insights from README
- Uses go-ethereum’s devp2p crawler to dump the DHT into all.json
- Filters all.json by the “eth” ENR entry to separate out per-chain (mainnet, sepolia, holesky, hoodi) and per-protocol prefix (all, les, snap) lists
- Publishes the filtered JSON in DNS zones under ethdisco.net

##### Main components & data flow
a. Ethereum Discovery DHT
b. Crawler (Go-Ethereum devp2p tool)
c. Raw output storage (all.json)
d. Filter process (reads all.json → produces per-zone JSON)
e. DNS zone generator/publisher (serves nodes.json & enrtree-info.json at each ethdisco.net subdomain)
f. Consumers (Ethereum clients or tooling querying ethdisco.net for peer lists)

Flow:
DHT ──► Crawler ──► all.json ──► Filter engine ──► { for each prefix+chain: nodes.json + enrtree-info.json } ──► DNS server ──► Clients

##### Architectural patterns & principles
- Pipeline/batch-processing: crawl → transform → publish
- Separation of concerns: distinct steps for crawling, filtering, and publishing
- Configuration-by-convention: directory naming enforces DNS zone structure
- Stateless intermediate steps: filter engine always reads the latest all.json

##### Technologies & frameworks
- Go (go-ethereum devp2p crawler)
- JSON for data interchange
- DNS (zone files or dynamic DNS backend) for publication
- Shell or Go scripts to orchestrate crawl → filter → publish

##### Be detailed but focused
- Don’t break out every single JSON file as its own component; instead show a repeating pattern for “prefix.chain.ethdisco.net.”
- Essence: pipeline of crawling, filtering, publishing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Discovery DHT
    DiscoveryDHT["Discovery DHT"]:::external

    %% Compute Components
    Crawler["Crawler (Go-Ethereum devp2p)"]:::compute
    FilterEngine["Filter Engine"]:::compute

    %% Storage Components
    RawJSON["Raw JSON Store<br/>(all.json)"]:::storage

    %% DNS Components
    DNSPublisher["DNS Publisher"]:::dns

    %% Clients
    Clients["Ethereum Clients / Tooling"]:::external

    %% Pipeline Flow
    DiscoveryDHT -->|crawls nodes| Crawler
    Crawler -->|writes all.json| RawJSON
    RawJSON -->|reads all.json| FilterEngine
    FilterEngine -->|emits nodes.json & enrtree-info.json| DNSZoneDirs
    DNSZoneDirs -->|served via DNS| DNSPublisher
    DNSPublisher -->|responds with JSON| Clients

    %% DNS Zone Directories Cluster
    subgraph "DNS Zone Directories"
        direction TB
        subgraph "all.mainnet.ethdisco.net"
            enr_all_mainnet["enrtree-info.json"]:::storage
            nodes_all_mainnet["nodes.json"]:::storage
        end
        subgraph "all.sepolia.ethdisco.net"
            enr_all_sepolia["enrtree-info.json"]:::storage
            nodes_all_sepolia["nodes.json"]:::storage
        end
        subgraph "all.holesky.ethdisco.net"
            enr_all_holesky["enrtree-info.json"]:::storage
            nodes_all_holesky["nodes.json"]:::storage
        end
        subgraph "all.hoodi.ethdisco.net"
            enr_all_hoodi["enrtree-info.json"]:::storage
            nodes_all_hoodi["nodes.json"]:::storage
        end
        subgraph "les.mainnet.ethdisco.net"
            enr_les_mainnet["enrtree-info.json"]:::storage
            nodes_les_mainnet["nodes.json"]:::storage
        end
        subgraph "les.sepolia.ethdisco.net"
            enr_les_sepolia["enrtree-info.json"]:::storage
            nodes_les_sepolia["nodes.json"]:::storage
        end
        subgraph "snap.mainnet.ethdisco.net"
            enr_snap_mainnet["enrtree-info.json"]:::storage
            nodes_snap_mainnet["nodes.json"]:::storage
        end
        subgraph "snap.sepolia.ethdisco.net"
            enr_snap_sepolia["enrtree-info.json"]:::storage
            nodes_snap_sepolia["nodes.json"]:::storage
        end
        subgraph "snap.holesky.ethdisco.net"
            enr_snap_holesky["enrtree-info.json"]:::storage
            nodes_snap_holesky["nodes.json"]:::storage
        end
        subgraph "snap.hoodi.ethdisco.net"
            enr_snap_hoodi["enrtree-info.json"]:::storage
            nodes_snap_hoodi["nodes.json"]:::storage
        end
    end

    %% Documentation
    README["README.md"]:::doc
    LICENSE["LICENSE"]:::doc

    %% Click Events
    click RawJSON "https://github.com/ethereum/discv4-dns-lists/blob/master/all.json"
    click enr_all_mainnet "https://github.com/ethereum/discv4-dns-lists/blob/master/all.mainnet.ethdisco.net/enrtree-info.json"
    click nodes_all_mainnet "https://github.com/ethereum/discv4-dns-lists/blob/master/all.mainnet.ethdisco.net/nodes.json"
    click enr_all_sepolia "https://github.com/ethereum/discv4-dns-lists/blob/master/all.sepolia.ethdisco.net/enrtree-info.json"
    click nodes_all_sepolia "https://github.com/ethereum/discv4-dns-lists/blob/master/all.sepolia.ethdisco.net/nodes.json"
    click enr_all_holesky "https://github.com/ethereum/discv4-dns-lists/blob/master/all.holesky.ethdisco.net/enrtree-info.json"
    click nodes_all_holesky "https://github.com/ethereum/discv4-dns-lists/blob/master/all.holesky.ethdisco.net/nodes.json"
    click enr_all_hoodi "https://github.com/ethereum/discv4-dns-lists/blob/master/all.hoodi.ethdisco.net/enrtree-info.json"
    click nodes_all_hoodi "https://github.com/ethereum/discv4-dns-lists/blob/master/all.hoodi.ethdisco.net/nodes.json"
    click enr_les_mainnet "https://github.com/ethereum/discv4-dns-lists/blob/master/les.mainnet.ethdisco.net/enrtree-info.json"
    click nodes_les_mainnet "https://github.com/ethereum/discv4-dns-lists/blob/master/les.mainnet.ethdisco.net/nodes.json"
    click enr_les_sepolia "https://github.com/ethereum/discv4-dns-lists/blob/master/les.sepolia.ethdisco.net/enrtree-info.json"
    click nodes_les_sepolia "https://github.com/ethereum/discv4-dns-lists/blob/master/les.sepolia.ethdisco.net/nodes.json"
    click enr_snap_mainnet "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.mainnet.ethdisco.net/enrtree-info.json"
    click nodes_snap_mainnet "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.mainnet.ethdisco.net/nodes.json"
    click enr_snap_sepolia "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.sepolia.ethdisco.net/enrtree-info.json"
    click nodes_snap_sepolia "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.sepolia.ethdisco.net/nodes.json"
    click enr_snap_holesky "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.holesky.ethdisco.net/enrtree-info.json"
    click nodes_snap_holesky "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.holesky.ethdisco.net/nodes.json"
    click enr_snap_hoodi "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.hoodi.ethdisco.net/enrtree-info.json"
    click nodes_snap_hoodi "https://github.com/ethereum/discv4-dns-lists/blob/master/snap.hoodi.ethdisco.net/nodes.json"
    click README "https://github.com/ethereum/discv4-dns-lists/blob/master/README.md"
    click LICENSE "https://github.com/ethereum/discv4-dns-lists/tree/master/LICENSE"

    %% Styles
    classDef compute fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef storage fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#F0E68C,stroke:#000,stroke-width:1px
    classDef dns fill:#FFD700,stroke:#000,stroke-width:1px
    classDef doc fill:#D3D3D3,stroke:#000,stroke-width:1px
```
