# smartbugs/smartbugs-wild architecture

> Architecture diagram of the GitHub repository smartbugs/smartbugs-wild, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smartbugs/smartbugs-wild
- Repository: https://github.com/smartbugs/smartbugs-wild

## Overview

This repository is primarily a **data publication and curation pipeline** for a smart-contract dataset, not an application runtime.

1. **Core purpose**
   - The repo packages the “SmartBugs Wild Dataset”: **47,398 unique Ethereum smart contracts** plus associated metadata.
   - The main outputs are contract source files, contract metadata, and derived analysis tables.

2. **Upstream data source**
   - The dataset starts from the **Ethereum blockchain**.
   - Contract addresses were selected using a **Google BigQuery** query over Ethereum tables, filtering for contracts with at least one transaction.

3. **Acquisition boundary**
   - After candidate addresses are identified, the repository relies on **Etherscan** as the external service for fetching:
     - contract source code
     - contract balance information
   - The README explicitly notes that the collection scripts live under `script/`.

4. **Collection scripts**
   - `script/get_contracts.py` is the contract source downloader/collector.
   - `script/get_balance.py` is the balance collector.
   - These scripts are the active data-ingestion layer; they bridge blockchain-derived address lists to Etherscan-backed metadata.

5. **Primary dataset storage**
   - `contracts/` contains the actual contract source files, one Solidity file per contract address, named as `<contract_address>.sol`.
   - This folder is the largest and most important artifact in the repo.

6. **Metadata bundle**
   - `contracts.csv.tar.gz` holds the metadata for all contracts.
   - This is the structured companion to the raw source files, suitable for indexing, filtering, and analysis.

7. **Derived analysis artifacts**
   - `balances.json` stores contract balance results.
   - `duplicates.json` records deduplicated/duplicate identification results.
   - `nb_lines.csv` provides line-count metrics for contracts.
   - `script/all_contract.csv` appears to be a consolidated contract list used by the collection pipeline.

8. **Curation / filtering step**
   - A key transformation in the pipeline is **duplicate removal**.
   - The final dataset is the result of:
     - collecting contract addresses
     - downloading source code
     - filtering out duplicates
   - So the repo represents a curated subset of all available Solidity contracts.

9. **Repository structure as architecture**
   - Inputs: Ethereum blockchain data + Etherscan APIs
   - Processing: SQL address selection + Python collection scripts + deduplication
   - Outputs: source corpus in `contracts/` + metadata/metrics files at repo root
   - This makes the repository a **data pipeline output repository** with scripts for reproducibility.

10. **Operational/tooling implications**
   - The only material runtime/tooling dependencies implied by the repo are:
     - **SQL/BigQuery** for initial selection
     - **Python** for collection scripts
     - **Etherscan** for external metadata/source retrieval
   - No application server, UI, or long-running service is present.

11. **Governance/licensing boundary**
   - The root `LICENSE` applies to the repository generally.
   - Files under `contracts/` are treated separately because they are publicly sourced from Etherscan and may retain original licenses.
   - This is an important boundary if you diagram data provenance and legal constraints.

12. **How to diagram it**
   - Model the repo as a pipeline with three layers:
     - **source selection** from Ethereum/BigQuery
     - **enrichment/download** from Etherscan via Python scripts
     - **curated dataset artifacts** stored locally as source files and CSV/JSON metadata
   - The central asset is the contract corpus; everything else supports collection, filtering, and analysis.

Main entry points: [`contracts`](https://github.com/smartbugs/smartbugs-wild/tree/master/contracts) (contracts), [`script/all_contract.csv`](https://github.com/smartbugs/smartbugs-wild/blob/master/script/all_contract.csv) (all contracts), [`script/get_contracts.py`](https://github.com/smartbugs/smartbugs-wild/blob/master/script/get_contracts.py) (get_contracts), [`script/get_balance.py`](https://github.com/smartbugs/smartbugs-wild/blob/master/script/get_balance.py) (get_balance)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g1["Source selection"]
  node_n1(("Ethereum<br/>blockchain source"))
  node_n2{{"BigQuery filter<br/>SQL selection"}}
  node_n3["all contracts<br/>candidate list<br/>[all_contract.csv]"]
end

subgraph group_g2["Collection"]
  node_n4["get_contracts<br/>Python collector<br/>[get_contracts.py]"]
  node_n5["get_balance<br/>Python collector<br/>[get_balance.py]"]
  node_n6(("Etherscan<br/>external API"))
end

subgraph group_g3["Curated outputs"]
  node_n7[("contracts<br/>source corpus")]
  node_n8[("contracts csv<br/>metadata archive")]
  node_n9["balances<br/>analysis json<br/>[balances.json]"]
  node_n10["duplicates<br/>analysis json<br/>[duplicates.json]"]
  node_n11["line counts<br/>analysis csv<br/>[nb_lines.csv]"]
end

subgraph group_g4["Repo boundary"]
  node_n12["LICENSE"]
end

node_n1 -->|"tables"| node_n2
node_n2 -->|"candidate addresses"| node_n3
node_n3 -->|"input list"| node_n4
node_n3 -->|"input list"| node_n5
node_n4 -->|"source fetch"| node_n6
node_n5 -->|"balance fetch"| node_n6
node_n6 -->|"contract sources"| node_n7
node_n4 -->|"metadata"| node_n8
node_n5 -->|"results"| node_n9
node_n7 -->|"deduplication"| node_n10
node_n7 -->|"metrics"| node_n11
node_n10 -->|"curation"| node_n8
node_n7 -.->|"license boundary"| node_n12

click node_n3 "https://github.com/smartbugs/smartbugs-wild/blob/master/script/all_contract.csv"
click node_n4 "https://github.com/smartbugs/smartbugs-wild/blob/master/script/get_contracts.py"
click node_n5 "https://github.com/smartbugs/smartbugs-wild/blob/master/script/get_balance.py"
click node_n7 "https://github.com/smartbugs/smartbugs-wild/tree/master/contracts"
click node_n8 "https://github.com/smartbugs/smartbugs-wild/blob/master/contracts.csv.tar.gz"
click node_n9 "https://github.com/smartbugs/smartbugs-wild/blob/master/balances.json"
click node_n10 "https://github.com/smartbugs/smartbugs-wild/blob/master/duplicates.json"
click node_n11 "https://github.com/smartbugs/smartbugs-wild/blob/master/nb_lines.csv"
click node_n12 "https://github.com/smartbugs/smartbugs-wild/tree/master/LICENSE"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n1,node_n2,node_n3 toneBlue
class node_n4,node_n5,node_n6 toneAmber
class node_n7,node_n8,node_n9,node_n10,node_n11 toneMint
class node_n12 toneRose
```

## Components

### Source selection

Ethereum data is queried and filtered into a candidate contract list.

- **Ethereum**: blockchain source · Upstream Ethereum contract history and transactions.
- **BigQuery filter**: SQL selection · Query over Ethereum tables to select contracts with transactions.
- [all contracts](https://github.com/smartbugs/smartbugs-wild/blob/master/script/all_contract.csv): candidate list · Consolidated contract address list used by the collection pipeline. · `script/all_contract.csv`

### Collection

Python scripts fetch source code and metadata from Etherscan.

- [get_contracts](https://github.com/smartbugs/smartbugs-wild/blob/master/script/get_contracts.py): Python collector · Downloads contract source code from Etherscan. · `script/get_contracts.py`
- [get_balance](https://github.com/smartbugs/smartbugs-wild/blob/master/script/get_balance.py): Python collector · Fetches contract balance information from Etherscan. · `script/get_balance.py`
- **Etherscan**: external API · External service used for source code and balance retrieval.

### Curated outputs

Final contract corpus and derived analysis artifacts stored in the repo.

- [contracts](https://github.com/smartbugs/smartbugs-wild/tree/master/contracts): source corpus · One Solidity file per contract address; the main dataset asset. · `contracts`
- [contracts csv](https://github.com/smartbugs/smartbugs-wild/blob/master/contracts.csv.tar.gz): metadata archive · Structured metadata bundle for the contract corpus. · `contracts.csv.tar.gz`
- [balances](https://github.com/smartbugs/smartbugs-wild/blob/master/balances.json): analysis json · Contract balance results derived during collection. · `balances.json`
- [duplicates](https://github.com/smartbugs/smartbugs-wild/blob/master/duplicates.json): analysis json · Duplicate identification and deduplication results. · `duplicates.json`
- [line counts](https://github.com/smartbugs/smartbugs-wild/blob/master/nb_lines.csv): analysis csv · Per-contract line-count metrics. · `nb_lines.csv`

### Repo boundary

Repository-level documentation and licensing context.

- [LICENSE](https://github.com/smartbugs/smartbugs-wild/tree/master/LICENSE): license · Repository-wide licensing boundary; contract files may retain source licenses. · `LICENSE`

## Connections

- Ethereum → BigQuery filter: tables · Ethereum blockchain data is queried through BigQuery.
- BigQuery filter → all contracts: candidate addresses · Filtered contract addresses are consolidated for collection.
- all contracts → get_contracts: input list · Collector uses the address list to fetch source code.
- all contracts → get_balance: input list · Balance collector reuses the same contract list.
- get_contracts → Etherscan: source fetch · Downloads Solidity source and related metadata from Etherscan.
- get_balance → Etherscan: balance fetch · Retrieves balance information from Etherscan.
- Etherscan → contracts: contract sources · Fetched source files are stored as the contract corpus.
- get_contracts → contracts csv: metadata · Collection output is packaged into the metadata archive.
- get_balance → balances: results · Balance collection writes the balance table.
- contracts → duplicates: deduplication · Corpus is filtered to identify duplicates and unique contracts.
- contracts → line counts: metrics · Contract sources are measured for line counts.
- duplicates → contracts csv: curation · Duplicate filtering contributes to the curated dataset bundle.
- contracts → LICENSE: license boundary · Repository license applies at the package level; source files may differ.
