How git-disl/bert4eth works
Project type and purpose
• This is a research‐oriented machine‐learning pipeline (“BERT4ETH”) for pre-training and fine-tuning a BERT-style Transformer on Ethereum transaction sequences to detect fraud (phishing) and perform de-anonymization tasks.
• Key indicators: a “Data/” folder with raw CSVs, a “Model/” folder containing data-generation and training scripts, and a README describing pre-training, fine-tuning, and evaluation.
File structure
• Top‐level:
• Data/ – raw transaction datasets (ENS pairs, phishing account lists, Tornado cash history)
• Material/ – PDFs and slide decks illustrating model architecture and multi-hop graphs
• Model/ – Python modules for sequence generation (gen_seq.py), pre-training data generation (gen_pretrain_data.py), model definition (modeling.py), optimization & training runners (run_pretrain.py, run_phishing_detection.py, run_dean_ENS.py, etc.), and utility scripts (vocab.py, output_embed.py)
• run.sh – orchestration entry point
• Presence of a JSON config (bert_config.json) signals a parameterized TensorFlow model.
• No frontend; this is purely a data‐processing + ML model codebase.
README for additional insights
• Architecture is a single‐stream pipeline:
a) Download & unzip raw data into Data/
b) Generate tokenized transaction sequences (gen_seq.py)
c) Mask & prepare MLM pre-training data (gen_pretrain_data.py)
d) Pre-train BERT4ETH in TensorFlow (run_pretrain.py)
e) Extract output embeddings (output_embed.py)
f) Downstream tasks: phishing detection (RF & DNN), ENS de-anonymization, Tornado Cash de-anonymization, and fine-tuning for phishing (run_finetune_phisher.py)
• Technologies: Python ≥3.6, TensorFlow 2.x, NumPy; uses in-house negative sampling strategies, cascaded MLP classifiers.