How manoharnaidu/flag-reproduce works
FLAG Reproduction Benchmark is a research benchmark for fraud-detection GNNs, comparing FLAG variants and seven baselines under shared datasets, splits, metrics, and seeds. Its main workflow prepares graph/subgraph and text features, runs configured experiments, then records results for analysis. The graph includes dataset ingestion, semantic and Markov-diffusion sampling, GPU-only LLM enhancement, model/training/evaluation layers, and text-augmented dataset support. Several edges reflect imports or calls listed in the SOURCE INDEX; unsampled subsystems and reporting links are not connected speculatively.
Main entry points: runner.py Experiment runner · cli.py Sampling configuration · run.py Training entrypoint · generate_text.py LLM text generation
Components 23
Data preparation
- GLBench datasets
src/flagbench/datasets/glbench.py - Dataset download
scripts/download/glbench.py - Subgraph preparation
scripts/preprocess/sample_subgraphs.py - Fraud dataset schema
src/flagbench/fraud_text/base.py - Amazon text data
src/flagbench/fraud_text/amazon.py - YelpChi text data
src/flagbench/fraud_text/yelpchi.py - Amazon Video graph
experiments/amazon_video/build_amazon_video.py
Text and sampling
- Sampling configuration
src/flagbench/sampling/cli.py - Semantic sampling
src/flagbench/sampling/semantic.py - Markov diffusion
src/flagbench/sampling/markov_diffusion.py - LLM text generation
scripts/llm/generate_text.py - LLM enhancement
src/flagbench/llm/enhance.py - LLM encoding
scripts/preprocess/encode_llm_text.py
Experiment runtime
- Experiment registry
src/flagbench/registry/registry.py - GNN backbones
src/flagbench/adapters/backbone.py - Model training
src/flagbench/training/trainer.py - Experiment runner
src/flagbench/experiments/runner.py - Training entrypoint
scripts/train/run.py - Device selection
src/flagbench/utils/device.py
Results and extensions
- Classification metrics
src/flagbench/metrics/classification.py - Result records
src/flagbench/experiments/results.py - Run reporting
scripts/analyze/build_main_run_report.py
Other components
- Benchmark user
Connections 29
- Benchmark user → Dataset download: prepares data
README.md - Dataset download → GLBench datasets: downloads dataset
scripts/download/glbench.py - GLBench datasets → Subgraph preparation: provides graph
scripts/preprocess/sample_subgraphs.py - Benchmark user → Subgraph preparation: prepares benchmark
README.md - Subgraph preparation → Sampling configuration: configures sampling
scripts/preprocess/sample_subgraphs.py - Sampling configuration → Semantic sampling: builds sampler
src/flagbench/sampling/cli.py - Sampling configuration → Markov diffusion: selects strategy
src/flagbench/sampling/cli.py - Markov diffusion → Semantic sampling: reuses selection
src/flagbench/sampling/markov_diffusion.py - Benchmark user → LLM text generation: requests text
README.md - LLM text generation → LLM enhancement: generates text
scripts/llm/generate_text.py - LLM text generation → Sampling configuration: uses config
scripts/llm/generate_text.py - LLM encoding → LLM enhancement: uses enhancement
scripts/preprocess/encode_llm_text.py - LLM encoding → Sampling configuration: shares config
scripts/preprocess/encode_llm_text.py - Amazon text data → Fraud dataset schema: uses schema
src/flagbench/fraud_text/amazon.py - YelpChi text data → Fraud dataset schema: uses schema
- Amazon Video graph → Amazon text data: uses dataset logic
experiments/amazon_video/build_amazon_video.py - Benchmark user → Training entrypoint: runs experiments
README.md - Training entrypoint → Experiment runner: runs experiment
scripts/train/run.py - Training entrypoint → Experiment registry: validates choices
scripts/train/run.py - Training entrypoint → Model training: uses trainer
scripts/train/run.py - Training entrypoint → Sampling configuration: configures sampling
scripts/train/run.py - Experiment runner → Experiment registry: resolves models
src/flagbench/experiments/runner.py - Experiment runner → GNN backbones: builds backbone
src/flagbench/experiments/runner.py - Experiment runner → Model training: trains model
src/flagbench/experiments/runner.py - Experiment runner → Classification metrics: evaluates predictions
src/flagbench/experiments/runner.py - Experiment runner → Result records: saves results
src/flagbench/experiments/runner.py - Experiment runner → Device selection: selects device
src/flagbench/experiments/runner.py - Result records → Classification metrics: summarizes metrics
src/flagbench/experiments/results.py - Benchmark user → Run reporting: reviews findings
README.md