How jmreddy2106/gendeptrace_benchmark works
GenDepTrace is a dependency-governance research pipeline: users build or load benchmark tasks, generate code with a deterministic or local HuggingFace model, then evaluate extracted dependency claims and produce admission decisions plus SBOM and provenance records. Its central evaluation stage resolves claims against package registries, checks resolved versions with OSV, scores risk, and applies policy. The repository also provides real-SCA baseline wrappers and experiment statistics. Relationships among evaluation internals follow its imports and the README; unsampled benchmark and reporting internals are represented only at their supported module paths.
Main entry points: evaluation.py Dependency evaluation · cli.py Command interface · generation.py Code generation · utils.py JSONL and hashing
Components 20
Benchmark and generation
- Command interface
gendepbench/cli.py - Task benchmark
gendepbench/benchmark.py - Code generation
gendepbench/generation.py
Dependency governance
- Dependency evaluation
gendepbench/evaluation.py - Claim extraction
gendepbench/extraction.py - Registry resolution
gendepbench/registries.py - OSV vulnerability check
gendepbench/osv.py - Risk scoring
gendepbench/risk.py - Admission policy
gendepbench/policy.py
Evidence and artifacts
- CycloneDX SBOM
gendepbench/sbom.py - Provenance records
gendepbench/provenance.py - Stage timing
gendepbench/timing.py - JSONL and hashing
gendepbench/utils.py
Experiment analysis
- SCA baselines
gendepbench/baselines.py - Experiment statistics
gendepbench/statistics.py
Other components
- Researcher
- HuggingFace models
- PyPI and npm
- OSV service
- SCA tools
Connections 20
- Researcher → Command interface: runs commands
gendepbench/cli.py - Command interface → Task benchmark: builds tasks
gendepbench/cli.py - Command interface → Code generation: starts generation
gendepbench/cli.py - Code generation → HuggingFace models: loads models
README.md - Command interface → Dependency evaluation: starts evaluation
gendepbench/cli.py - Command interface → Experiment statistics: summarizes runs
gendepbench/cli.py - Dependency evaluation → Claim extraction: extracts claims
gendepbench/evaluation.py - Dependency evaluation → Registry resolution: resolves claims
gendepbench/evaluation.py - Registry resolution → PyPI and npm: queries packages
gendepbench/registries.py - Dependency evaluation → OSV vulnerability check: checks resolved versions
gendepbench/evaluation.py - OSV vulnerability check → OSV service: queries vulnerabilities
gendepbench/osv.py - Dependency evaluation → Risk scoring: computes factors
gendepbench/evaluation.py - Dependency evaluation → Admission policy: decides admission
gendepbench/evaluation.py - Dependency evaluation → CycloneDX SBOM: creates SBOM
gendepbench/evaluation.py - Dependency evaluation → Provenance records: records lineage
gendepbench/evaluation.py - Dependency evaluation → Stage timing: tracks stages
gendepbench/evaluation.py - SCA baselines → SCA tools: invokes scanners
gendepbench/baselines.py - Dependency evaluation → JSONL and hashing: reads and writes data
gendepbench/evaluation.py - Code generation → JSONL and hashing: writes generations
gendepbench/generation.py - Command interface → JSONL and hashing: reads run data
gendepbench/cli.py