How warmjademe/plc_generation_empirical_study works
This repository is a research artifact for evaluating bounded generation of IEC 61131-3 Structured Text, centered on a benchmark and the Evidence-Guided Bounded Synthesis (EGBS) harness. Its principal workflow loads a public requirement and fixed interface, generates bounded candidates, validates them, and records evidence; the full correctness chain includes compilation, formal checks, and runtime tests. It also contains benchmark construction and qualification, comparison methods, and a deployment service. The graph uses documented relationships and indexed imports; unsampled deployment details are represented only where the index supports them.
Main entry points: orchestrator.py Bounded synthesis · validators.py Validator adapters · cli.py Harness CLI · pipeline.py Job pipeline
Components 21
Benchmark
- Task bank
datasets_100/build_dataset.py - Task design
datasets/tools/challenge_tasks.py - Benchmark audit
datasets_100/audit_dataset.py
EGBS harness
- Harness CLI
our_method/src/plc_loop/cli.py - Task loader
our_method/src/plc_loop/dataset.py - Bounded synthesis
our_method/src/plc_loop/orchestrator.py - Model client
our_method/src/plc_loop/client.py - Context builder
our_method/src/plc_loop/context.py - Candidate policy
our_method/src/plc_loop/policy.py - Evidence ledger
our_method/src/plc_loop/ledger.py - Validator adapters
our_method/src/plc_loop/validators.py
Deployment service
- Web API
our_method_Deploy/src/plc_deploy/main.py - Job pipeline
our_method_Deploy/src/plc_deploy/pipeline.py - Job store
our_method_Deploy/src/plc_deploy/store.py
Study comparisons
- External baselines
baseline0_external_runner.py - Internal ablations
ablation0_runner.py - Experiment launchers
orchestrate_datasets100_experiments.py
Other components
- Researcher
- MatIEC compiler
- PLCverif
- OpenPLC runtime
Connections 22
- Researcher → Harness CLI: runs
README.md - Task design → Task bank: supplies tasks
- Researcher → Benchmark audit: audits
README.md - Harness CLI → Task loader: loads task
- Harness CLI → Bounded synthesis: starts run
- Bounded synthesis → Task loader: uses contract
our_method/src/plc_loop/orchestrator.py - Bounded synthesis → Model client: requests candidates
our_method/src/plc_loop/orchestrator.py - Bounded synthesis → Context builder: builds context
our_method/src/plc_loop/orchestrator.py - Bounded synthesis → Candidate policy: selects anchor
our_method/src/plc_loop/orchestrator.py - Bounded synthesis → Evidence ledger: records evidence
our_method/src/plc_loop/orchestrator.py - Bounded synthesis → Validator adapters: validates candidates
our_method/src/plc_loop/orchestrator.py - Validator adapters → Task loader: reads test suites
our_method/src/plc_loop/validators.py - Validator adapters → MatIEC compiler: compiles with
README.md - Validator adapters → PLCverif: checks properties
README.md - Validator adapters → OpenPLC runtime: runs scan tests
README.md - Web API → Job pipeline: dispatches jobs
our_method_Deploy/src/plc_deploy/main.py - Web API → Job store: manages jobs
our_method_Deploy/src/plc_deploy/main.py - Job pipeline → Job store: reads job state
our_method_Deploy/src/plc_deploy/pipeline.py - Job pipeline → Bounded synthesis: runs synthesis
our_method_Deploy/src/plc_deploy/pipeline.py - Researcher → External baselines: runs comparison
README.md - Researcher → Internal ablations: runs controls
README.md - Experiment launchers → External baselines: launches experiments