How trishullab/proof-wala works
Based on our analysis of the file structure and README, ProofWala is a framework designed for multilingual proof data synthesis and automated theorem proving in interactive theorem provers (ITPs) like Coq and Lean. The system incorporates three main components: (1) an interface module that interacts with the ITPs (executing tactics, proof steps, etc.), (2) a proof step generation and training module for creating and training neural models that suggest proof steps, and (3) a parallel proof search module that leverages these models to run efficient, end-to-end proof searches (using Ray for distributed compute).
Main Components
a. User/Client Interface – where commands (e.g., training or proof search) are initiated.
b. ITP Interface Module – interfacing with external theorem-provers (Lean and Coq). Note that this module interacts with the itp-interface repository.
c. Data Collection Layer – managing the standardized multilingual proof step data. This layer includes the storage of training data, prompts, and recorded proof search outputs.
d. Proof Step Generation & Training Module – reading formatted training data, applying neural network models (built with frameworks like PyTorch), and training them.
e. Parallel Proof Search Module – executing end-to-end proof search guided by the trained models. This module handles configuration (via Hydra config files), distributed execution (using Ray), and logging of proof search sessions.
f. External Dependencies & Tools – such as graphviz for visualizing proof trees, and Torch, Hugging Face, and associated packages for deep learning and model inference.
Relationships and Interactions
• The User Interface triggers either training (via proof-wala-train) or proof search (via proof-wala-search).
• The Training Module accesses data from the Data Layer (prompts, proof trees, etc.) and relies on external deep learning libraries (PyTorch; Hugging Face Transformers).
• The ITP Interface Module interacts directly with Lean/Coq toolchains (as described in the README – see build steps in a Lean project folder and Coq directories).
• The Distributed Proof Search Module communicates with an already running Ray cluster (which is started by the init_ray scripts) and uses the configuration files within the "src/proof_wala/main/config" directory.
Architectural Patterns and Design Principles
• Modularity: Each of the main modules (interface, training, and search) is relatively self-contained and interacts via clearly defined configuration files and data formats.
• Extensibility: The system is designed to support cross-lingual proof synthesis, and it allows adding new ITPs or experimental configurations without major rework.
• Distributed Computation: The use of Ray for distributed proof search (parallel processing) is central to the architecture.
• Standardization: The proof data, prompts, and proofs are organized in a standardized way (as evidenced by the clear folder structure in src/proof_wala/data).
Technologies, Frameworks, and Libraries
• Python is clearly the main driver with files like pyproject.toml, requirements.txt, and many .py files.
• PyTorch (for training deep neural networks) and Hugging Face Transformers (for model inference and pipelines).
• Hydra for managing configuration during training and evaluation.
• Ray for distributed execution of proof search tasks.
• Graphviz for visualizing generated proof trees.
• Additional bash scripts (opam_script.sh, per_node_job.sh, etc.) for orchestration on clusters and SLURM/TACC compatibility.
Additional Detail
• Under “Proof Step Generation & Training,” note the role of files like "run_training.py", "training_data_formatter.py", and the various configuration YAML files.
• In “Parallel Proof Search,” mention scripts like "run_proof_search.py" and the importance of the session file (session_latest) for linking Ray nodes.
• That the system supports experiments via multiple benchmark configurations (as seen by the diverse config files in the "benchmark", "eval_settings", "training_settings" folders).