How lean-dojo/leanagent works
Main Blocks
• Repository Management & Data Extraction: This block (documented in both README and file comments in leanagent.py) covers:
• Repository selection (checking for supported Lean versions)
• Tracing and parsing of Lean files (using functions inside repository management and trace modules)
• Extraction of theorem statements and proofs.
• Dynamic Database Management: Create a clearly labeled block representing the custom JSON-based database that holds:
• Metadata for repositories, theorems, premises, and files.
• Methods for serialization (to_dict/from_dict) and updating through new extractions.
• Curriculum Learning & Progressive Training: Show a pipeline block that:
• Uses complexity analysis (curriculum based on theorem difficulty) to schedule training.
• Dynamically trains the Tactic Generator and Retriever (modules in the generator and retrieval folders) using progressive epochs.
• Integrates evaluation metrics like recall@10.
• The “Prover” module (files in the prover folder like proof_search.py and search_tree.py) using best-first tree search.
• The workflow starting with retrieving relevant premises, generating tactic candidates (using beam search), and then applying Lean for proof checking.
• The feedback mechanism that sends successful proofs back to the Dynamic Database and even generates pull requests.
• Fisher Information Matrix (FIM) Computation for EWC: Include an auxiliary component that is executed (via compute_fisher.py and run_compute_fisher.sh) to compute gradients and scale them for EWC regularization during training.
Out the Relationships & Data Flow
• The Curriculum Learning block (which “schedules” or “retrieves” the proper data splits based on complexity),
• And the retrieval training pipeline.
Architectural Patterns & Technologies
• Modular separation:
• "Generator" for data extraction and dataset generation.
• "Retrieval" for training the retrieval model using BM25 and related models.
• "Prover" for design-specific proof search and tree navigation.
• That the system follows a pipeline/microservices-style architecture within the codebase where each standalone component (with distinct directories) interacts with others via clearly defined interfaces.
• Signal the use of distributed processing tools such as Ray for distributed computation and PyTorch Lightning for multi-GPU training (DDP), which can be shown as annotations or side notes on the training/compute sections.
• You may also denote that external processes (e.g., GitHub API calls, Lean toolchain invocations) are part of the data extraction and proof execution stages.