# jetbrains-research/coqpilot architecture

> Architecture diagram of the GitHub repository jetbrains-research/coqpilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jetbrains-research/coqpilot
- Repository: https://github.com/jetbrains-research/coqpilot

## Overview

##### Project Type and Purpose
- Type: A full-fledged VS Code extension.
- Purpose: To automate the writing and refinement of Coq proofs by integrating LLM services (such as OpenAI, LM Studio, Grazie, and a predefined tactics service) with Coq’s tooling (via coq-lsp for typechecking and proof validation). It also includes benchmarking features to evaluate generated proofs.

##### Main Components
a. User Interface and VS Code Extension Layer:
– Contains the extension code (see src/extension), responsible for interacting with the user (commands like “perform completion under cursor,” selection-based completions, etc.), displaying logs, and managing settings.
– Handles VS Code–specific integration (e.g., UI components, status indicators, document editors).

b. Core Engine and Business Logic:
– Core proof generation logic (found in src/core and src/benchmark) that parses Coq files, finds “admit.” statements, and prepares contexts for LLM communication.
– Experimentation modules (under src/benchmark/framework/experiment) that manage multi-round fixing and iterative proof generation.
– Context theorems ranker (under src/core/contextTheoremRanker) selecting the relevant parts of a Coq file as context for LLM generation.

c. LLM Service Integration Modules:
– The LLM layer (located in src/llm and its subdirectories) encapsulates various services (OpenAI, LM Studio, Grazie, and a predefined tactics service).
– For each service, there are specific modules (e.g., src/llm/llmServices/openai, src/llm/llmServices/grazie) that handle the API integration, parameter resolution, and processing of generated completions.
– Model configuration settings and multi-round profiles (as detailed in the README) allow dynamic selection and ordering of models.

d. Coq-LSP Integration and Parsing:
– The integration with coq-lsp (see src/coqLsp) allows parsing of Coq source files and typechecking of generated proofs.
– There are dedicated parsers (in src/coqParser and parts of src/benchmark/framework/parseDataset) that process Coq files and extract proof obligations.

e. Subprocess and Utility Modules:
– Several utilities for subprocess management (in src/benchmark/framework/subprocessCalls) handle spawning Coq compilation processes, executing external commands, and managing inter-process communication.
– Generic utility libraries (in src/utils and under various “utils” directories in different modules) provide logging, file and JSON operations, error handling, and asynchronous scheduling.

f. Benchmarking and Reporting:
– Benchmark components (roots in src/benchmark and dataset directories) allow testing the proof generation performance, managing different datasets and experiments.
– Reporting modules (in src/benchmark/framework/reportBuilders) transform experiment outcomes into JSON or other report formats.

##### Relationships and Interactions
– The VS Code Extension (UI) accepts user commands (e.g., “prove all admits”), then channels these requests to the Core Engine.
– The Core Engine parses the active Coq file, extracts contexts, and dispatches individual proof generation tasks.
– Each generation task is forwarded to the LLM Integration Layer where the request is sent (potentially through multiple LLM services as fallback) and the service responses are validated.
– Coq proofs are verified using coq-lsp integration (often via subprocess calls) to ensure type-correctness.
– The benchmarking module may interact with both the core logic and LLM modules to collect performance data and generate reports.
– Utility modules (logging, async scheduling, file processing) support all other components with cross-cutting functionality.

##### Architectural Patterns and Design Principles
– Modular and layered architecture: Each set of functionalities (UI, core logic, LLM services, parsing, benchmarking) is well encapsulated.
– Separation of concerns: The user interface is separated from core proof-generation logic, which in turn is decoupled from LLM-specific integrations.
– Extensibility: The design supports adding or removing LLM services by configuring the associated models, as visible with multiple “models” arrays in the settings.
– Communication through well-defined interfaces: For example, internal protocols for IPC in subprocess calls are defined in src/benchmark/framework/utils/subprocessUtils/ipc.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface Layer"
        A["VS Code Extension"]:::ui
    end

    %% Core Engine Layer
    subgraph "Core Engine Layer"
        B["Core Engine / Business Logic"]:::core
        B1["Context Theorem Ranker"]:::core
        B -->|"uses"| B1
    end

    %% LLM Integration Layer
    subgraph "LLM Service Integration Layer"
        C["LLM Service Integration"]:::llm
        C1["OpenAI Service"]:::llm
        C2["Grazie Service"]:::llm
        C3["LM Studio Service"]:::llm
        C4["Predefined Tactics Service"]:::llm
        C -->|"impl"| C1
        C -->|"impl"| C2
        C -->|"impl"| C3
        C -->|"impl"| C4
    end

    %% Coq-LSP and Parsing Layer
    subgraph "Coq-LSP & Parsing Layer"
        D["Coq-LSP Integration"]:::lsp
        D1["Coq Parser"]:::lsp
        D2["Dataset Parser"]:::lsp
        D -->|"parses"| D1
        D -->|"parses"| D2
    end

    %% Benchmarking & Reporting Layer
    subgraph "Benchmarking & Reporting Layer"
        E["Experimentation & Benchmarking Modules"]:::bench
        F["Benchmark Reporting"]:::bench
        E -->|"reports"| F
    end

    %% Utility Modules Layer
    subgraph "Utility Modules Layer"
        G["Subprocess Calls"]:::util
        H["Utility Libraries"]:::util
    end

    %% Connections between layers
    A -->|"commands"| B
    B -->|"LLM task"| C
    C -->|"returns"| B
    B -->|"proof-check"| D
    D -->|"validation"| B
    B -->|"experiment"| E
    B -->|"utility"| G
    B -->|"utility"| H

    %% Class definitions for colours
    classDef ui fill:#FAD7A0,stroke:#333,stroke-width:2px;
    classDef core fill:#AED6F1,stroke:#333,stroke-width:2px;
    classDef llm fill:#A9DFBF,stroke:#333,stroke-width:2px;
    classDef lsp fill:#F5B7B1,stroke:#333,stroke-width:2px;
    classDef bench fill:#F9E79F,stroke:#333,stroke-width:2px;
    classDef util fill:#D2B4DE,stroke:#333,stroke-width:2px;

    %% Click Events
    click A "https://github.com/jetbrains-research/coqpilot/tree/main/src/extension"
    click B "https://github.com/jetbrains-research/coqpilot/tree/main/src/core"
    click B1 "https://github.com/jetbrains-research/coqpilot/tree/main/src/core/contextTheoremRanker"
    click C "https://github.com/jetbrains-research/coqpilot/tree/main/src/llm"
    click C1 "https://github.com/jetbrains-research/coqpilot/tree/main/src/llm/llmServices/openai"
    click C2 "https://github.com/jetbrains-research/coqpilot/tree/main/src/llm/llmServices/grazie"
    click C3 "https://github.com/jetbrains-research/coqpilot/tree/main/src/llm/llmServices/lmStudio"
    click C4 "https://github.com/jetbrains-research/coqpilot/tree/main/src/llm/llmServices/predefinedProofs"
    click D "https://github.com/jetbrains-research/coqpilot/tree/main/src/coqLsp"
    click D1 "https://github.com/jetbrains-research/coqpilot/tree/main/src/coqParser"
    click D2 "https://github.com/jetbrains-research/coqpilot/tree/main/src/benchmark/framework/parseDataset"
    click E "https://github.com/jetbrains-research/coqpilot/tree/main/src/benchmark/framework/experiment"
    click E "https://github.com/jetbrains-research/coqpilot/tree/main/src/benchmark/framework/benchmarkingCore"
    click F "https://github.com/jetbrains-research/coqpilot/tree/main/src/benchmark/framework/reportBuilders"
    click G "https://github.com/jetbrains-research/coqpilot/tree/main/src/benchmark/framework/subprocessCalls"
    click H "https://github.com/jetbrains-research/coqpilot/tree/main/src/utils"
```
