# ezelikman/parsel architecture

> Architecture diagram of the GitHub repository ezelikman/parsel, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ezelikman/parsel
- Repository: https://github.com/ezelikman/parsel

## Overview

##### Project type and purpose
- Project type: a command-line framework (Python)
- Purpose: take a “.ss” program specification (natural language plus tests), use language models (e.g. OpenAI Codex) to synthesize, transpile, and verify code in arbitrary target languages (Python, Lean, VirtualHome, etc.), optionally back-translate and expand/decompose functions.

##### File structure
Top-level files:
– parsel.py: main CLI entry point
– parsify.py: parser for “.ss” specification format → AST
– graph.py: builds dependency graph of functions/tests
– fn.py: function representation & search logic
– codex.py: wrapper around OpenAI API calls
– backtranslate.py: post-processing of generated code into higher-level spec
– execute_virtual_home.py: adapter to run code in VirtualHome simulator
– consts/: language-specific configuration (py_consts, lean_consts, vh_consts)
– programs/: sample input specs (.ss, .lean, .py files)
– requirements.txt, Jupyter notebook (parsel.ipynb)

##### README for additional insights
- CLI usage examples show synthesis flags (–g for auto test generation, –n for natural-language naming, –e for expand, –a for autofill)
- Extensibility: to add a new target language, drop a new consts/XXX_consts.py file and register it in consts/__init__.py
- OpenAI dependency: must supply organization_id:api_key in keys/codex_key.txt

##### Main components & relationships
a. Input subsystem
- File loader: reads .ss spec and test files from programs/ directory
- Parser (parsify.py) → produces an intermediate AST of functions, constraints, NL descriptions

b. Analysis & planning
- Graph builder (graph.py): builds dependency graph among functions/tests
- Function search engine (fn.py): orchestrates combinatorial search over implementations

c. Synthesis & model interaction
- Codex client (codex.py): handles calls to OpenAI’s API using credentials
- Back-translator (backtranslate.py): converts code back into specification or higher-level form, if needed
- Constants modules (consts/*): encode target-language templates, keywords, syntax

d. Execution & verification
- Test harness: dynamically loads generated Python/Lean code, runs unit tests or simulated environment (execute_virtual_home.py for VirtualHome)
- Result checker: validates outputs against constraints, feeds back to search engine

e. Output subsystem
- Final code emitter: writes transpiled code to filesystem (.py, .lean, .ss) and prints logs to CLI
- Notebook interface (parsel.ipynb): optional GUI entry point for exploratory use

##### Architectural patterns & design principles
- Modular/Plugin-based design: separate language constants, synthesis logic, execution adapters
- Pipeline pattern: clear stages—parse → plan/build graph → synthesize (LLM loop) → test → output
- Feedback loop: test results guide iterative LLM calls (constraint-driven search)
- Separation of concerns: parsing, graph analysis, API interaction, execution all isolated

##### Technologies, frameworks & external services
- Python 3.x
- OpenAI Codex API
- Lean theorem prover (optional target)
- VirtualHome simulator (for robotics planning tasks)
- Jupyter Notebook for interactive demos

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI & Input
    subgraph "CLI & Input"
        CLI["CLI: parsel.py"]:::internal
        Programs["Programs (.ss, .py, .lean)"]:::external
        Requirements["requirements.txt"]:::external
        README["README.md"]:::external
    end

    %% Parsing & Analysis
    subgraph "Parsing & Analysis"
        Parser["Parser: parsify.py"]:::internal
        Graph["Graph Builder: graph.py"]:::internal
    end

    %% Synthesis Engine
    subgraph "Synthesis Engine"
        Search["Search Engine: fn.py<br/>(flags: -g,-n,-e,-a)"]:::internal
        subgraph "Model Interaction"
            Codex["Codex Client: codex.py"]:::internal
            BackTrans["Back-Translator: backtranslate.py"]:::internal
        end
    end

    %% Constants & Extensions
    subgraph "Constants & Extensions"
        Consts["Constants Registry: consts/__init__.py"]:::internal
        PyConst["Python Constants: py_consts.py"]:::extension
        LeanConst["Lean Constants: lean_consts.py"]:::extension
        VHConst["VirtualHome Constants: vh_consts.py"]:::extension
    end

    %% Execution & Verification
    subgraph "Execution & Verification"
        TestHarness["Test Harness"]:::internal
        ExecVH["VirtualHome Adapter: execute_virtual_home.py"]:::internal
        ResultChecker["Result Checker"]:::internal
    end

    %% Output & UI
    subgraph "Output & UI"
        Emitter["Code Emitter"]:::internal
        Notebook["Notebook Interface: parsel.ipynb"]:::extension
    end

    %% External Services
    OpenAI["OpenAI API"]:::external
    LeanCLI["Lean CLI"]:::external
    VirtualHome["VirtualHome Simulator"]:::external

    %% Data Flows
    Programs -->|load specs| CLI
    CLI -->|invoke| Parser
    Parser -->|AST| Graph
    Graph -->|dep graph| Search
    Search -->|synthesis request| Codex
    Codex -->|API calls| OpenAI
    Codex -->|raw code| BackTrans
    BackTrans -->|spec back-translate| Search
    Consts -->|templates| Parser
    Consts -->|templates| Codex
    Consts -->|templates| BackTrans
    Search -->|generated code| TestHarness
    TestHarness -->|vh tasks| ExecVH
    ExecVH -->|sim results| TestHarness
    TestHarness -->|results| ResultChecker
    ResultChecker -->|feedback| Search
    TestHarness -->|lean tests| LeanCLI
    ResultChecker -->|pass/fail| Emitter
    Emitter -->|write files| Programs
    CLI -->|interactive| Notebook

    %% Click Events
    click CLI "https://github.com/ezelikman/parsel/blob/main/parsel.py"
    click Parser "https://github.com/ezelikman/parsel/blob/main/parsify.py"
    click Graph "https://github.com/ezelikman/parsel/blob/main/graph.py"
    click Search "https://github.com/ezelikman/parsel/blob/main/fn.py"
    click Codex "https://github.com/ezelikman/parsel/blob/main/codex.py"
    click BackTrans "https://github.com/ezelikman/parsel/blob/main/backtranslate.py"
    click Consts "https://github.com/ezelikman/parsel/blob/main/consts/__init__.py"
    click PyConst "https://github.com/ezelikman/parsel/blob/main/consts/py_consts.py"
    click LeanConst "https://github.com/ezelikman/parsel/blob/main/consts/lean_consts.py"
    click VHConst "https://github.com/ezelikman/parsel/blob/main/consts/vh_consts.py"
    click ExecVH "https://github.com/ezelikman/parsel/blob/main/execute_virtual_home.py"
    click Programs "https://github.com/ezelikman/parsel/tree/main/programs/"
    click Requirements "https://github.com/ezelikman/parsel/blob/main/requirements.txt"
    click README "https://github.com/ezelikman/parsel/blob/main/README.md"
    click Notebook "https://github.com/ezelikman/parsel/blob/main/parsel.ipynb"

    %% Styles
    classDef internal fill:#cce5ff,stroke:#0366d6,stroke-width:2px
    classDef external fill:#d4edda,stroke:#22863a,stroke-width:2px
    classDef extension fill:#f9f9f9,stroke:#6a737d,stroke-width:2px,stroke-dasharray: 5 5
```
