How ezelikman/parsel works
Project type and purpose
• Project type: a command-line framework (Python)
• Purpose: take a “.ss” program specification (natural language plus tests), use language models (e.g. OpenAI Codex) to synthesize, transpile, and verify code in arbitrary target languages (Python, Lean, VirtualHome, etc.), optionally back-translate and expand/decompose functions.
File structure
Top-level files:
– parsel.py: main CLI entry point
– parsify.py: parser for “.ss” specification format → AST
– graph.py: builds dependency graph of functions/tests
– fn.py: function representation & search logic
– codex.py: wrapper around OpenAI API calls
– backtranslate.py: post-processing of generated code into higher-level spec
– execute_virtual_home.py: adapter to run code in VirtualHome simulator
– consts/: language-specific configuration (py_consts, lean_consts, vh_consts)
– programs/: sample input specs (.ss, .lean, .py files)
– requirements.txt, Jupyter notebook (parsel.ipynb)
README for additional insights
• CLI usage examples show synthesis flags (–g for auto test generation, –n for natural-language naming, –e for expand, –a for autofill)
• Extensibility: to add a new target language, drop a new consts/XXX_consts.py file and register it in consts/__init__.py
• OpenAI dependency: must supply organization_id:api_key in keys/codex_key.txt
Main components & relationships
a. Input subsystem
• File loader: reads .ss spec and test files from programs/ directory
• Parser (parsify.py) → produces an intermediate AST of functions, constraints, NL descriptions
b. Analysis & planning
• Graph builder (graph.py): builds dependency graph among functions/tests
• Function search engine (fn.py): orchestrates combinatorial search over implementations
c. Synthesis & model interaction
• Codex client (codex.py): handles calls to OpenAI’s API using credentials
• Back-translator (backtranslate.py): converts code back into specification or higher-level form, if needed
• Constants modules (consts/*): encode target-language templates, keywords, syntax
d. Execution & verification
• Test harness: dynamically loads generated Python/Lean code, runs unit tests or simulated environment (execute_virtual_home.py for VirtualHome)
• Result checker: validates outputs against constraints, feeds back to search engine
e. Output subsystem
• Final code emitter: writes transpiled code to filesystem (.py, .lean, .ss) and prints logs to CLI
• Notebook interface (parsel.ipynb): optional GUI entry point for exploratory use
Architectural patterns & design principles
• Modular/Plugin-based design: separate language constants, synthesis logic, execution adapters
• Pipeline pattern: clear stages—parse → plan/build graph → synthesize (LLM loop) → test → output
• Feedback loop: test results guide iterative LLM calls (constraint-driven search)
• Separation of concerns: parsing, graph analysis, API interaction, execution all isolated
Technologies, frameworks & external services
• Python 3.x
• OpenAI Codex API
• Lean theorem prover (optional target)
• VirtualHome simulator (for robotics planning tasks)
• Jupyter Notebook for interactive demos