How wukan1986/expr_codegen works
Project type and purpose
• expr_codegen is a domain‐specific language (DSL) library for translating mathematical/financial expressions into executable code (Polars, Pandas, etc.), with an optional Streamlit‐based web UI and CLI/scripts for demos.
• It automates expression simplification, common subexpression elimination (CSE), DAG‐based staging/grouping, and emits backend‐specific code via Jinja2 templates.
File structure
• Top‐level:
• streamlit_app.py – the interactive web UI entry point
• codes/, data/, examples/ – utility scripts, data prep, and usage demos
• expr_codegen/ – core library package
• tests/ – unit and performance tests
• CI/CD: .github/, pyproject.toml, requirements.txt
• Core library layout (expr_codegen/):
• expr.py, dag.py, model.py, tool.py – core expression manipulation, DAG builder, and orchestration
• codes.py – high‑level codegen API (e.g. codegen_exec)
• subpackages per backend: pandas, polars_group, polars_over, latex – each contains code generators, printers, Jinja2 templates
README insights
• Describes the workflow: parse expressions (Sympy → simplify → cse) → build DAG → layer & group by category (ts, cs, gp) → render code via Jinja2 templates into target library calls → execute or save.
• Mentions supported frontends (Streamlit UI, direct Python API, demos) and backends (Polars, Pandas, GPU).
Main components and their interactions
• Streamlit UI (streamlit_app.py)
• Python API / demo scripts (codes/demo_exec.py, examples/*.py)
• Expression parser & transformer (expr_codegen/expr.py)
• Simplifier & CSE (via Sympy)
• DAG builder & layer grouper (expr_codegen/dag.py)
• Orchestration tool (expr_codegen/tool.py)
• Polars codegen (expr_codegen/polars)
• Polars_group & Polars_over variants
• Pandas codegen (expr_codegen/pandas)
• LaTeX printer (expr_codegen/latex)
d. Templates & printers
• Jinja2 templates (.j2) per backend
• Custom StrPrinter subclasses to render AST nodes
• Sympy (simplify, cse, StrPrinter)
• Jinja2
• Polars, Pandas, optional Polars_TA operators
f. CI/CD and packaging
• GitHub Actions workflow for publishing
• pyproject.toml, requirements.txt
Architectural patterns & design principles
• Pipeline pattern: expressions flow through parse → simplify → cse → DAG → grouping → codegen
• Plugin/strategy pattern: backend‐specific codegen and printers are pluggable subpackages
• DAG pattern: represents data dependencies, stages code per group
• Separation of concerns: core DSL logic decoupled from rendering templates and UI
2. Beneath, sketch the Core Pipeline left to right:
• Parse & transform (expr.py) → simplify & cse (Sympy) → build DAG (dag.py) → layer/group (tool.py)
5. At the bottom, show Output: generated Python modules or in‐memory DataFrame execution.