# wukan1986/expr_codegen architecture

> Architecture diagram of the GitHub repository wukan1986/expr_codegen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wukan1986/expr_codegen
- Repository: https://github.com/wukan1986/expr_codegen

## Overview

##### Project type and purpose
- expr_codegen is a domain‐specific language (DSL) library for translating mathematical/financial expressions into executable code (Polars, Pandas, etc.), with an optional Streamlit‐based web UI and CLI/scripts for demos.
- It automates expression simplification, common subexpression elimination (CSE), DAG‐based staging/grouping, and emits backend‐specific code via Jinja2 templates.

##### File structure
- Top‐level:
- streamlit_app.py – the interactive web UI entry point
- codes/, data/, examples/ – utility scripts, data prep, and usage demos
- expr_codegen/ – core library package
- tests/ – unit and performance tests
- CI/CD: .github/, pyproject.toml, requirements.txt
- Core library layout (expr_codegen/):
- expr.py, dag.py, model.py, tool.py – core expression manipulation, DAG builder, and orchestration
- codes.py – high‑level codegen API (e.g. codegen_exec)
- subpackages per backend: pandas, polars_group, polars_over, latex – each contains code generators, printers, Jinja2 templates

##### README insights
- Describes the workflow: parse expressions (Sympy → simplify → cse) → build DAG → layer & group by category (ts, cs, gp) → render code via Jinja2 templates into target library calls → execute or save.
- Mentions supported frontends (Streamlit UI, direct Python API, demos) and backends (Polars, Pandas, GPU).

##### Main components and their interactions
- Streamlit UI (streamlit_app.py)
- Python API / demo scripts (codes/demo_exec.py, examples/*.py)
- Expression parser & transformer (expr_codegen/expr.py)
- Simplifier & CSE (via Sympy)
- DAG builder & layer grouper (expr_codegen/dag.py)
- Orchestration tool (expr_codegen/tool.py)
- Polars codegen (expr_codegen/polars)
- Polars_group & Polars_over variants
- Pandas codegen (expr_codegen/pandas)
- LaTeX printer (expr_codegen/latex)
d. Templates & printers
- Jinja2 templates (.j2) per backend
- Custom StrPrinter subclasses to render AST nodes
- Sympy (simplify, cse, StrPrinter)
- Jinja2
- Polars, Pandas, optional Polars_TA operators
f. CI/CD and packaging
- GitHub Actions workflow for publishing
- pyproject.toml, requirements.txt

##### Architectural patterns & design principles
- Pipeline pattern: expressions flow through parse → simplify → cse → DAG → grouping → codegen
- Plugin/strategy pattern: backend‐specific codegen and printers are pluggable subpackages
- DAG pattern: represents data dependencies, stages code per group
- Separation of concerns: core DSL logic decoupled from rendering templates and UI

2. Beneath, sketch the Core Pipeline left to right:
- Parse & transform (expr.py) → simplify & cse (Sympy) → build DAG (dag.py) → layer/group (tool.py)
5. At the bottom, show Output: generated Python modules or in‐memory DataFrame execution.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Entry Points"
        UI["Streamlit UI"]:::ui
        CLI["CLI Demo Scripts"]:::ui
        EX["Demo Examples"]:::ui
    end

    subgraph "Core DSL Engine"
        P["Parse & Transform\n(expr.py)"]:::core
        S["Simplify & CSE\n(tool.py + Sympy)"]:::core
        D["Build DAG & Group\n(dag.py)"]:::core
        M["Data Models\n(model.py)"]:::core
        C["High-Level Codegen API\n(codes.py)"]:::core
    end

    subgraph "Backend Plugins"
        subgraph "Pandas Plugin"
            PC["Code Generator\n(code.py)"]:::backend
            PH1["Helpers\n(helper.py)"]:::backend
            PH2["TA Extensions\n(ta.py)"]:::backend
            PP["Printer\n(printer.py)"]:::backend
            PT["Template\n(template.py.j2)"]:::backend
        end
        subgraph "Polars Group Plugin"
            PGC["Code Generator\n(code.py)"]:::backend
            PGP["Printer\n(printer.py)"]:::backend
            PGT["Template\n(template.py.j2)"]:::backend
        end
        subgraph "Polars Over Plugin"
            POC["Code Generator\n(code.py)"]:::backend
            POP["Printer\n(printer.py)"]:::backend
            POT["Template\n(template.py.j2)"]:::backend
        end
        subgraph "LaTeX Plugin"
            LP["Printer\n(printer.py)"]:::backend
        end
    end

    subgraph "External Dependencies"
        ExtSym["Sympy"]:::external
        ExtJ["Jinja2"]:::external
        ExtPan["Pandas"]:::external
        ExtPol["Polars"]:::external
        ExtPTA["Polars_TA"]:::external
    end

    subgraph "Output"
        Out["Generated Code / DataFrame Execution"]:::output
    end

    subgraph "Testing & CI/CD"
        TEST["Unit & Performance Tests"]:::external
        PY["pyproject.toml"]:::external
        REQ["requirements.txt"]:::external
        CI["GitHub Actions\n(publish)"]:::external
    end

    %% Flow
    UI --> P
    CLI --> P
    EX --> P
    P --> S
    S --> D
    D --> C

    C --> PC
    C --> PGC
    C --> POC
    C --> LP

    PC --> Out
    PGC --> Out
    POC --> Out
    LP --> Out

    S --> ExtSym
    PT --> ExtJ
    PGT --> ExtJ
    POT --> ExtJ

    PC --> ExtPan
    PH1 --> ExtPTA
    PH2 --> ExtPTA
    PGC --> ExtPol
    POC --> ExtPol

    %% Click Events
    click UI "https://github.com/wukan1986/expr_codegen/blob/main/streamlit_app.py"
    click CLI "https://github.com/wukan1986/expr_codegen/blob/main/codes/demo_exec.py"
    click EX "https://github.com/wukan1986/expr_codegen/tree/main/examples/"
    click P "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/expr.py"
    click S "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/tool.py"
    click D "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/dag.py"
    click M "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/model.py"
    click C "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/codes.py"
    click PC "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/pandas/code.py"
    click PH1 "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/pandas/helper.py"
    click PH2 "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/pandas/ta.py"
    click PP "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/pandas/printer.py"
    click PT "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/pandas/template.py.j2"
    click PGC "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/polars_group/code.py"
    click PGP "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/polars_group/printer.py"
    click PGT "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/polars_group/template.py.j2"
    click POC "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/polars_over/code.py"
    click POP "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/polars_over/printer.py"
    click POT "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/polars_over/template.py.j2"
    click LP "https://github.com/wukan1986/expr_codegen/blob/main/expr_codegen/latex/printer.py"
    click TEST "https://github.com/wukan1986/expr_codegen/tree/main/tests/"
    click PY "https://github.com/wukan1986/expr_codegen/blob/main/pyproject.toml"
    click REQ "https://github.com/wukan1986/expr_codegen/blob/main/requirements.txt"
    click CI "https://github.com/wukan1986/expr_codegen/blob/main/.github/workflows/python-publish.yml"

    %% Styles
    classDef ui fill:#D0E1F9,stroke:#2E86C1,color:#1B4F72
    classDef core fill:#D1F2D7,stroke:#27AE60,color:#145A32
    classDef backend fill:#FAD7A0,stroke:#E67E22,color:#874918
    classDef external fill:#D5D8DC,stroke:#85929E,color:#566573
    classDef output fill:#E8DAEF,stroke:#884EA0,color:#4A235A
```
