# ucbepic/twix architecture

> Architecture diagram of the GitHub repository ucbepic/twix, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ucbepic/twix
- Repository: https://github.com/ucbepic/twix

## Overview

##### Project Type and Purpose
- This is a full-stack application combining:
- A Python package (“twix/…”) that provides core data-extraction APIs (OCR phrase extraction, LLM-based template inference, data extraction, transform, user-feedback hooks).
- An interactive web UI (“twix-ui/…”) built with React (frontend) and a lightweight Python backend (app.py) that exposes REST endpoints to the Python package.

##### File Structure Analysis
- Top-level:
- twix/ – core library modules (extract.py, transform.py, user_apis.py, models wrappers, etc.)
- twix-ui/ – UI project
– src/components – React components for PDF upload/viewer, processing stages, template editor, data display
– backend/app.py – REST API server
– public/index.html, package.json, tailwind.config.js
- tests/ – Jupyter notebooks demonstrating API usage and data directories illustrating input/output artifacts
- docs/ – documentation and demo assets
- setup.py, twix.egg-info – packaging and metadata
- Architectural cues: MVC-style separation in UI, a layered service in the backend that imports the ‘twix’ library, filesystem-backed storage of intermediate artifacts, and calls out to OpenAI APIs for LLM operations and to OCR engines for phrase extraction.

##### Insights from README
- Confirms two main user flows:
a) Python package usage via import or CLI (end-to-end API calls).
b) UI playground for interactive editing (npm start + Python backend).
- External dependencies: OpenAI API (LLM) and any OCR tool (vision_feature flag)
- Key features: modular APIs (extract_phrase, predict_template, extract_data, transform, plus template-editing endpoints).

a. Main Components
1. User Interface (Browser/React SPA in twix-ui/src)
2. UI Backend (Python server in twix-ui/backend/app.py)
3. Core Service Layer (twix Python package modules)
4. External Services
- OpenAI LLM API
- OCR Engine (Tesseract or vision-based LLM)
5. Storage Layer (Local filesystem for uploads, intermediate JSON, logs)
6. Tests/Notebooks (Optional: pipeline demonstration clients)
b. Relationships & Data Flow
- User clicks “Upload PDF” → React component → HTTP POST to Backend
- Backend endpoint → Calls twix.extract_phrase / transform / predict_template → interacts with OCR & OpenAI → writes result JSON to filesystem → returns JSON payload to frontend
c. Architectural Patterns & Principles
- Layered architecture: UI layer → API layer → Core business logic → External services
- Clear separation of concerns: React components handle presentation, backend handles orchestration, twix library handles domain logic
- Pluggability/extensibility: model wrappers (twix/models) and user_apis for custom LLM/OCR/field-prediction plugins
d. Key Technologies
- Frontend: React, TailwindCSS, fetch/Axios
- Backend: Python (Flask or FastAPI style in app.py)
- Core: Python 3.10+, twix library modules, OpenAI Python SDK
- Storage: Local disk (uploads/, out/, template.json, etc.)
- Testing & Demos: PyTest or Jupyter notebooks

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    subgraph "User Interface (Browser)" 
        direction TB
        Browser["React SPA"]:::frontend
        ComponentsDir["components"]:::frontend
        Hooks["usePDFProcessor.js"]:::frontend
        Services["api.js"]:::frontend
        AppJS["App.js"]:::frontend
        IndexJS["index.js"]:::frontend
        HTML["index.html"]:::frontend
    end

    %% API Layer
    subgraph "API Server"
        direction TB
        API["API Server (app.py)"]:::backend
    end

    %% Core Library Layer
    subgraph "twix Library" 
        direction TB
        Extract["extract.py"]:::core
        Transform["transform.py"]:::core
        UserAPIs["user_apis.py"]:::core
        Model["model.py"]:::core
        Pattern["pattern.py"]:::core
        Key["key.py"]:::core
        Cost["cost.py"]:::core
        subgraph "Model Wrappers"
            direction TB
            GPT4O["gpt_4o.py"]:::core
            GPT4OMini["gpt_4o_mini.py"]:::core
            Vision["gpt_4o_vision.py"]:::core
            VisionMini["gpt_4o_mini_vision.py"]:::core
        end
    end

    %% External Services
    subgraph "External Services"
        direction TB
        OpenAI["OpenAI API"]:::external
        OCR["OCR Engine"]:::external
    end

    %% Storage Layer
    subgraph "Filesystem Storage"
        direction TB
        Uploads["twix-ui/uploads"]:::storage
        Out["twix-ui/out"]:::storage
        TestsIn["tests/data"]:::storage
        TestsOut["tests/out"]:::storage
    end

    %% Test Clients
    subgraph "Test Clients (Notebooks)"
        direction TB
        Notebook1["test_twix.ipynb"]:::test
        Notebook2["test_twix_large.ipynb"]:::test
        Notebook3["test_twix_transform.ipynb"]:::test
        Notebook4["test_twix_user_apis.ipynb"]:::test
    end

    %% Documentation
    Docs["docs/"]:::doc

    %% Packaging
    Setup["setup.py"]:::packaging
    EggInfo["twix.egg-info/"]:::packaging

    %% Connections
    Browser -->|"HTTP requests"| API
    API -->|"calls functions"| Extract
    API --> Extract
    API --> Transform
    API --> UserAPIs
    Extract -->|"calls"| OCR
    Extract -->|"writes/reads"| Uploads
    Transform -->|"writes/reads"| Out
    UserAPIs -->|"reads/writes template"| Out
    Extract -->|"calls"| OpenAI
    Model --> OpenAI
    Pattern --> OpenAI
    UserAPIs --> OpenAI
    Extract --> TestsIn
    Tests --> Extract
    Tests --> Transform
    Tests --> UserAPIs
    API -->|"serves files"| Out
    API -->|"serves uploads"| Uploads
    Browser -->|"displays results"| Out
    Browser -->|"uploads"| Uploads

    %% Click Events
    click ComponentsDir "https://github.com/ucbepic/twix/tree/main/twix-ui/src/components"
    click Hooks "https://github.com/ucbepic/twix/blob/main/twix-ui/src/hooks/usePDFProcessor.js"
    click Services "https://github.com/ucbepic/twix/blob/main/twix-ui/src/services/api.js"
    click AppJS "https://github.com/ucbepic/twix/blob/main/twix-ui/src/App.js"
    click IndexJS "https://github.com/ucbepic/twix/blob/main/twix-ui/src/index.js"
    click HTML "https://github.com/ucbepic/twix/blob/main/twix-ui/src/index.html"
    click API "https://github.com/ucbepic/twix/blob/main/twix-ui/backend/app.py"
    click Extract "https://github.com/ucbepic/twix/blob/main/twix/extract.py"
    click Transform "https://github.com/ucbepic/twix/blob/main/twix/transform.py"
    click UserAPIs "https://github.com/ucbepic/twix/blob/main/twix/user_apis.py"
    click Model "https://github.com/ucbepic/twix/blob/main/twix/model.py"
    click Pattern "https://github.com/ucbepic/twix/blob/main/twix/pattern.py"
    click Key "https://github.com/ucbepic/twix/blob/main/twix/key.py"
    click Cost "https://github.com/ucbepic/twix/blob/main/twix/cost.py"
    click GPT4O "https://github.com/ucbepic/twix/blob/main/twix/models/gpt_4o.py"
    click GPT4OMini "https://github.com/ucbepic/twix/blob/main/twix/models/gpt_4o_mini.py"
    click Vision "https://github.com/ucbepic/twix/blob/main/twix/models/gpt_4o_vision.py"
    click VisionMini "https://github.com/ucbepic/twix/blob/main/twix/models/gpt_4o_mini_vision.py"
    click Uploads "https://github.com/ucbepic/twix/tree/main/twix-ui/uploads"
    click Out "https://github.com/ucbepic/twix/tree/main/twix-ui/out"
    click TestsIn "https://github.com/ucbepic/twix/tree/main/tests/data"
    click TestsOut "https://github.com/ucbepic/twix/tree/main/tests/out"
    click Notebook1 "https://github.com/ucbepic/twix/blob/main/tests/test_twix.ipynb"
    click Notebook2 "https://github.com/ucbepic/twix/blob/main/tests/test_twix_large.ipynb"
    click Notebook3 "https://github.com/ucbepic/twix/blob/main/tests/test_twix_transform.ipynb"
    click Notebook4 "https://github.com/ucbepic/twix/blob/main/tests/test_twix_user_apis.ipynb"
    click Docs "https://github.com/ucbepic/twix/tree/main/docs/"
    click Setup "https://github.com/ucbepic/twix/blob/main/setup.py"
    click EggInfo "https://github.com/ucbepic/twix/tree/main/twix.egg-info/"

    %% Styles
    classDef frontend fill:#DCEEFB,stroke:#1E40AF,color:#1E40AF;
    classDef backend fill:#E0F2FE,stroke:#0369A1,color:#0369A1;
    classDef core fill:#ECFDF5,stroke:#059669,color:#059669;
    classDef external fill:#E6FFFA,stroke:#14B8A6,color:#0D9488,stroke-dasharray: 5 5;
    classDef storage fill:#FEF9C3,stroke:#CA8A04,color:#92400E,stroke-dasharray: 5 5;
    classDef test fill:#FFF7ED,stroke:#F97316,color:#C2410C;
    classDef doc fill:#F3F4F6,stroke:#6B7280,color:#374151;
    classDef packaging fill:#FCE7F3,stroke:#DB2777,color:#9D174D;
```
