How ucbepic/twix works
Project Type and Purpose
• This is a full-stack application combining:
• A Python package (“twix/…”) that provides core data-extraction APIs (OCR phrase extraction, LLM-based template inference, data extraction, transform, user-feedback hooks).
• An interactive web UI (“twix-ui/…”) built with React (frontend) and a lightweight Python backend (app.py) that exposes REST endpoints to the Python package.
File Structure Analysis
• Top-level:
• twix/ – core library modules (extract.py, transform.py, user_apis.py, models wrappers, etc.)
• twix-ui/ – UI project
– src/components – React components for PDF upload/viewer, processing stages, template editor, data display
– backend/app.py – REST API server
– public/index.html, package.json, tailwind.config.js
• tests/ – Jupyter notebooks demonstrating API usage and data directories illustrating input/output artifacts
• docs/ – documentation and demo assets
• setup.py, twix.egg-info – packaging and metadata
• Architectural cues: MVC-style separation in UI, a layered service in the backend that imports the ‘twix’ library, filesystem-backed storage of intermediate artifacts, and calls out to OpenAI APIs for LLM operations and to OCR engines for phrase extraction.
Insights from README
• Confirms two main user flows:
a) Python package usage via import or CLI (end-to-end API calls).
b) UI playground for interactive editing (npm start + Python backend).
• External dependencies: OpenAI API (LLM) and any OCR tool (vision_feature flag)
• Key features: modular APIs (extract_phrase, predict_template, extract_data, transform, plus template-editing endpoints).
a. Main Components
1. User Interface (Browser/React SPA in twix-ui/src)
2. UI Backend (Python server in twix-ui/backend/app.py)
3. Core Service Layer (twix Python package modules)
4. External Services
• OpenAI LLM API
• OCR Engine (Tesseract or vision-based LLM)
5. Storage Layer (Local filesystem for uploads, intermediate JSON, logs)
6. Tests/Notebooks (Optional: pipeline demonstration clients)
b. Relationships & Data Flow
• User clicks “Upload PDF” → React component → HTTP POST to Backend
• Backend endpoint → Calls twix.extract_phrase / transform / predict_template → interacts with OCR & OpenAI → writes result JSON to filesystem → returns JSON payload to frontend
c. Architectural Patterns & Principles
• Layered architecture: UI layer → API layer → Core business logic → External services
• Clear separation of concerns: React components handle presentation, backend handles orchestration, twix library handles domain logic
• Pluggability/extensibility: model wrappers (twix/models) and user_apis for custom LLM/OCR/field-prediction plugins
d. Key Technologies
• Frontend: React, TailwindCSS, fetch/Axios
• Backend: Python (Flask or FastAPI style in app.py)
• Core: Python 3.10+, twix library modules, OpenAI Python SDK
• Storage: Local disk (uploads/, out/, template.json, etc.)
• Testing & Demos: PyTest or Jupyter notebooks