How kevinzwong/pitcher works
Project Type and Purpose
• This project is a multi-component system that combines a set of Python-based utilities (for data extraction, summarization, image and graph generation, and code generation) with a full-stack web application (“pitcher”) built using Next.js.
• The README mentions “Pitcher” along with installation steps for code generation and hints at an “iris” installation, which indicates that the project aims to offer both backend processing (e.g., extracting data, generating visuals, summarizing code) and an interactive, possibly presentation or dashboard-oriented frontend.
File Structure
• At the top level, several Python files (api_server.py, api_server_async.py, drive_extract.py, marp_slides.py, etc.) and directories (el_pitcher, extract, graphs) suggest a backend ecosystem that handles various processing tasks.
• The “el_pitcher” and “extract” folders include modules for pitch generation, text/code extraction, summarization, and retrieval-augmented generation (rag.py). The “graphs” folder, with its numerous scripts and resource folders, indicates dynamic graph/chart generation as part of the functionality.
• The “codegen” directory and its subdirectories suggest a mechanism for automated code modifications, potentially used to scaffold or update project parts.
• The “pitcher” folder, which includes a fully-fledged Next.js application with structured pages, React components, Convex integration, and a modern frontend stack, represents the primary user interface.
• Additional files (like presentation.html, sample_presentation.html, and slides-related scripts) imply that generating and managing presentations might be a central use case.
Architectural Components
a. Backend Processing Layer:
• API Servers: Files such as api_server.py and api_server_async.py indicate REST or asynchronous API endpoints that expose backend functionality.
• Data Extraction and Summarization: The “extract” folder contains modules for chunking, summarizing, and processing text/code data.
• Pitch Generation Core: The “el_pitcher” module includes scripts (pgen.py, pget.py, pitch.py, rag.py) to generate “pitches,” which could be text narratives, slides, or presentations.
• Graph and Image Generation: The “graphs” folder and files like image_gen.py and imagerag.py are responsible for generating visual representations, charts, or imagery.
• Codegen Module: The “.codegen” directory holds tools for automated code generation and codemod functionalities that likely assist in maintaining or extending parts of the system.
b. Frontend (Full-Stack Web Application):
• The “pitcher” folder contains the Next.js project, complete with pages, layouts, components (centralized in subfolders like landing, nav, and ui), and integration with Convex for backend services and potential database storage.
• The app includes multiple interactive screens (dashboard, onboarding, contact, etc.), clearly indicating separation between presentation (landing pages) and administrative or personalized experiences (dashboard, settings).
c. External and Supporting Components:
• Convex Integration: Located within pitcher/convex, this component provides a backend “serverless” integration for data handling, authentication, and API management.
• Iris: The README mentions “iris installation” with an external link, indicating potential support for interoperability or data exchange with an external system.
• Static/Generated Assets: Folders like extracted_images and generated_images hold asset files produced by the processing modules.
Relationships and Interactions
• The frontend (Next.js app) communicates with the backend API (api_server.py/api_server_async.py) to trigger processing tasks and to pull results for display.
• When a “pitch” is requested, the API layer may call modules in “el_pitcher” or “extract” that process content (from slides, textual data, or code) and potentially build on summarization and graph generation capabilities.
• The code generation subsystem (.codegen) might be used for dynamically updating parts of the codebase (or scaffolding new features), which then influences both the backend and frontend components.
• Convex acts as a glue for backend data exchange, handling authentication and serving data to both client and other backend services.
Key Architectural Patterns & Design Principles
• Modular Design: The project is organized into clearly separated modules (extraction, pitch generation, graph/image generation, frontend) that speak to a separation of concerns.
• Full-Stack Separation: There is a clear delineation between Python-based backend processing (APIs, data extraction, utility scripts) and the Next.js frontend.
• Integration & Extensibility: The inclusion of a code generation module and external system links (like Iris) shows an extensible design that can be adapted or automated.
• Asynchronous Processing: The presence of an async API server suggests non-blocking, possibly concurrent handling of processing-intensive tasks.