How benlebich/ask-my-pdf-1hxbyzcwbscgzdxz works
Project Type and Purpose
• This is a full-stack Ruby on Rails application supplemented by React/TypeScript components (via Shakapacker).
• Its purpose is to take an input PDF, convert its pages into vector embeddings, store/cache those embeddings, expose a query interface, retrieve relevant pages for a user’s question, and finally produce a “completion” by calling the OpenAI Davinci API.
File-Structure Analysis & Architectural Insights
• Top-level “app” directory follows Rails MVC:
• app/controllers → HTTP endpoints (PagesController, QuestionsController, UsersController)
• app/models → ActiveRecord models (CachedResponse stores embeddings, questions, answers)
• app/views → ERB and React mounts for server-rendered pages
• app/services/pdf_to_embeddings.rb → encapsulates PDF parsing and embedding logic
• JavaScript lives under app/javascript:
• components/*.tsx → React components (AskMyPdf.tsx, Home.tsx)
• packs/*.js → webpack entry points for browser and SSR
• Persistence & config:
• config/database.yml → PostgreSQL (or other) for model storage
• config/storage.yml → local/disk or cloud storage for file uploads
• config/cable.yml → ActionCable WebSockets (unused for core flow unless streaming)
• Docker & Deployment:
• Dockerfile + .dockerignore → containerize Rails + web server (Puma)
• fly.toml → configuration for Fly.io deployment
• Data folder contains precomputed CSV embeddings for offline workflows or seed data.
Key Components & Interactions
A. Client (Web Browser)
B. Rails Web Application (Puma)
1. Controllers (Pages, Questions)
2. Service Layer (PDF→Embeddings)
3. Model Layer (CachedResponse)
4. Views / React Bundles
C. Database (PostgreSQL)
D. File Storage (Local Disk or S3 via ActiveStorage)
E. External API (OpenAI Davinci)
F. Optional: Webpack Dev Server (for local JS hot-reload)
Data-flow & Relationships
• User uploads/selects a PDF via the React UI → HTTP POST to Rails endpoint
• Rails Controller invokes PdfToEmbeddings service:
• Service reads PDF (ActiveStorage), splits pages, calls embedding routine
• Embeddings and page metadata persisted in CachedResponse model → Database
• User submits a query → QuestionsController retrieves top-K relevant embeddings from Database
• Rails assembles context pages and forwards a prompt to OpenAI API → receives completion
• Completion + metadata saved to CachedResponse (cache lookup on same question next time)
• Response JSON rendered back to React UI → displayed to user
Architectural Patterns & Design Principles
• MVC for HTTP and page rendering
• Service Object pattern (pdf_to_embeddings.rb) to isolate business logic
• ActiveRecord for persistence and caching
• Webpacker / Shakapacker for React integration
• External API integration abstracted in service calls
• Docker for environment consistency