How geekfujiwara/ocr2gpt works
Project Type and Purpose
• Project Type: This is a Power Apps canvas application that integrates Microsoft’s AI Builder components. The app uses AI Builder’s OCR engine to extract text from documents and then leverages GPT (via AI Builder’s formatGPT) to format or extract structured data.
• Purpose: Designed for document processing, the solution uses OCR to “read” document contents (like receipts, invoices, contracts, etc.) and applies prompt templates to extract or reformat key data. It supports multiple document types (e.g., gasoline receipts, family registries, NDAs, invoices, and lab reports) and even handles multiple languages.
Analyzing the File Structure
• The provided file tree only shows README.md, meaning that the repository serves primarily as documentation for users to import the solution via ZIP from the releases page.
• Although there isn’t an expansive code structure in the repository, the README details several functional components: the canvas app itself, AI Builder for OCR and formatGPT, and a data import mechanism (through Excel files for prompt templates).
Key Components and Their Relationships
a. Power Apps Canvas Application
– Represents the main user interface where the end-user interacts with the app.
– It is responsible for triggering the OCR and formatting functionalities.
b. OCR (AI Builder Text Recognition Engine)
– A service that processes input documents to extract their full text.
– This component feeds raw OCR output into the formatting logic.
c. formatGPT (AI Builder 'formatGPT' Service)
– Processes the extracted text to reformat or extract the specific data points as dictated by the prompt templates.
– Demonstrates the integration of GPT-based AI with the OCR output.
d. Prompt Template Management
– Comprised of data objects (tables) that store various pre-defined prompt templates (for receipts, NDAs, invoices, etc.).
– Templates can be dynamically updated or registered, and the app can import sample data via Excel.
e. Data Import & Storage
– Highlight the mechanism where sample data is imported from Excel into the app’s data store.
– This facilitates the mapping of template data which is later used by the formatting service.
f. External Services or Downstream Integration
– Although not overly detailed in the README, indicate that the output (often in JSON format) may be consumed by external systems.
Architectural Relationships and Data Flow
• Finally, after processing, the final output (formatted JSON or structured data) is made available in the canvas app, first for display or further downstream integration.
Architectural Patterns and Design Principles
• Separation of Concerns: Each major function (OCR extraction, text formatting, template management) is encapsulated in its own component.
• Extensibility: Note that new prompt templates can be added, and the solution is designed to easily extend its capabilities to other document types.
• Integration Focus: Use of Microsoft Power Platform and AI Builder indicates a service-oriented design with components provided by external cloud services.
Essential Elements
• Clear component labels (e.g., “Power Apps Canvas App”, “OCR Engine: AI Builder”, “Text Formatter: formatGPT”, “Prompt Template Store”).