How donismaelito/quill works
Main Components
• Frontend:
• Use the “src/frontend/quill” directory as the Next.js (React) web UI.
• API endpoints under “src/frontend/quill/src/app/api/…” and clearly show components like chat interfaces, document upload & preview, and layout elements.
• Backend Services:
• Python-based backend: the “src” folder holds multiple modules such as “appeals,” “assessment,” “document_creation,” “document_templating,” and different RAG versions (“rag”, “rag_v2”, “rag_v3”, “rag_v4”).
• The “src/main.py” acts as an entry point to run the backend.
• Integration scripts like “llama3_2_finetune.ipynb” and the experimental “openai_assistant” folder.
• Document Processing & OCR:
• “src/document_creation” and “src/document_templating” contain scripts like “write_pdf.py”, “mistral_ocr.py”, and are responsible for form processing and document creation.
• Integration with OCR tools like Tesseract and poppler, as detailed in the README.
• Retrieval Augmented Generation (RAG) Engine:
• Multiple RAG modules (rag, rag_v2, rag_v3, rag_v4) indicate an evolving system for combining local LLM responses with a vector database.
• Sub-components for vector database storage (e.g., “chroma.sqlite3” in vector_db folders) and JSON-based user information.
• External API & Experimentation Modules:
• “openai_assistant” shows integration with the OpenAI Assistant API.
• Other experiments, such as “llama3.2-vision” and “janus_finetune”, should be represented as optional or experimental services.
• Configuration & Data:
• The “config” directory holds configuration data (config.yaml).
• The “data” and “uploads” directories store test files, user inputs, and form samples.
• Dependency and package management files include “requirements.txt”, “package.json”, “package-lock.json”, and Python packaging files in “src/deepseek”.
Relationships and Interactions
• Backend process modules (e.g., document creation, RAG, and OCR processing) as separate blocks that interact with the vector database and external services.
• Data flows such as:
• User uploads a document via the UI.
• The backend (within “document_creation” or one of the RAG components) processes the document with OCR and LLM calls.
• Data is stored or fetched from a vector database (shown in rag_v2, rag_v3 components).
• Final outputs (e.g., filled PDFs) are sent back to the UI.
• Feedback loops for update modes and persistent storage, as seen in the updated RAG system.
Architectural Patterns & Design Principles
• Modular design: Each “src” subfolder is a distinct module with dedicated responsibilities (e.g., appeals, assessment, templating).
• Separation of concerns:
• The Next.js frontend is entirely separated from the Python backend.
• Document processing, data extraction, and form filling are distinct processes.
• Extensibility: The experimental modules (OpenAI assistant API, Janus, and multiple rag versions) can be depicted as plug-in components that can be enabled or extended.
Relevant Technologies & Frameworks
• Python for backend script execution and document processing.
• Node.js/Next.js for the web UI.
• Deep learning models (R1, LLaVA, GPT-4o, etc.) indicated in the README for LLM tasks.
• Tesseract and poppler for OCR.
• Vector databases (using chroma.sqlite3 files) for the RAG system.
• API integrations (OpenAI Assistant API) as experimental services.
Specific Elements
• Clear Labels: Each component (Frontend, Backend, Document Processing, RAG Engine, External Services, Data/Config) should be labeled.
– Backend logic and processing modules another.
– External services and databases a third distinct style.
• Annotated Dependencies: Optionally include key technologies (e.g., “Next.js”, “Python”, “Tesseract”, “Vector DB”) next to respective blocks.