# thu-vu92/structured-rag-pdf architecture

> Architecture diagram of the GitHub repository thu-vu92/structured-rag-pdf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thu-vu92/structured-rag-pdf
- Repository: https://github.com/thu-vu92/structured-rag-pdf

## Overview

##### Main Components
- Frontend/UI: Represented by the Streamlit application found in app/streamlit_app.py. This is the user-facing side where users upload PDFs or view processed data.
- Backend / Business Logic: The core processing is implemented in app/functions.py. This component orchestrates the document extraction and communicates with the language models.
- PDF Document Handler: The project includes PDF files in the data folder (as well as a sample in data/Oppenheimer-2006-Applied_Cognitive_Psychology.pdf). This component is responsible for reading and possibly preprocessing the PDF files.
- Language Model Integration: Integration with Langchain and OpenAI (as mentioned in the README) should be depicted, as these external services/modules process the extracted text and transform it, according to your specific logic.
- Development Environment: The .devcontainer folder (with devcontainer.json) indicates a reproducible containerized development setup. Representing this can highlight deployment or development consistency.

##### Relationships and Data Flow
- The Streamlit UI initiates actions (upload/select PDF) and passes user requests to the backend functions.
- The functions module retrieves or accepts the PDF document from the data folder and possibly from user uploads from the frontend.
- The PDF content is then processed: Langchain orchestrates the extraction flow and calls upon OpenAI models to parse or structure data.
- The output is then passed back through the backend to the Streamlit UI for visualization.
- Optionally, show the role of the containerized environment facilitating the deployment/testing of the application.

##### Architectural Patterns and Key Design Elements
- Service Separation: Although the project is relatively simple, separating the frontend (Streamlit) from the backend logic (functions) is important. It maintains a clear separation of concerns.
- Use of External Services: Since the project leverages both Langchain and OpenAI models, make sure to denote these as external API/computation services.
- Containerization: Include an element for the devcontainer which signals that the development (and possibly deployment) is managed via a containerized environment.

## Diagram (Mermaid)

```mermaid
graph TD
    %% User Interaction
    subgraph "User Interaction"
        UI["Streamlit Application"]:::frontend
    end

    %% Processing Layer
    subgraph "Processing Layer"
        BE["Backend / Extraction Functions"]:::backend
        PDF["PDF Document Handler"]:::fileSystem
    end

    %% External Services
    subgraph "External Services"
        LC["Langchain Processor"]:::external
        OA["OpenAI API"]:::external
    end

    %% Development Environment
    subgraph "Development Environment"
        DEV["Containerized Environment"]:::devcontainer
    end

    %% Experimental/Prototyping
    subgraph "Experimental"
        EXP["Experimentation Notebook"]:::prototype
    end

    %% Data Flow
    UI -->|"triggers"| BE
    BE -->|"readsPDF"| PDF
    PDF -->|"PDFcontent"| BE
    BE -->|"callsExtraction"| LC
    LC -->|"processRequest"| OA
    OA -->|"returnResults"| LC
    LC -->|"structuredData"| BE
    BE -->|"outputsData"| UI

    %% Click Events
    click UI "https://github.com/thu-vu92/structured-rag-pdf/blob/main/app/streamlit_app.py"
    click BE "https://github.com/thu-vu92/structured-rag-pdf/blob/main/app/functions.py"
    click PDF "https://github.com/thu-vu92/structured-rag-pdf/tree/main/data"
    click DEV "https://github.com/thu-vu92/structured-rag-pdf/blob/main/.devcontainer/devcontainer.json"
    click EXP "https://github.com/thu-vu92/structured-rag-pdf/blob/main/data_extraction_llms.ipynb"

    %% Styles
    classDef frontend fill:#F9E79F,stroke:#F1C40F,stroke-width:2px;
    classDef backend fill:#D6EAF8,stroke:#2874A6,stroke-width:2px;
    classDef fileSystem fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px;
    classDef external fill:#D5F5E3,stroke:#27AE60,stroke-width:2px;
    classDef devcontainer fill:#FADBD8,stroke:#C0392B,stroke-width:2px;
    classDef prototype fill:#FCF3CF,stroke:#F39C12,stroke-width:2px;
```
