# comais-labs/busca_documentos architecture

> Architecture diagram of the GitHub repository comais-labs/busca_documentos, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/comais-labs/busca_documentos
- Repository: https://github.com/comais-labs/busca_documentos

## Overview

##### Main Components

a. User Interface Layer:
- Streamlit Web Interface
- Upload Component
- Search Interface
- Document Download Component

b. Processing Layer:
- Document Processor
- PDF Processor (PyMuPDF)
- DOCX Processor (python-docx)
- Text Extraction Pipeline

c. Embedding Layer:
- Sentence Transformer Model
- (paraphrase-multilingual-mpnet-base-v2)
- CUDA Acceleration (optional component)

d. Search Layer:
- FAISS Vector Store
- LangChain Search Pipeline

e. Storage Layer:
- Document Storage
- Embedding Storage

##### Flow Representation

- User → Streamlit Interface
- Streamlit Interface → Document Processor (upload flow)
- Document Processor → Text Extraction
- Text Extraction → Embedding Generation
- Embedding Generation → FAISS Storage
- Search Query → Embedding Generation
- FAISS Storage → Search Results
- Search Results → User Interface

##### Specific Design Elements

- Rectangles: Main components
- Hexagons: Processing units
- Rounded rectangles: User interface elements

- Blue: User interface components
- Green: Processing components
- Yellow: Storage components
- Purple: ML/AI components (embedding model)
- Gray: External dependencies

##### Important Details

- Data transformation points
- Clear labeling of technologies used

##### Technical Details to Note

- Python 3.11 requirement
- LangChain's role in orchestrating operations
- FAISS's vector storage and retrieval mechanism
- Demonstrate the multilingual capability of the embedding model

Remember to include a version indicator and any relevant notes about scaling considerations or performance optimizations, particularly regarding the GPU acceleration option.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph UI["User Interface Layer"]
        UI_MAIN["Streamlit Web Interface"]:::ui
        UPLOAD["Upload Component"]:::ui
        SEARCH["Search Interface"]:::ui
        DOWNLOAD["Document Download"]:::ui
    end

    subgraph PROC["Processing Layer"]
        DOC_PROC{"Document Processor"}:::processing
        PDF["PDF Processor<br/>(PyMuPDF)"]:::processing
        DOCX["DOCX Processor<br/>(python-docx)"]:::processing
        TEXT_EX["Text Extraction<br/>Pipeline"]:::processing
    end

    subgraph EMB["Embedding Layer"]
        TRANS["Sentence Transformer<br/>(multilingual-mpnet)"]:::ml
        CUDA["CUDA Acceleration"]:::tech
    end

    subgraph SEARCH["Search Layer"]
        FAISS[(FAISS Vector<br/>Store)]:::storage
        LANG["LangChain Search<br/>Pipeline"]:::processing
    end

    subgraph STORE["Storage Layer"]
        DOC_STORE[(Document<br/>Storage)]:::storage
        EMB_STORE[(Embedding<br/>Storage)]:::storage
    end

    %% Connections
    USER((User)) --> UI_MAIN
    UI_MAIN --> UPLOAD
    UI_MAIN --> SEARCH
    UI_MAIN --> DOWNLOAD

    UPLOAD --> DOC_PROC
    DOC_PROC --> PDF
    DOC_PROC --> DOCX
    PDF --> TEXT_EX
    DOCX --> TEXT_EX
    TEXT_EX --> TRANS
    CUDA -.-> TRANS
    TRANS --> FAISS
    TRANS --> EMB_STORE
    DOC_PROC --> DOC_STORE

    SEARCH --> TRANS
    FAISS --> LANG
    LANG --> UI_MAIN

    %% Click Events
    click UI_MAIN "https://github.com/comais-labs/busca_documentos/blob/main/main.py"
    click UPLOAD "https://github.com/comais-labs/busca_documentos/blob/main/main.py"
    click SEARCH "https://github.com/comais-labs/busca_documentos/blob/main/main.py"
    click DOWNLOAD "https://github.com/comais-labs/busca_documentos/blob/main/main.py"

    %% Styling
    classDef ui fill:#3498db,stroke:#2980b9,color:white
    classDef processing fill:#2ecc71,stroke:#27ae60,color:white
    classDef storage fill:#f1c40f,stroke:#f39c12,color:black
    classDef ml fill:#9b59b6,stroke:#8e44ad,color:white
    classDef tech fill:#95a5a6,stroke:#7f8c8d,color:white

    %% Legend
    subgraph Legend
        L_UI["UI Component"]:::ui
        L_PROC["Processing Component"]:::processing
        L_STORE["Storage Component"]:::storage
        L_ML["ML Component"]:::ml
        L_TECH["Technical Component"]:::tech
    end
```
