How comais-labs/busca_documentos works
Main Components
a. User Interface Layer:
• Streamlit Web Interface
• Upload Component
• Search Interface
• Document Download Component
b. Processing Layer:
• Document Processor
• PDF Processor (PyMuPDF)
• DOCX Processor (python-docx)
• Text Extraction Pipeline
c. Embedding Layer:
• Sentence Transformer Model
• (paraphrase-multilingual-mpnet-base-v2)
• CUDA Acceleration (optional component)
d. Search Layer:
• FAISS Vector Store
• LangChain Search Pipeline
e. Storage Layer:
• Document Storage
• Embedding Storage
Flow Representation
• User → Streamlit Interface
• Streamlit Interface → Document Processor (upload flow)
• Document Processor → Text Extraction
• Text Extraction → Embedding Generation
• Embedding Generation → FAISS Storage
• Search Query → Embedding Generation
• FAISS Storage → Search Results
• Search Results → User Interface
Specific Design Elements
• Rectangles: Main components
• Hexagons: Processing units
• Rounded rectangles: User interface elements
• Blue: User interface components
• Green: Processing components
• Yellow: Storage components
• Purple: ML/AI components (embedding model)
• Gray: External dependencies
Important Details
• Data transformation points
• Clear labeling of technologies used
Technical Details to Note
• Python 3.11 requirement
• LangChain's role in orchestrating operations
• FAISS's vector storage and retrieval mechanism
• Demonstrate the multilingual capability of the embedding model
Remember to include a version indicator and any relevant notes about scaling considerations or performance optimizations, particularly regarding the GPU acceleration option.