How adychechani/finrag works
Main Components
• User Interface: Represented by app.py along with the templates (index.html) and static assets (style.css). This is the entry point for users interacting with the system.
• Data Source and Storage: The data folder holds financial documents (PDFs) such as Apple’s financial statements and NVIDIA’s annual report. This component is the source of raw financial data.
• Indexing Module: The indexer.py script is responsible for parsing, chunking, creating embeddings, and performing an “upsert” operation. This module indexes the raw data to make it searchable.
• Query Processing Module: The query_processor.py handles rewriting queries and re-ranking relevant contexts. It acts as a bridge between user queries (possibly received via app.py) and the indexed data.
• RAG / Generation Process: The README outlines a generation step (which includes a factualness check) to produce analysis. Although this is not explicitly tied to a separate file, it should be represented in the architecture as a distinct component that follows retrieval. The RAG_script.ipynb indicates this process may be prototyped or experimented with in a notebook environment.
Relationships and Interactions
• Data Flow: Start with raw financial data from the data folder. This moves to the Indexing Module (indexer.py) where the data is processed into chunks with generated embeddings.
• Query Flow: When a user submits a query through the User Interface, it is handled by the web layer (app.py) and forwarded to the Query Processing Module (query_processor.py). This module might modify or optimize the query for better retrieval.
• External Interactions: If applicable, note any external services like APIs for live market data or news – even if these aren’t part of the current file tree, the README hints at integration with up-to-date information sources.
Architectural Patterns and Design Principles
• Modular Design: The separation of indexing, query processing, and presentation (UI layer) highlights a clear modular structure. Each module handles its own responsibility.
• Integration Points: Even if the generation step isn’t fully implemented as a separate service, denote it as a critical stage. Additionally, point out any configuration/deployment considerations (like .env and requirements.txt) that reflect the runtime and dependency management.
Relevant Technologies and Frameworks
• The project likely uses Python (given the .py files and Jupyter Notebook) and web frameworks (Flask is a strong possibility given the presence of app.py, templates, and static content).
• Libraries for NLP and embedding generation might be used within indexer.py and query_processor.py.
• If database or vector search systems (like Pinecone, FAISS, or Elasticsearch) are used for “upsert” operations, they should be noted even if not explicitly shown in the file tree.