How danny024/financellm works
Project Type and Purpose
• FinanceLLM is a retrieval-augmented generation (RAG) system for financial question-answering.
• It ingests structured financial data (data/train.json), builds a FAISS vector store, retrieves relevant document chunks, and uses a local LLM (via Ollama) to generate step-by-step answers.
• A Streamlit app (app.py) serves as the user interface; a Jupyter notebook (notebook.ipynb) supports development and analysis; evaluation modules produce retrieval and answer quality metrics.
File Structure Analysis
Top-level:
• app.py – Streamlit front-end
• Dockerfile, entry.sh – containerization
• data/ – raw JSON and generated Markdown
• vector_db/ – persistent FAISS indices and metadata
• modules/ – core logic:
– data_processor.py (JSON→Markdown, chunking)
– rag_pipeline.py (embedding, retrieval, LLM chain)
– evaluator.py (computes correctness, precision, recall, hit rate, NDCG)
• requirements.txt, env – dependencies and environment
• notebook.ipynb – interactive pipeline exploration
Key README Insights
• Technologies: Python 3.12, LangChain, FAISS, Ollama (deepseek-r1:1.5b, nomic-embed-text), Streamlit, Docker
• External services: Local Ollama server, optional S3/AWS for data storage, optional OpenAI API
Main Components and Interactions
a. Components
1. User Interface (Streamlit app)
2. Data Processor (modules/data_processor.py)
3. Vector Store (FAISS indices in vector_db/)
4. RAG Pipeline (modules/rag_pipeline.py + Ollama LLM)
5. Evaluation Engine (modules/evaluator.py)
6. Storage: JSON source, Markdown chunks, CSV metrics
7. External Services: Ollama server, (optional) AWS S3, (optional) OpenAI API
b. Data Flow and Relationships
1. User uploads train.json → Data Processor converts to Markdown → splits into chunks
2. Chunks → Embedding via nomic-embed-text → indexed into FAISS (vector_db/)
3. User query → RAG Pipeline:
• Retrieve top-k vectors from FAISS
• Pass retrieved context + prompt to Ollama LLM
• Receive generated answer → display in UI
4. Batch evaluation: Query set from JSON → RAG Pipeline → Evaluator computes metrics → writes CSV files
5. Notebook path: same modules used interactively
c. Architectural Patterns and Principles
• Modular separation: UI, processing, retrieval, generation, evaluation
• Pipeline design: Clear stages (ingest → index → retrieve → generate → evaluate)
• Persistence layer: FAISS indices serve as the document store
• Containerization: Dockerfile and entry script for reproducible environments
• Extensibility: modules/ namespace allows swapping embedding or LLM backends
d. Key Technologies and Frameworks
• Streamlit (UI)
• LangChain (chain orchestration)
• FAISS (vector store)
• Ollama (local LLM inference)
• Python modules for data processing and evaluation
Detailed Capture of Essential Elements
• Sequence:
1. Data ingestion & chunking
2. Embedding & indexing
3. Retrieval & generation
4. Evaluation & metrics persistence
5. UI interactions (upload, query, view metrics)
• Environment setup: Docker container boundary, .env config for credentials
• Metrics flows (evaluation_results.csv → aggregation → mean_retrieval_metrics.csv)