How iiiiiiiinseong/finance-agent works
Main Components
• Data Layer
• Raw Documents (PDFs in data/raw_docs/woori_deposit_trust_docs)
• Structured FAQ JSONL (data/processed/faq_woori_structured.jsonl)
• ETL & Indexing Layer
• Product Loader (data/loader/product_loader.py)
• FAQ Structure Script (data/processed/faq_woori.py)
• Index Builders (scripts/build_faq_index.py, scripts/build_deposit_index.py)
• FAISS Indices (index/faq_faiss/, index/deposit_faiss/)
• Retrieval & Generation Layer (RAG Services)
• FAQ RAG (services/rag/faq_rag/retriever.py, faq_chain.py)
• Product RAG (services/rag/product_rag/retriever.py, chain.py)
• Orchestration & Fallback
• Router/Manager Agent (services/orchestrator/router_node.py)
• Fallback LLM Node (services/fallback/fallback_llm.py)
• Advisor Stub (future; services/advisor/advisor_stub.py)
• Configuration
• config.py (holds API keys, model names, chunk sizes, top-K parameters)
• UI Layer
• Streamlit App (apps/streamlit_app/app.py)
• External Services
• OpenAI Embeddings & LLM API
• (Optional) BM25 search component
Relationships & Data Flow
• Data Ingestion Flow (offline):
Raw Documents → ETL Loader → Chunked Documents → Embedding via OpenAI → FAISS Index
FAQ JSONL → Embedding → FAQ FAISS Index
• Query Flow (online):
User Input in Streamlit → Router Node
→ Parallel calls to FAQ RAG and Product RAG
• Each retrieves top-k contexts from FAISS (and optionally BM25)
• Each executes a LangGraph chain to generate answer with citations
→ Fallback LLM for out-of-scope queries
→ Merge outputs in Manager Agent → Return to UI
• Configuration Globals guide all modules via config.py
Architectural Patterns & Principles
• Modular RAG services for each domain (FAQ vs. Product)
• Hybrid retrieval pattern (FAISS + BM25)
• Pipelines defined by LangGraph (retrieve → generate)
• Orchestrator as coordinator (router + manager agent + fallback)
• Separation of concerns: ETL, indexing, retrieval, orchestration, presentation
Essential Elements to Capture
• Offline ETL & index-build process (as a separate flow)
• Dual retrieval pipelines (FAQ vs. Product)
• Orchestrator logic (parallel calls, fallback merge)
• Configuration centralization
• Streamlit UI invoking the router
• External dependencies (OpenAI API)