How ahmedgh970/finqa-engine works
FinQA Engine is a financial-document question-answering and benchmarking system. Its main workflow is offline filing ingestion and indexing, followed by online requests through the API into a configurable retrieval-and-answer graph; the response includes sources. The graph’s grading, neighboring-chunk expansion, and verified calculation support the documented advanced workflow, while a naive baseline and a separate tool-using agent provide alternatives. Evaluation covers retrieval metrics, LLM judging, and Ragas. Some connections are documented only at the product level because their implementation files were not excerpted.
Main entry points: nodes.py Workflow Nodes · runner.py Indexing Runner · main.py FastAPI · graph.py Workflow Graph
Components 30
Ingestion and Indexing
- Parse and Chunk
src/ingestion/parsers - Ingestion Runner
src/ingestion/runner.py - Chunk Contract
src/ingestion/schema.py - Chunk JSONL
src/ingestion/storage.py - Indexing Runner
src/indexing/runner.py - Embeddings
src/vectorstore/embeddings.py
Answer Workflow
- Workflow Graph
src/workflow/graph.py - Workflow Nodes
src/workflow/nodes.py - Workflow State
src/workflow/state.py - Chunk Expansion
src/workflow/expansion.py - Verified Calculator
src/workflow/calculator.py - Grounded Prompts
src/llm/prompts.py - Ollama Client
src/llm/client.py
Retrieval
- Qdrant
src/vectorstore/qdrant_store.py - Retriever Registry
src/retrieval/registry.py - Search Strategies
src/retrieval
Agents and Evaluation
- Agent Runner
src/agents/runner.py - Tool-Using RAG
src/agents/agent_rag.py - Agent Tools
src/agents/tools.py - Evaluation Runs
src/evaluation/run_retrieval.py - Retrieval Metrics
src/evaluation/retrieval - Ragas Evaluation
src/evaluation/run_ragas.py - Ragas Config
src/evaluation/ragas/config.py - LLM Judge
src/evaluation/run_judge.py - Evaluation Schema
src/evaluation/common/schema.py
Other components
- SEC Filings
- Web User
- Ollama
Connections 32
- Web User → Demo UI: uses
- Demo UI → FastAPI: calls demo API
- FastAPI → Workflow Graph: runs graph
src/api/main.py - FastAPI → Demo UI: streams results
src/api/main.py - SEC Filings → Parse and Chunk: provides documents
- Parse and Chunk → Chunk Contract: emits chunks
src/ingestion/schema.py - Ingestion Runner → Parse and Chunk: dispatches parsing
- Ingestion Runner → Chunk JSONL: persists chunks
- Chunk JSONL → Chunk Contract: serializes chunks
src/ingestion/storage.py - Indexing Runner → Chunk JSONL: reads chunks
src/indexing/runner.py - Indexing Runner → Embeddings: embeds text
src/indexing/runner.py - Indexing Runner → Qdrant: indexes vectors
src/indexing/runner.py - Retriever Registry → Search Strategies: builds strategy
src/retrieval/registry.py - Retriever Registry → Qdrant: builds dense client
src/retrieval/registry.py - Search Strategies → Qdrant: searches vectors
src/retrieval/registry.py - Workflow Graph → Workflow Nodes: routes nodes
src/workflow/graph.py - Workflow Graph → Workflow State: tracks state
src/workflow/graph.py - Workflow Nodes → Retriever Registry: retrieves passages
- Workflow Nodes → Chunk Expansion: expands passages
src/workflow/nodes.py - Workflow Nodes → Verified Calculator: verifies calculations
src/workflow/nodes.py - Workflow Nodes → Grounded Prompts: builds answer prompt
src/workflow/nodes.py - Workflow Nodes → Ollama Client: grades and generates
src/workflow/nodes.py - Grounded Prompts → Chunk Contract: formats chunk context
src/llm/prompts.py - Ollama Client → Ollama: requests completion
src/llm/client.py - Agent Runner → Tool-Using RAG: runs agent
src/agents/runner.py - Tool-Using RAG → Agent Tools: invokes tools
src/agents/agent_rag.py - Agent Tools → Search Strategies: retrieves chunks
src/agents/tools.py - Evaluation Runs → Retrieval Metrics: evaluates retrieval
src/evaluation/run_retrieval.py - Ragas Evaluation → Ragas Config: loads settings
src/evaluation/run_ragas.py - Ragas Evaluation → Evaluation Schema: uses evaluation data
- LLM Judge → Evaluation Schema: judges answers
- Evaluation Runs → Evaluation Schema: uses benchmark schema