How hananedupouy/llms-in-finance works
Project type and purpose
• This is an educational monorepo of Jupyter‐notebook demos applying Generative AI (LLM Agents, RAG, Multimodal models) to financial workflows.
• Rather than a single deployed product, it’s a collection of modules showcasing patterns: multi‐agent orchestration, retrieval pipelines, chart interpretation, synthetic data generation, and evaluation.
File structure
• Top‐level modules:
· Agents/… (several subfolders: AutoGen, OpenAI Agents SDK, Anthropic, LlamaIndex, CrewAI, etc.)
· RAG/… (workflows using different LLM providers + evaluation submodule)
· Multimodal_llms/financial_analysis (chart/PDF loaders + multimodal LLM experiments)
· Datasets/Synthetic_data (data generation notebooks + images)
· Papers/ (reference summaries, images)
• No monolithic backend/frontend—it’s a set of pipelines orchestrated in notebooks.
• Key “infrastructure” files: pyproject.toml under CRAG (packaging), data folders under AutoGen, evaluation tool configs.
README for additional insights
• Describes the four pillars: AI Agents, RAG, Multimodals, Papers.
• Mentions frameworks (OpenAI Agents SDK, Anthropic, AutoGen, LlamaIndex, CrewAI, LangChain).
• Emphasizes retrieval, parsing, chart interpretation, evaluation with DeepEval/GiskardAI.
Components
a. Main components (group into layers):
1. User interface layer: Jupyter Notebooks (entry points for each demo)
2. Orchestration layer: Agent frameworks (AutoGen, OpenAI Agents SDK, Anthropic agents, LlamaIndex agents, CrewAI, LangChain)
3. Retrieval/Index layer: LlamaIndex packs (Corrective_RAG), prompt caching, PDF/data loaders
4. Multimodal Analysis layer: chart/PDF loaders + multimodal LLMs (GPT-4o, Claude Sonnet, o1 Reasoning)
5. Data layer: financial data (CSV, PDFs), synthetic data generator (LLamaIndex notebooks)
6. Evaluation layer: DeepEval, GiskardAI modules under RAG/evaluation
7. External services layer: LLM APIs (OpenAI, Anthropic, Sonnet), storage (S3, local file system)
b. Relationships / interactions:
• Notebooks invoke Orchestration layer → orchestrator spins up multiple agents → agents call LLM APIs.
• Agents or RAG workflows consult the Retrieval/Index layer (LlamaIndex packs, cached prompts) → fetch documents or cached embeddings → send to LLM for response.
• Multimodal notebooks load charts/PDFs → pass to multimodal LLMs → receive analysis.
• Synthetic data notebooks generate datasets → feed into Agents or RAG demos for testing.
• Evaluation notebooks retrieve model outputs → use DeepEval/GiskardAI → produce evaluation reports.
c. Architectural patterns / design principles:
• Modular decomposition: each framework/demo lives in its own folder.
• Pipeline style: data ingestion → indexing → LLM call → post‐processing → evaluation.
• Multi‐agent choreography: agents collaborate via an orchestrator.
• Extensibility points: add new Agent framework simply by adding a folder with notebooks & config.
d. Key technologies & libraries:
• Jupyter Notebooks (UI)
• Agent frameworks: openai-agents-sdk, AutoGen, LlamaIndex, CrewAI, LangChain, Anthropic client
• Retrieval/Index: LlamaIndex packs, prompt caching modules
• Multimodal: GPT-4o, Claude Sonnet, o1 Reasoning, LlamaParse
• Evaluation: DeepEval, GiskardAI
• Data storage: local CSV/PDF, possible cloud buckets (if extended)