# mvpalheta/ragfolio architecture

> Architecture diagram of the GitHub repository mvpalheta/ragfolio, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mvpalheta/ragfolio
- Repository: https://github.com/mvpalheta/ragfolio

## Overview

##### Project type and purpose
- RAGfolio is a full-stack, cloud-native investment analysis application.
- Purpose: Provide a three-stream (fundamental, momentum, sentiment) RAG-powered analysis on stocks, aggregate results, and present recommendations via API and web UI.

##### File structure
- Top-level directories:
- app/ (FastAPI backend)
- front/ (Next.js + TailwindCSS frontend)
- ingestion/ (data ingestion scripts for SEC filings & news)
- utils/ (helper scripts)
- app/ substructure:
- analyzers/ implement three parallel RAG streams (inherit from base_analyzer)
- config/ holds queries, settings, ticker mappings
- models/ Pydantic schemas for API & analysis results
- prompts/ LLM prompt templates for each analysis task
- routers/ FastAPI endpoints (agent, llm, search)
- services/ business logic: orchestration (agent_service), vector DB client (retriever), embedder, LLM interface, prompt manager, ticker extractor
- utils/ decorators for logging/error handling
- main.py: FastAPI entry point
- Docker-related: Dockerfile under app/, Dockerfile under front/, docker-compose.yml
- ingestion/: Python scripts to populate Qdrant from SEC API and yfinance news

##### README for insights
- Three-stream hybrid search architecture: dense (Sentence-Transformers), sparse (BM25), late interaction (ColBERTv2) via Qdrant
- Parallel workflow orchestration inspired by Anthropic Agents
- External dependencies: Qdrant vector DB, Groq API (LLM), Docker Compose

a. Main components
- Client (Browser / CURL / Postman)
- Frontend (Next.js app, port 3000)
- Backend API (FastAPI, port 8000)
- Orchestrator (agent_service)
- Analyzer streams (fundamental_analyzer, momentum_analyzer, sentiment_analyzer)
- Document Retriever (Qdrant client)
- Vector Store (Qdrant)
- Embedding service (embedder)
- LLM service (Groq / OpenAI / LLM API)
- Ingestion pipeline (SEC & news ingestion scripts)
- Configuration & Prompts (queries.yaml, .md prompt templates)
- Data stores (vector DB, optional file storage)

b. Relationships & interactions
1. User → Frontend → Backend API (POST /agent)
2. API routers delegate to Agent Service
3. Agent Service in parallel:
- For each stream, build query via Prompt Manager
- Embed query via Embedder → send to Document Retriever → Qdrant → raw docs
- Pass docs + prompt to LLM Service → analysis result
4. Collect three stream results → Final Aggregation prompt → LLM Service → Final recommendation
5. Return JSON to client
6. Ingestion scripts run offline or on schedule: fetch documents → preprocess → embed via embedder → upsert to Qdrant

c. Architectural patterns & design principles
- Workflow parallelization: streams run concurrently
- Layered separation: routers → services → analyzers/model → data store
- Single responsibility: each service class handles one integration
- Config-driven: queries and prompts externalized
- Containerized microservices: separate Docker contexts for front/back

d. Key technologies
- Frontend: Next.js, TailwindCSS, React
- Backend: FastAPI, Pydantic
- Vector DB: Qdrant
- Embeddings: sentence-transformers
- Retrieval: BM25, ColBERTv2
- LLM API: Groq or similar
- Orchestration: Python asyncio parallel tasks
- Ingestion: custom Python scripts

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User and UI
    User["User\n(Browser / CLI)"]:::external
    Frontend["Frontend\n(Next.js + Tailwind)"]:::ui
    User -->|"HTTP Request"| Frontend
    click Frontend "https://github.com/mvpalheta/ragfolio/tree/main/front/"

    %% Frontend Dockerfile
    FrontendDocker["Frontend Dockerfile"]:::infra
    Frontend --> FrontendDocker
    click FrontendDocker "https://github.com/mvpalheta/ragfolio/tree/main/front/Dockerfile"

    %% Backend entry
    User -->|"POST /agent"| BackendEntry
    Frontend -->|"POST /agent"| BackendEntry
    subgraph "Backend (FastAPI)" 
        direction TB
        BackendEntry["app/main.py\n(FastAPI Entry)"]:::backend
        click BackendEntry "https://github.com/mvpalheta/ragfolio/blob/main/app/main.py"

        %% Routers
        subgraph "Routers" 
            direction TB
            AgentRouter["Agent Router\n(POST /agent)"]:::backend
            click AgentRouter "https://github.com/mvpalheta/ragfolio/blob/main/app/routers/agent.py"
            LLMPRouter["LLM Router"]:::backend
            click LLMPRouter "https://github.com/mvpalheta/ragfolio/blob/main/app/routers/llm.py"
            SearchRouter["Search Router"]:::backend
            click SearchRouter "https://github.com/mvpalheta/ragfolio/blob/main/app/routers/search.py"
        end
        BackendEntry --> AgentRouter
        BackendEntry --> LLMPRouter
        BackendEntry --> SearchRouter

        %% Services
        subgraph "Services" 
            direction TB
            AgentService["Agent Service\n(workflow orchestrator)"]:::service
            click AgentService "https://github.com/mvpalheta/ragfolio/blob/main/app/services/agent_service.py"
            DocumentRetriever["Document Retriever"]:::service
            click DocumentRetriever "https://github.com/mvpalheta/ragfolio/blob/main/app/services/document_retriever.py"
            RetrieverWrapper["Sparse/Dense Retriever"]:::service
            click RetrieverWrapper "https://github.com/mvpalheta/ragfolio/blob/main/app/services/retriever.py"
            Embedder["Embedder Service"]:::service
            click Embedder "https://github.com/mvpalheta/ragfolio/blob/main/app/services/embedder.py"
            LLMService["LLM Service"]:::service
            click LLMService "https://github.com/mvpalheta/ragfolio/blob/main/app/services/llm_service.py"
            PromptManager["Prompt Manager"]:::service
            click PromptManager "https://github.com/mvpalheta/ragfolio/blob/main/app/services/prompt_manager.py"
            TickerExtractor["Ticker Extractor"]:::service
            click TickerExtractor "https://github.com/mvpalheta/ragfolio/blob/main/app/services/ticker_extractor.py"
            ConfigLoader["Config Loader"]:::service
            click ConfigLoader "https://github.com/mvpalheta/ragfolio/blob/main/app/services/config_loader.py"
        end
        AgentRouter --> AgentService
        AgentService --> PromptManager
        PromptManager --> ConfigLoader
        AgentService --> Embedder
        AgentService --> DocumentRetriever
        DocumentRetriever --> RetrieverWrapper
        AgentService --> LLMService
        AgentService --> TickerExtractor

        %% Analyzers
        subgraph "Analyzer Streams (Parallel)" 
            direction TB
            BaseAnalyzer["Base Analyzer"]:::service
            click BaseAnalyzer "https://github.com/mvpalheta/ragfolio/blob/main/app/analyzers/base_analyzer.py"
            Fundamental["Fundamental Analyzer"]:::service
            click Fundamental "https://github.com/mvpalheta/ragfolio/blob/main/app/analyzers/fundamental_analyzer.py"
            Momentum["Momentum Analyzer"]:::service
            click Momentum "https://github.com/mvpalheta/ragfolio/blob/main/app/analyzers/momentum_analyzer.py"
            Sentiment["Sentiment Analyzer"]:::service
            click Sentiment "https://github.com/mvpalheta/ragfolio/blob/main/app/analyzers/sentiment_analyzer.py"
        end
        AgentService -->|forks| Fundamental
        AgentService -->|forks| Momentum
        AgentService -->|forks| Sentiment
        Fundamental --> BaseAnalyzer
        Momentum --> BaseAnalyzer
        Sentiment --> BaseAnalyzer
    end

    %% Backend Dockerfile
    BackendDocker["Backend Dockerfile"]:::infra
    BackendEntry --> BackendDocker
    click BackendDocker "https://github.com/mvpalheta/ragfolio/tree/main/app/Dockerfile"

    %% Configuration and Prompts
    subgraph "Configuration & Prompts"
        direction TB
        Settings["settings.py"]:::infra
        click Settings "https://github.com/mvpalheta/ragfolio/blob/main/app/config/settings.py"
        Queries["queries.yaml"]:::infra
        click Queries "https://github.com/mvpalheta/ragfolio/blob/main/app/config/queries.yaml"
        TickerMap["ticker_mappings.yaml"]:::infra
        click TickerMap "https://github.com/mvpalheta/ragfolio/blob/main/app/config/ticker_mappings.yaml"
        subgraph "Prompt Templates"
            direction TB
            FA["fundamental_analysis.md"]:::infra
            click FA "https://github.com/mvpalheta/ragfolio/blob/main/app/prompts/fundamental_analysis.md"
            MA["momentum_analysis.md"]:::infra
            click MA "https://github.com/mvpalheta/ragfolio/blob/main/app/prompts/momentum_analysis.md"
            SA["sentiment_analysis.md"]:::infra
            click SA "https://github.com/mvpalheta/ragfolio/blob/main/app/prompts/sentiment_analysis.md"
            RAG["rag_response.md"]:::infra
            click RAG "https://github.com/mvpalheta/ragfolio/blob/main/app/prompts/rag_response.md"
            Final["final_recommendation.md"]:::infra
            click Final "https://github.com/mvpalheta/ragfolio/blob/main/app/prompts/final_recommendation.md"
            TE["ticker_extraction.md"]:::infra
            click TE "https://github.com/mvpalheta/ragfolio/blob/main/app/prompts/ticker_extraction.md"
        end
    end
    PromptManager --> Settings
    PromptManager --> Queries
    PromptManager --> TickerMap
    PromptManager --> FA
    PromptManager --> MA
    PromptManager --> SA
    PromptManager --> RAG
    PromptManager --> Final
    PromptManager --> TE

    %% Data Models
    subgraph "Data Models"
        direction TB
        AgentModels["agent.py"]:::backend
        click AgentModels "https://github.com/mvpalheta/ragfolio/blob/main/app/models/agent.py"
        APIModels["api.py"]:::backend
        click APIModels "https://github.com/mvpalheta/ragfolio/blob/main/app/models/api.py"
        EmbedModels["embeddings.py"]:::backend
        click EmbedModels "https://github.com/mvpalheta/ragfolio/blob/main/app/models/embeddings.py"
    end
    AgentRouter --> AgentModels
    SearchRouter --> EmbedModels
    LLMPRouter --> APIModels

    %% Utils
    Utils["Logging & Error Decorators"]:::service
    click Utils "https://github.com/mvpalheta/ragfolio/blob/main/app/utils/decorators.py"
    AgentService --> Utils

    %% Vector Store and LLM
    Qdrant["Qdrant Vector DB"]:::db
    LLMAPI["External LLM API\n(Groq/OpenAI)"]:::external
    DocumentRetriever --> Qdrant
    LLMService --> LLMAPI

    %% Ingestion Pipeline
    subgraph "Ingestion Pipeline (Offline)"
        direction TB
        CreateColl["create-collection.py"]:::infra
        click CreateColl "https://github.com/mvpalheta/ragfolio/blob/main/ingestion/create-collection.py"
        SECIngest["ingestion-sec-api.py"]:::infra
        click SECIngest "https://github.com/mvpalheta/ragfolio/blob/main/ingestion/ingestion-sec-api.py"
        NewsIngest["ingestion-yfinance-news.py"]:::infra
        click NewsIngest "https://github.com/mvpalheta/ragfolio/blob/main/ingestion/ingestion-yfinance-news.py"
    end
    CreateColl --> Qdrant
    SECIngest --> Qdrant
    NewsIngest --> Qdrant

    %% Orchestration & Deployment
    DockerCompose["docker-compose.yml"]:::infra
    click DockerCompose "https://github.com/mvpalheta/ragfolio/blob/main/docker-compose.yml"
    DockerCompose --> FrontendDocker
    DockerCompose --> BackendDocker

    %% Misc Utilities
    DownloadModels["Download Models Script"]:::infra
    click DownloadModels "https://github.com/mvpalheta/ragfolio/blob/main/utils/download_models.py"

    %% Styles
    classDef ui fill:#E0F7FA,stroke:#006064,color:#000
    classDef backend fill:#E8F5E9,stroke:#1B5E20,color:#000
    classDef service fill:#FFF3E0,stroke:#E65100,color:#000
    classDef db fill:#E1F5FE,stroke:#0277BD,color:#000
    classDef external fill:#F3E5F5,stroke:#6A1B9A,color:#000
    classDef infra fill:#ECEFF1,stroke:#37474F,color:#000
```
