# electify-eu/europarl-ai architecture

> Architecture diagram of the GitHub repository electify-eu/europarl-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/electify-eu/europarl-ai
- Repository: https://github.com/electify-eu/europarl-ai

## Overview

##### External Services (Purple hexagons)
- OpenAI API (for embeddings and LLM)
- Alternative LLMs (Mixtral, Claude)

##### Flow and Interactions

- User Query → Frontend → RAG System → Vector Database → LLM → Frontend → User

##### Detailed Component Breakdown

a. RAG Pipeline (Show as a subprocess):
- Document Retrieval → Context Selection → Query Processing → Response Generation

b. Evaluation System (Show as a parallel track):
- Experiments → Metrics Collection → Performance Analysis

##### Experiment Components

Subdivisions for:
- Party Anonymization
- LLM Comparison
- Retrieval Parameters
- Embedding Model Evaluation

##### The user interaction flow through the Streamlit interface

- External service indicators

Keep the abstraction level focused on the main architectural components while showing the experimental nature of the system through the evaluation framework.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles
    classDef frontend fill:#a8d1f0,stroke:#333
    classDef rag fill:#90EE90,stroke:#333
    classDef data fill:#FFE5B4,stroke:#333
    classDef external fill:#DDA0DD,stroke:#333,shape:hexagon
    classDef experiment fill:#F0E68C,stroke:#333

    %% Frontend Layer
    subgraph Frontend
        App["Streamlit App"]
        Pages["Pages"]
        Utils["Streamlit Utils"]
    end
    class App,Pages,Utils frontend
    click App "https://github.com/electify-eu/europarl-ai/blob/main/App.py"
    click Pages "https://github.com/electify-eu/europarl-ai/tree/main/pages/"
    click Utils "https://github.com/electify-eu/europarl-ai/tree/main/streamlit_app/utils/"

    %% RAG System Layer
    subgraph RAGSystem
        DB["Vector Database"]
        Models["RAG Models"]
        Eval["Evaluation System"]
    end
    class DB,Models,Eval rag
    click DB "https://github.com/electify-eu/europarl-ai/blob/main/RAG/database/vector_database.py"
    click Models "https://github.com/electify-eu/europarl-ai/blob/main/RAG/models/RAG.py"
    click Eval "https://github.com/electify-eu/europarl-ai/blob/main/RAG/evaluation/evaluation.py"

    %% Data Layer
    subgraph DataStorage
        Manifesto["Manifesto Storage"]
        Debates["Debates Storage"]
        Questions["Questions Storage"]
    end
    class Manifesto,Debates,Questions data
    click Manifesto "https://github.com/electify-eu/europarl-ai/tree/main/data/manifestos/"
    click Debates "https://github.com/electify-eu/europarl-ai/tree/main/data/debates/"
    click Questions "https://github.com/electify-eu/europarl-ai/tree/main/data/questions/"

    %% External Services
    OpenAI["OpenAI API"]
    Mixtral["Mixtral"]
    Claude["Claude"]
    class OpenAI,Mixtral,Claude external

    %% Experiment Components
    subgraph Experiments
        Anon["Party Anonymization"]
        LLMComp["LLM Comparison"]
        RetParam["Retrieval Parameters"]
        EmbedEval["Embedding Evaluation"]
        FinalEval["Final Evaluation"]
    end
    class Anon,LLMComp,RetParam,EmbedEval,FinalEval experiment
    click Anon "https://github.com/electify-eu/europarl-ai/tree/main/RAG/evaluation/Experiments/1_anonymize_parties/"
    click LLMComp "https://github.com/electify-eu/europarl-ai/tree/main/RAG/evaluation/Experiments/2_llm/"
    click RetParam "https://github.com/electify-eu/europarl-ai/tree/main/RAG/evaluation/Experiments/3_retrieval_fetch_k/"
    click EmbedEval "https://github.com/electify-eu/europarl-ai/tree/main/RAG/evaluation/Experiments/4_embedding_model/"
    click FinalEval "https://github.com/electify-eu/europarl-ai/tree/main/RAG/evaluation/Experiments/5_final_eval/"

    %% Relationships
    App --> DB
    DB --> Models
    Models --> OpenAI
    Models --> Mixtral
    Models --> Claude
    Models --> App
    
    DataStorage --> DB
    
    Eval --> Models
    Eval --> DB
    Experiments --> Eval

    %% Legend
    subgraph Legend
        F["Frontend Component"]:::frontend
        R["RAG Component"]:::rag
        D["Data Storage"]:::data
        E["External Service"]:::external
        X["Experiment"]:::experiment
    end
```
