# hamedbabaei/llms4om architecture

> Architecture diagram of the GitHub repository hamedbabaei/llms4om, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hamedbabaei/llms4om
- Repository: https://github.com/hamedbabaei/llms4om

## Overview

##### Project type and purpose
- This is a Python-based machine-learning framework (no browser UI) for Ontology Matching (OM) that orchestrates data ingestion, encoding, retrieval/LLM-based matching, post-processing, and evaluation across multiple datasets.
- Key purpose: provide a modular, pipeline-driven toolkit (LLMs4OM) to compare source and target ontology concepts via retrieval and/or RAG approaches using lightweight encoders, vector retrievers, and LLMs.

##### High-level components
A. Data Layer
- Ontology files & datasets (datasets/*)
- Pre-computed embeddings & assets (assets/)
- Result & visualization storage (experiments/, notebooks/)
B. Configuration & Environment
- BaseConfig (.env, .env-example, requirements.txt)
- Dockerfile / docker_builder.sh
C. Core Library Modules (ontomap package)
1. Ontology parsers (ontomap/ontology)
2. Encoders (ontomap/encoder: lightweight, RAG, fewshot, etc.)
3. Matchers (ontomap/ontology_matchers)
– Retrieval models (TFIDF, BERT, Specter, Ada…)
– Hybrid RAG models (OpenAI, LLaMA, Mistral + retriever)
– Other algorithms (ICV, few-shot, lightweight)
4. Pipeline orchestration (ontomap/pipeline/OMPipeline)
5. Post-processing (ontomap/postprocess)
6. Evaluation & metrics (ontomap/evaluation)
7. Utilities & I/O (ontomap/utils, ontomap/tools)
D. Experimentation & Automation
- Entrypoint scripts (scripts/*.sh, .py)
- Jupyter notebooks for analysis (notebooks/)
- Experiment artifacts (experiments/outputs, results, tables, visualizations)

##### Core data flow and interactions
1. **Configuration load**: BaseConfig reads CLI args/.env → initializes hyper-parameters (approach, encoder, thresholds, device, paths).
2. **Data ingestion**: Ontology parser loads source/target ontologies + reference alignments.
3. **Encoding**: Chosen encoder transforms each source concept (and optionally context: parents/children) into query vectors or prompt-ready text.
4. **Matching** (branch by approach):
- Retrieval only: encoder → vector DB (in-memory FAISS or similar) → retriever model scores top-k target concepts.
- RAG/hybrid: same retrieval step + assemble LLM prompt pairs → call LLM API (OpenAI, LLaMA locally) → LLM returns match probability/classification.
5. **Post-processing**: Apply thresholds (ir_score, llm_confidence), filter/aggregate results.
6. **Evaluation**: Compare predictions vs. ground truth → compute precision/recall/F1 via evaluator.
7. **Results export**: Write CSV/LaTeX tables, pivot visualizations, log to experiments/.

##### Architectural patterns & design principles
- Pipeline pattern: each stage (ingest → encode → match → postprocess → eval) is a pluggable module.
- Strategy/Factory pattern: different encoder and matcher classes implement a common interface (`generate(input_data)`).
- Separation of concerns: parsing, encoding, matching, post-processing, evaluation live in separate subpackages.
- Config-driven: switching approaches, encoders, devices via BaseConfig.
- Extensibility: new matcher or encoder simply registers under ontology_matchers or encoder.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration & Environment
    subgraph "Configuration & Environment"
        Env[".env / .env-example"]:::cfg
        Req["requirements.txt"]:::cfg
        Dockerfile["Dockerfile"]:::cfg
        DockerBuilder["docker_builder.sh"]:::cfg
        BaseConfig["BaseConfig"]:::cfg
    end

    %% Data Layer
    subgraph "Data Layer"
        Datasets["Ontology Datasets"]:::data
        Assets["Pre-computed Embeddings & Assets"]:::data
        VectorDB["Vector Index / DB"]:::data
        Experiments["Experiment Results & Visualizations"]:::data
        Notebooks["Notebooks (analysis)"]:::data
    end

    %% Core Library
    subgraph "Core Library"
        subgraph "Ontology Parsers"
            Parser["Parser Module"]:::core
        end
        subgraph "Encoders"
            EncodersMain["encoders.py"]:::core
            LightweightEnc["lightweight.py"]:::core
            RAGEnc["rag.py"]:::core
            FewshotEnc["fewshot.py"]:::core
            NaiveEnc["naivconvoaei.py"]:::core
        end
        subgraph "Matchers"
            Retrieval["Retrieval"]:::core
            RetrievalModels["Retrieval Models"]:::core
            RAGM["RAG"]:::core
            RAGModels["RAG Models"]:::core
            RAGData["RAG Dataset"]:::core
            FewshotM["Few-Shot"]:::core
            FewshotModels["Few-Shot Models"]:::core
            ICV["ICV"]:::core
            ICVModels["ICV Models"]:::core
            ICVTasks["ICV Tasks"]:::core
            ICVUtils["ICV Utils"]:::core
            LightweightM["Lightweight"]:::core
            LightweightModelsM["Lightweight Models"]:::core
            LLMM["LLM-only"]:::core
            LLMModelsM["LLM Models"]:::core
        end
        Pipeline["Pipeline Orchestrator"]:::core
        Postprocess["Post-processing"]:::core
        Evaluation["Evaluation & Metrics"]:::core
        EvalMetrics["Metrics"]:::core
        UtilsIO["IO Utilities"]:::core
        Workdir["Workdir Tool"]:::core
    end

    %% External Services
    subgraph "External Services"
        LLMExt["External LLM APIs"]:::ext
    end

    %% Experimentation & Automation
    subgraph "Experimentation & Automation"
        Scripts["Entrypoint Scripts"]:::cfg
        NotebooksExp["Jupyter Notebooks"]:::cfg
    end

    %% Data Flow
    Env -->|"env vars"| BaseConfig
    Req -->|"dependencies"| BaseConfig
    BaseConfig -->|"config"| Pipeline
    Scripts -->|"execute"| Pipeline

    Datasets -->|"ontologies"| Parser
    Parser -->|"parsed data"| EncodersMain
    EncodersMain -->|"vectors/text"| Retrieval
    EncodersMain -->|"vectors/text"| RAGM
    EncodersMain -->|"vectors/text"| FewshotM
    EncodersMain -->|"vectors/text"| ICV
    EncodersMain -->|"vectors/text"| LightweightM
    EncodersMain -->|"prompts"| LLMM

    Retrieval -->|"query vectors"| VectorDB
    VectorDB -->|"top-k"| Retrieval
    Retrieval -->|"scores"| Postprocess

    RAGM -->|"query vectors"| VectorDB
    VectorDB -->|"top-k"| RAGM
    RAGM -->|"prompts"| LLMExt
    LLMExt -->|"responses"| RAGM
    RAGM -->|"scores"| Postprocess

    FewshotM -->|"prompts"| LLMExt
    LLMExt -->|"responses"| FewshotM
    FewshotM -->|"scores"| Postprocess

    ICV -->|"icv data"| Postprocess
    LightweightM -->|"scores"| Postprocess
    LLMM -->|"responses"| Postprocess

    Postprocess -->|"filtered results"| Evaluation
    Evaluation -->|"metrics"| Experiments
    Evaluation -->|"metrics"| Notebooks

    NotebooksExp -->|"analysis"| Experiments

    Assets -->|"embeddings"| VectorDB
    Experiments -->|"storage"| Experiments

    %% Click Events
    click Env "https://github.com/hamedbabaei/llms4om/blob/main/.env-example"
    click Req "https://github.com/hamedbabaei/llms4om/blob/main/requirements.txt"
    click Dockerfile "https://github.com/hamedbabaei/llms4om/tree/main/Dockerfile"
    click DockerBuilder "https://github.com/hamedbabaei/llms4om/blob/main/docker_builder.sh"
    click Parser "https://github.com/hamedbabaei/llms4om/tree/main/ontomap/ontology/"
    click EncodersMain "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/encoder/encoders.py"
    click LightweightEnc "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/encoder/lightweight.py"
    click RAGEnc "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/encoder/rag.py"
    click FewshotEnc "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/encoder/fewshot.py"
    click NaiveEnc "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/encoder/naivconvoaei.py"
    click Retrieval "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/retrieval/retrieval.py"
    click RetrievalModels "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/retrieval/models.py"
    click RAGM "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/rag/rag.py"
    click RAGModels "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/rag/models.py"
    click RAGData "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/rag/dataset.py"
    click FewshotM "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/fewshot/fewshot.py"
    click FewshotModels "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/fewshot/models.py"
    click ICV "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/icv/icv.py"
    click ICVModels "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/icv/models.py"
    click ICVTasks "https://github.com/hamedbabaei/llms4om/tree/main/ontomap/ontology_matchers/icv/tasks/"
    click ICVUtils "https://github.com/hamedbabaei/llms4om/tree/main/ontomap/ontology_matchers/icv/utils/"
    click LightweightM "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/lightweight/lightweight.py"
    click LightweightModelsM "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/lightweight/models.py"
    click LLMM "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/llm/llm.py"
    click LLMModelsM "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/ontology_matchers/llm/models.py"
    click Pipeline "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/pipeline/om_pipeline.py"
    click Postprocess "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/postprocess/process.py"
    click Evaluation "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/evaluation/evaluator.py"
    click EvalMetrics "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/evaluation/metrics.py"
    click UtilsIO "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/utils/io.py"
    click Workdir "https://github.com/hamedbabaei/llms4om/blob/main/ontomap/tools/workdir.py"
    click Datasets "https://github.com/hamedbabaei/llms4om/tree/main/datasets/"
    click Assets "https://github.com/hamedbabaei/llms4om/tree/main/assets/"
    click Experiments "https://github.com/hamedbabaei/llms4om/tree/main/experiments/"
    click Notebooks "https://github.com/hamedbabaei/llms4om/tree/main/notebooks/"
    click Scripts "https://github.com/hamedbabaei/llms4om/tree/main/scripts/"
    click NotebooksExp "https://github.com/hamedbabaei/llms4om/blob/main/notebooks/plot-retriever-results.ipynb"

    %% Styles
    classDef core fill:#add8e6,stroke:#333,stroke-width:1px
    classDef data fill:#90ee90,stroke:#333,stroke-width:1px
    classDef ext fill:#ffa500,stroke:#333,stroke-width:1px
    classDef cfg fill:#d3d3d3,stroke:#333,stroke-width:1px
```
