# digvijay123321/cosinesim_checker architecture

> Architecture diagram of the GitHub repository digvijay123321/cosinesim_checker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/digvijay123321/cosinesim_checker
- Repository: https://github.com/digvijay123321/cosinesim_checker

## Overview

##### Main Components

- Web Interface & User Interaction:
- Represented by app.py and the templates folder (index.html and result.html).
- This layer acts as the entry point of the application where users submit documents or trigger evaluations.

- Document Management:
- The project houses two document directories: Doc1 and Doc2.
- These directories contain different versions of documents (text files, PDFs, and JSON files for metadata) that will be processed and compared.

- Core Processing & Evaluation Engine:
- Multiple Python scripts (eval.py, evaluate_documents.py, evalution.py, and old_eval.py) form the heart of the evaluation process.
- They likely coordinate the comparison tasks, orchestrating how document content is processed and similarity calculated.

- Embedding Generation:
- embedding_generator.py is a dedicated module to generate embeddings.
- It uses five different models (OpenAI, T5-small, BERT, RoBERTa, and Modern BERT) to create vector representations of documents for cosine similarity analysis.

- Data Storage & Metrics:
- The Database directory holds JSON files (company_data.json and comparisons.json) that store company data and evaluation metrics.
- This component is critical for persisting input data, computed embeddings, and similarity scores.

- External and Utility Components:
- Dockerfile indicates containerization for deployment, ensuring consistency across environments.
- saving_to_gcp.py suggests integration with Google Cloud Platform for storing or processing data in the cloud.
- clear_temp.py and test.py, along with additional scripts (t1.py, t2.py, t3.py), likely provide auxiliary functions such as cleanup, testing, or experimental runs.

##### Relationships & Interactions

- User requests come into the Web Interface (app.py), which routes them to the core processing layers.
- The processing engine retrieves the necessary documents from the Doc1 and Doc2 directories.
- Embedding Generation is triggered to compute document embeddings via embedding_generator.py.
- The evaluation modules then compare the embeddings (using cosine similarity) and produce related metrics.
- Results and comparisons are stored in the Database JSON files.
- Additionally, utility scripts may handle tasks such as clearing temporary files or pushing results to external cloud storage (GCP).
- Deployment concerns (like the Docker container) should be represented as an encapsulating layer to show how the entire system is packaged and run.

##### Architectural Patterns & Technologies

- Separation of Concerns: Clear demarcation between UI (web templates, app.py), business logic (evaluation and embedding generation), and storage (JSON in Database) reflects a modular architecture.
- Service Integration: Integration with external models, cloud services (GCP), and containerization (Docker) are important details.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Docker Container"
        UI["Web Interface (Flask)"]:::ui
        Templates["Templates (index.html, result.html)"]:::ui
        Doc1["Doc1 Directory"]:::doc
        Doc2["Doc2 Directory"]:::doc
        EG["embedding_generator.py"]:::embedding
        subgraph "Evaluation Engine"
            eval1["eval.py"]:::processing
            eval2["evaluate_documents.py"]:::processing
            eval3["evalution.py"]:::processing
            eval4["old_eval.py"]:::processing
        end
        subgraph "Database Storage"
            compData["company_data.json"]:::db
            compMetric["comparisons.json"]:::db
        end
    end

    subgraph "External & Utilities"
        Dockerfile["Dockerfile"]:::external
        GCP["saving_to_gcp.py"]:::external
        CT["clear_temp.py"]:::external
        Test["test.py"]:::external
        T1["t1.py"]:::external
        T2["t2.py"]:::external
        T3["t3.py"]:::external
    end

    %% Data Flow Connections inside Docker Container
    UI -->|"submit"| Doc1
    UI -->|"submit"| Doc2
    Doc1 -->|"loadDoc"| EG
    Doc2 -->|"loadDoc"| EG
    EG -->|"computeEmbeddings"| eval1
    eval1 -->|"orchestrate"| eval2
    eval2 -->|"orchestrate"| eval3
    eval3 -->|"orchestrate"| eval4
    eval4 -->|"storeResult"| compMetric
    compData --- compMetric
    compMetric -->|"displayResults"| UI

    %% External and Utility interactions
    Dockerfile -->|"builds"| UI
    eval3 -->|"pushToGCP"| GCP
    eval4 -->|"cleanup"| CT
    eval4 -->|"testRun"| Test
    eval4 -->|"experiment"| T1
    eval4 -->|"experiment"| T2
    eval4 -->|"experiment"| T3

    %% Click Events
    click UI "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/app.py"
    click Templates "https://github.com/digvijay123321/cosinesim_checker/tree/main/Evalution/templates"
    click Doc1 "https://github.com/digvijay123321/cosinesim_checker/tree/main/Evalution/Doc1"
    click Doc2 "https://github.com/digvijay123321/cosinesim_checker/tree/main/Evalution/Doc2"
    click EG "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/embedding_generator.py"
    click eval1 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/eval.py"
    click eval2 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/evaluate_documents.py"
    click eval3 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/evalution.py"
    click eval4 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/old_eval.py"
    click compData "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/Database/company/company_data.json"
    click compMetric "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/Database/metrics/comparisons.json"
    click Dockerfile "https://github.com/digvijay123321/cosinesim_checker/tree/main/Evalution/Dockerfile"
    click GCP "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/saving_to_gcp.py"
    click CT "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/clear_temp.py"
    click Test "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/test.py"
    click T1 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/t1.py"
    click T2 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/t2.py"
    click T3 "https://github.com/digvijay123321/cosinesim_checker/blob/main/Evalution/t3.py"

    %% Styles
    classDef ui fill:#add8e6,stroke:#000,stroke-width:1px;
    classDef doc fill:#90ee90,stroke:#000,stroke-width:1px;
    classDef processing fill:#f9d79f,stroke:#000,stroke-width:1px;
    classDef embedding fill:#f5b7b1,stroke:#000,stroke-width:1px;
    classDef db fill:#aed6f1,stroke:#000,stroke-width:1px;
    classDef external fill:#d7bde2,stroke:#000,stroke-width:1px;
```
