# najnin26/research-paper-recommandation-system architecture

> Architecture diagram of the GitHub repository najnin26/research-paper-recommandation-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/najnin26/research-paper-recommandation-system
- Repository: https://github.com/najnin26/research-paper-recommandation-system

## Overview

##### Project Type and Purpose
- This is a machine learning research application that uses Sentence-BERT embeddings to both recommend relevant research papers and predict their subject areas.
- It involves data preprocessing, vectorization, model training (using an MLP classifier), saving/loading of model components, and eventual deployment for inference.
- The file structure indicates a separation between the application deployment code (in App/app.py) and the core machine learning code (development and experiments in Code, with persistent model files in Code/Model).

##### Analyzing the File Structure
- Top-Level Files:
- .gitattributes: Repository configuration
- README.md: Provides details on the project methodology, pipeline components, and dependencies.
- App Directory:
- Contains app.py, likely encapsulating the deployment/inference interface. This might be an API endpoint or a simple application script to serve predictions.
- Code Directory:
- Contains the experimental notebook (Research_Paper_recommendation_and_subject_area_prediction_using_sentence_transformer.ipynb) which details the research and training process.
- A Model subdirectory with several pickle files (sentences.pkl, text_vectorizer_config.pkl, text_vectorizer_weights.pkl, vocab.pkl) indicating saved model artifacts and vectorizer configurations which are central to the prediction and recommendation functions.

##### Main Components to Highlight
a. Data Preprocessing Module

b. Text Vectorization Module
- A component for “Text Vectorization using Sentence-BERT” that converts raw text into high-dimensional embeddings.
- Use of Sentence Transformer (SBERT library).

c. Model Training Module
- “MLP Classifier Training” component that takes vectorized data to train a classification model for subject area prediction.

d. Recommendation System Module
- A module where the semantic similarity between the research papers is computed using high-dimensional embeddings.
- This is the heart of the recommendation system that suggests related papers.

e. Model Persistence / Storage
- “Model and Vectorizer Storage” component that shows saving and loading of the trained model as well as pre-built configurations and vocabularies (pointing to files in Code/Model).

f. Deployment / Inference Module

g. External Dependencies
- Symbols or annotations for relevant libraries/frameworks (Python, Tensorflow, Scikit-learn, Transformers, Pandas, NumPy, and Matplotlib).

##### Relationships and Interactions
- From raw research paper data to preprocessing.
- From preprocessing to text vectorization.
- From vectorization to both the training of the MLP classifier and the recommendation system engine.
- From these components to the model storage unit where the artifacts (pickle files) are saved.
- Finally, from the model storage to the deployment layer (app.py), illustrating how the saved model and vectorizer are loaded to make predictions.

##### Architectural Patterns and Design Principles
- A pipeline design that cleanly separates concerns:
- Experimentation and training (in the Code directory) versus production/inference (in the App directory).
- Reuse of saved model components (configuration, vectorizer, vocab, etc.) to ensure consistency between training and prediction phases.

##### Final Elements
- Clear labels for each component, as described.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Dependencies
    EX["External Dependencies: Python,Tensorflow,Scikit-learn,Sentence-Transformers,Pandas,NumPy,Matplotlib"]:::external

    %% Training Pipeline Subgraph
    subgraph "Training Pipeline"
        A["Raw Research Paper Data"]:::raw
        B["Data Cleaning and Preprocessing"]:::preproc
        C["Text Vectorization using Sentence-BERT"]:::preproc
        D["MLP Classifier Training"]:::training
        E["Model Evaluation(Metrics)"]:::training
        F["Recommendation System"]:::training
    end

    %% Model Persistence / Storage
    G["Model and Vectorizer Storage"]:::storage

    %% Production / Inference Subgraph
    subgraph "Production"
        H["Inference Service"]:::deployment
    end

    %% Data Flow Connections
    A -->|"feeds"| B
    B -->|"outputs"| C
    C -->|"triggers"| D
    D -->|"evaluated_by"| E
    C -->|"supports"| F
    D -->|"saves to"| G
    F -->|"saves to"| G
    G -->|"loaded by"| H
    EX -->|"libraries_used" | B

    %% Click Events
    click B "https://github.com/najnin26/research-paper-recommandation-system/blob/main/Code/Research_Paper_recommendation_and_subject_area_prediction_using_sentence_transformer.ipynb"
    click C "https://github.com/najnin26/research-paper-recommandation-system/blob/main/Code/Research_Paper_recommendation_and_subject_area_prediction_using_sentence_transformer.ipynb"
    click D "https://github.com/najnin26/research-paper-recommandation-system/blob/main/Code/Research_Paper_recommendation_and_subject_area_prediction_using_sentence_transformer.ipynb"
    click F "https://github.com/najnin26/research-paper-recommandation-system/blob/main/Code/Research_Paper_recommendation_and_subject_area_prediction_using_sentence_transformer.ipynb"
    click G "https://github.com/najnin26/research-paper-recommandation-system/tree/main/Code/Model"
    click H "https://github.com/najnin26/research-paper-recommandation-system/blob/main/App/app.py"

    %% Class Definitions for styling
    classDef raw fill:#dddddd,stroke:#000000,stroke-width:2px;
    classDef preproc fill:#a3d5ff,stroke:#1f77b4,stroke-width:2px;
    classDef training fill:#c7f3b7,stroke:#2ca02c,stroke-width:2px;
    classDef storage fill:#ffc107,stroke:#ff9500,stroke-width:2px;
    classDef deployment fill:#ff9f9f,stroke:#d62728,stroke-width:2px;
    classDef external fill:#dabfff,stroke:#9467bd,stroke-width:2px;
```
