# satish24011993/research_paper_classifier architecture

> Architecture diagram of the GitHub repository satish24011993/research_paper_classifier, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/satish24011993/research_paper_classifier
- Repository: https://github.com/satish24011993/research_paper_classifier

## Overview

##### Project type and purpose
- This is a full-stack machine-learning pipeline for research-paper abstract classification and disease–entity extraction.
- It includes data ingestion/preprocessing, model fine-tuning (classification), external LLM integration (disease extraction), evaluation, a REST API (FastAPI), and containerized/cloud deployment.

##### File structure
Top-level files:
- app.py – FastAPI service
- main_pipeline.py – data preprocessing + training + save model
- inference_and_evaluation.py – load model + evaluate + local inference
- deploy.sh & Dockerfile – containerization and deployment scripts
- requirements.txt – dependencies
- results/ – model artifacts, checkpoints, performance reports
- Dataset_1_/ – expected input data structure (Cancer/Non-Cancer .txt files)

Patterns and components:
- “main_pipeline.py” defines ETL → model training → artifact saving.
- “inference_and_evaluation.py” reuses saved artifacts to measure performance or run ad-hoc inference.
- “app.py” wraps inference and disease-extraction logic behind REST endpoints.
- Dockerfile + deploy.sh orchestrate container builds and pushes to GCP Cloud Run or Hugging Face Spaces.

##### README for additional insights
- Tech stack: Python 3.9+, Transformers (DistilBERT + LoRA), LangChain–Google Gemini for NER, FastAPI, Uvicorn, Docker, GCP SDK, Hugging Face CLI.
- Workflow described end-to-end: from raw abstracts → classification → entity extraction → performance metrics → REST API → Docker → cloud deployment.
- Notes on scalability (batch endpoints, Kafka/Redis streams) and agentic orchestration (optional bonus).

##### Essential detail checklist
- End-to-end pipeline stages and artifacts
- Separation between training and serving
- Storage location of trained models and label mappings
- External API integration point (Gemini)
- Containerization and cloud deployment flows
- Optional scalability extensions

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "ML Pipeline"
        A["Dataset_1_/"]:::dataStore
        B["main_pipeline.py\n(Preprocessing & Training)"]:::codeModule
        A -->|"local file I/O"| B
        B -->|"saves artifacts"| C
    end

    subgraph "Results Directory"
        C[(results/)]:::dataStore
        C1[/fine_tuned_model/]:::dataStore
        C2[/model_checkpoints/]:::dataStore
        C3["performance_report.txt"]:::dataStore
        C4["traditional_models_performance_report.json"]:::dataStore
    end
    B -->|"outputs to"| C
    D["inference_and_evaluation.py"]:::codeModule
    C -->|"loads artifacts"| D
    D -->|"writes report"| C3
    D -->|"writes report"| C4

    subgraph "Serving"
        E["app.py\n(FastAPI Service)"]:::codeModule
        F[/Client/]:::external
        G["Google Gemini API"]:::external
        F -->|"/analyze_abstract (HTTP REST)"| E
        E -->|"loads model"| C1
        E -->|"calls NER"| G
        G -->|"disease entities"| E
        E -->|"returns JSON"| F
    end

    subgraph "Deployment"
        J["requirements.txt"]:::codeModule
        H["Dockerfile"]:::deploy
        I["deploy.sh"]:::deploy
        J --> H
        H -->|"build container"| I
        I -->|"push to Artifact Registry"| K["GCP Cloud Run"]:::external
        I -->|"push via HF CLI"| L["Hugging Face Spaces"]:::external
    end

    subgraph "Optional Scalability"
        M["Kafka/Redis Streams"]:::external
        M -.-> B
        M -.-> D
    end

    classDef codeModule fill:#AED6F1,stroke:#1F618D,color:#0B5345;
    classDef dataStore fill:#F5B041,stroke:#B9770E,color:#7D6608;
    classDef external fill:#ABEBC6,stroke:#239B56,color:#145A32;
    classDef deploy fill:#D7BDE2,stroke:#8E44AD,color:#4A235A;

    click B "https://github.com/satish24011993/research_paper_classifier/blob/main/main_pipeline.py"
    click C "https://github.com/satish24011993/research_paper_classifier/tree/main/results/"
    click C1 "https://github.com/satish24011993/research_paper_classifier/tree/main/results/fine_tuned_model/"
    click C2 "https://github.com/satish24011993/research_paper_classifier/tree/main/results/model_checkpoints/"
    click C3 "https://github.com/satish24011993/research_paper_classifier/blob/main/results/performance_report.txt"
    click C4 "https://github.com/satish24011993/research_paper_classifier/blob/main/results/traditional_models_performance_report.json"
    click D "https://github.com/satish24011993/research_paper_classifier/blob/main/inference_and_evaluation.py"
    click E "https://github.com/satish24011993/research_paper_classifier/blob/main/app.py"
    click H "https://github.com/satish24011993/research_paper_classifier/tree/main/Dockerfile"
    click I "https://github.com/satish24011993/research_paper_classifier/blob/main/deploy.sh"
    click J "https://github.com/satish24011993/research_paper_classifier/blob/main/requirements.txt"
```
