# yohanes213/academic-research-assistant architecture

> Architecture diagram of the GitHub repository yohanes213/academic-research-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yohanes213/academic-research-assistant
- Repository: https://github.com/yohanes213/academic-research-assistant

## Overview

##### Project type and purpose
- This is a containerized, full-stack microservices application called “Academic Research Assistant.”
- Its goal is to scrape and index academic papers, store them in a vector database, and provide an AI-powered Q&A interface via a Streamlit frontend.

##### File structure
- Top-level directories each map to a Docker-ized service:
- airflow/ – workflow scheduler (periodic ingestion)
- paper_ingestion/ – scraper and ingestion API
- vector_db/ – vector storage API backed by Pinecone
- research_assistant/ – AI query service using Gemini, Tavily, LangSmith
- frontend/ – Streamlit web UI
- docker-compose.yml orchestrates all five containers plus shared network.

##### README for additional insights
- Ports:
- Frontend: 8501
- Paper Ingestion: 8000
- Vector DB: 5000
- Research Assistant: 5001
- Airflow: 8080
- External dependencies: PubMed (scraper), Pinecone, Google Gemini, Tavily, LangSmith.

a. Frontend UI
b. Paper Ingestion Service
c. Vector Database Service
d. Research Assistant Service
e. Airflow Scheduler
f. External systems: PubMed, Pinecone, Gemini API, Tavily API, LangSmith

##### Relationships and interactions
- User → Frontend (HTTP/Streamlit UI)
- Frontend → Research Assistant (`POST /response`)
- Frontend → Vector DB (`GET /query`) and Paper Ingestion (`GET /scrape`) for manual triggers
- Research Assistant → Vector DB (`GET /query` to fetch embeddings)
- Research Assistant → Gemini API, Tavily API, LangSmith for model inference & monitoring
- Paper Ingestion → PubMed (scrape), then → Vector DB (`POST /upsert`)
- Airflow → Paper Ingestion (`GET /scrape`) on schedule

##### Architectural patterns and design principles
- Microservices: each service is isolated in its own container and communicates over HTTP
- Layered architecture: UI layer (frontend), API layer (ingestion, vector_db, research_assistant), orchestration layer (Airflow), data/external layer (PubMed, Pinecone, LLMs)
- Event-driven ingestion pipeline scheduled by Airflow
- Use of external managed services for scalability (Pinecone, Gemini)

##### Technologies and libraries to annotate
- Streamlit (frontend)
- FastAPI + Uvicorn (paper_ingestion, vector_db, research_assistant)
- Apache Airflow (scheduler)
- Pinecone SDK, PubMed scraping logic
- Gemini (Google AI Studio), Tavily, LangSmith

- Research Assistant to Gemini, Tavily, LangSmith.
- Vector DB to Pinecone.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External user
    User([User]):::external

    subgraph "Docker Compose"
        direction TB
        Frontend["Frontend (8501)"]:::ui
        PaperIngest["Paper Ingestion (8000)"]:::svc
        VectorDB["Vector DB (5000)"]:::svc
        Research["Research Assistant (5001)"]:::svc
        Airflow["Airflow Scheduler (8080)"]:::orchestration
    end

    %% External services
    PubMed([PubMed]):::external
    Pinecone([Pinecone]):::external
    Gemini([Gemini API]):::external
    Tavily([Tavily API]):::external
    LangSmith([LangSmith]):::external

    %% Interactions
    User -->|"uses UI"| Frontend
    Frontend -->|"POST /response"| Research
    Frontend -->|"GET /query"| VectorDB
    Frontend -->|"GET /scrape"| PaperIngest

    Research -->|"GET /query"| VectorDB
    Research -->|"model inference"| Gemini
    Research -->|"model inference"| Tavily
    Research -->|"monitoring"| LangSmith

    PaperIngest -->|"POST /upsert"| VectorDB
    PubMed -->|"scrape"| PaperIngest

    Airflow -->|"weekly DAG: GET /scrape"| PaperIngest

    VectorDB -->|"vector storage"| Pinecone

    %% Click events
    click Frontend "https://github.com/yohanes213/academic-research-assistant/tree/main/frontend/"
    click PaperIngest "https://github.com/yohanes213/academic-research-assistant/tree/main/paper_ingestion/"
    click VectorDB "https://github.com/yohanes213/academic-research-assistant/tree/main/vector_db/"
    click Research "https://github.com/yohanes213/academic-research-assistant/tree/main/research_assistant/"
    click Airflow "https://github.com/yohanes213/academic-research-assistant/tree/main/airflow/"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef svc fill:#90EE90,stroke:#000,stroke-width:1px
    classDef orchestration fill:#D3D3D3,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
```
