# bytedance/pasa architecture

> Architecture diagram of the GitHub repository bytedance/pasa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bytedance/pasa
- Repository: https://github.com/bytedance/pasa

## Overview

##### Project Type and Purpose
- The project is an academic search agent leveraging large language models.
- It employs two specialized agents: the Crawler (that generates search queries, crawls and aggregates academic papers via search tools like Google Search API and arXiv API) and the Selector (which evaluates the relevance of papers based on titles and abstracts).
- It is designed to autonomously handle scholarly queries in a high-quality academic context.

##### File Structure Analysis
- Top-Level Files:
- Files such as agent_prompt.json, models.py, paper_agent.py, and paper_node.py indicate the agent logic and node-based processing.
- scripts like run_paper_agent.py serve as the main entry point.
- The "src" directory includes resources (for example, the architecture image, cover image, and supporting utilities in utils.py).
- Other modules (metrics.py, requirements.txt) emphasize evaluation and dependency management.
- The file tree suggests a relatively lean codebase where the core system behavior is defined across several Python scripts with various responsibilities.

##### Insights from README
- The README provides a clear high-level explanation of the two LLM agents (Crawler and Selector) and their roles.
- It illustrates interaction with external APIs (Google Search API and arXiv/APIs), and has training instructions for SFT and PPO later in the document.

##### Minimum Components
a. Main Components:
- User Input / Frontend: Represents the interface where academic queries are input (e.g., through the website or command-line interface triggered via run_paper_agent.py).
- Crawler Agent:
▪ Responsible for processing the user query, generating search queries, and invoking external search APIs.
▪ Interacts with external search tools (Google Search API, arXiv/ar5iv API) and assembles a paper queue.
- Paper Queue:
▪ A central staging area where papers fetched by the Crawler are accumulated.
- Selector Agent:
▪ Consumes papers from the paper queue.
▪ Uses models to score relevance against the original query.
- Data/Model Repositories:
▪ Checkpoint folders that contain downloaded models (pasa-7b-crawler and pasa-7b-selector).
▪ Datasets (both synthetic AutoScholarQuery and real-world RealScholarQuery), paper databases, etc.
- Auxiliary Components:
▪ Training modules (SFT and PPO scripts available in the README).
▪ Utilities (as defined in utils.py) that help connect to APIs and preprocess input.

b. Relationships and Interactions:
▪ From User Input to Crawler Agent.
▪ From Crawler Agent to external services (displaying search APIs calls).
▪ External search results are then aggregated into the Paper Queue.
▪ The Selector Agent consumes papers from the Paper Queue.
▪ Finally, results are output back to the user or further processed.

##### Architectural Patterns and Design Principles
- The system uses a modular, agent-based design:
- Separate responsibilities (Crawler for search/extraction; Selector for evaluation) enforce the single responsibility principle.
- The separation of concerns is also visible in training routines (SFT and PPO training scripts for both agents, indicating a design that is extensible and separately tunable).

## Diagram (Mermaid)

```mermaid
graph TD
    %% User Interaction Layer
    subgraph "User Interaction Layer"
        UI["User Input / Frontend"]:::interface
    end

    %% Core Agents Group
    subgraph "Core Agents"
        CA["Crawler Agent"]:::agent
        AP["Agent Prompt"]:::agentConfig
        SA["Selector Agent"]:::agent
        CA -.->|"configured_by"| AP
    end

    %% Processing & Data Flow
    subgraph "Processing & Data Flow"
        PQ["Paper Queue"]:::queue
    end

    %% External Services
    subgraph "External Services"
        GAPI["Google Search API"]:::external
        ARAPI["arXiv API"]:::external
    end

    %% Auxiliary Components
    subgraph "Auxiliary Components"
        UTIL["Auxiliary Utilities"]:::utility
        METR["Evaluation Metrics"]:::metrics
    end

    %% Data Flow Connections
    UI -->|"submit_query"| CA
    CA -->|"calls"| GAPI
    CA -->|"calls"| ARAPI
    GAPI -->|"aggregates"| PQ
    ARAPI -->|"aggregates"| PQ
    PQ -->|"provides_papers"| SA
    SA -->|"returns_results"| UI

    %% Auxiliary Utilities used by Agents
    UTIL ---|"supports"| CA
    UTIL ---|"supports"| SA

    %% Evaluation Metrics linked with Selector Agent
    METR ---|"evaluates"| SA

    %% Click Events
    click UI "https://github.com/bytedance/pasa/blob/main/run_paper_agent.py"
    click CA "https://github.com/bytedance/pasa/blob/main/paper_agent.py"
    click AP "https://github.com/bytedance/pasa/blob/main/agent_prompt.json"
    click PQ "https://github.com/bytedance/pasa/blob/main/paper_node.py"
    click SA "https://github.com/bytedance/pasa/blob/main/models.py"
    click UTIL "https://github.com/bytedance/pasa/blob/main/utils.py"
    click METR "https://github.com/bytedance/pasa/blob/main/metrics.py"

    %% Styles
    classDef interface fill:#f9c,stroke:#333,stroke-width:2px;
    classDef agent fill:#bbf,stroke:#333,stroke-width:2px;
    classDef agentConfig fill:#aaf,stroke:#333,stroke-width:2px;
    classDef queue fill:#fcf,stroke:#333,stroke-width:2px;
    classDef external fill:#cff,stroke:#333,stroke-width:2px;
    classDef utility fill:#cfc,stroke:#333,stroke-width:2px;
    classDef metrics fill:#ff9,stroke:#333,stroke-width:2px;
```
