How bytedance/pasa works
Project Type and Purpose
• The project is an academic search agent leveraging large language models.
• It employs two specialized agents: the Crawler (that generates search queries, crawls and aggregates academic papers via search tools like Google Search API and arXiv API) and the Selector (which evaluates the relevance of papers based on titles and abstracts).
• It is designed to autonomously handle scholarly queries in a high-quality academic context.
File Structure Analysis
• Top-Level Files:
• Files such as agent_prompt.json, models.py, paper_agent.py, and paper_node.py indicate the agent logic and node-based processing.
• scripts like run_paper_agent.py serve as the main entry point.
• The "src" directory includes resources (for example, the architecture image, cover image, and supporting utilities in utils.py).
• Other modules (metrics.py, requirements.txt) emphasize evaluation and dependency management.
• The file tree suggests a relatively lean codebase where the core system behavior is defined across several Python scripts with various responsibilities.
Insights from README
• The README provides a clear high-level explanation of the two LLM agents (Crawler and Selector) and their roles.
• It illustrates interaction with external APIs (Google Search API and arXiv/APIs), and has training instructions for SFT and PPO later in the document.
Minimum Components
a. Main Components:
• User Input / Frontend: Represents the interface where academic queries are input (e.g., through the website or command-line interface triggered via run_paper_agent.py).
• Crawler Agent:
▪ Responsible for processing the user query, generating search queries, and invoking external search APIs.
▪ Interacts with external search tools (Google Search API, arXiv/ar5iv API) and assembles a paper queue.
• Paper Queue:
▪ A central staging area where papers fetched by the Crawler are accumulated.
• Selector Agent:
▪ Consumes papers from the paper queue.
▪ Uses models to score relevance against the original query.
• Data/Model Repositories:
▪ Checkpoint folders that contain downloaded models (pasa-7b-crawler and pasa-7b-selector).
▪ Datasets (both synthetic AutoScholarQuery and real-world RealScholarQuery), paper databases, etc.
• Auxiliary Components:
▪ Training modules (SFT and PPO scripts available in the README).
▪ Utilities (as defined in utils.py) that help connect to APIs and preprocess input.
b. Relationships and Interactions:
▪ From User Input to Crawler Agent.
▪ From Crawler Agent to external services (displaying search APIs calls).
▪ External search results are then aggregated into the Paper Queue.
▪ The Selector Agent consumes papers from the Paper Queue.
▪ Finally, results are output back to the user or further processed.
Architectural Patterns and Design Principles
• The system uses a modular, agent-based design:
• Separate responsibilities (Crawler for search/extraction; Selector for evaluation) enforce the single responsibility principle.
• The separation of concerns is also visible in training routines (SFT and PPO training scripts for both agents, indicating a design that is extensible and separately tunable).