How andyly213/ai-peer-review-platform works
Project type and purpose
• This is an open-source multi-agent simulation tool written in Python.
• It models the academic peer-review process as a token-based economy, using AutoGen to orchestrate AI agents and Ollama as the LLM backend.
• It integrates with the PeerRead dataset for realistic paper and review data.
Main architectural components
a. Entry Point & Configuration
• main.py / src/simulation/peer_review_simulation.py
• config/ (environment variables for Ollama model and API endpoint)
b. Data Layer
• src/data/paper_database.py: loads PeerRead or test dataset
• src/data/create_test_dataset.py + test_dataset/: generate and store a small subset
• src/data/token_system.py: manages the token economy logic
c. Core Multi-Agent System
• src/core/multi_agent_system.py: base class
• src/core/enhanced_multi_agent_system.py / customizable_agent_system.py: specialized system behaviors
• src/core/agent_factory.py & agent_templates.py: factory pattern for creating agents
d. Agents
• src/agents/editor_agent.py: simulates editors who request reviews
• src/agents/researcher_agent.py: simulates researchers who write and review
• src/agents/researcher_templates.py: prompt templates defining agent behavior
e. Simulation Orchestrator
• peer_review_simulation.py: ties data loading, agent instantiation, and the simulation loop
f. External Services
• Ollama LLM API (configured via OLLAMA_API_BASE)
• AutoGen library for multi-agent interaction
Relationships and data flows
• Start at main.py → load environment config → initialize PaperDatabase
• Database loads papers from test_dataset/ or full PeerRead
• TokenSystem instantiates initial token counts
• AgentFactory creates EditorAgent and multiple ResearcherAgents (using agent_templates)
• EnhancedMultiAgentSystem registers agents and orchestrates turn-based interactions
• Editors submit papers → Researchers pull review tasks → Researchers earn/spend tokens
• All agent-to-agent calls go through AutoGen which dispatches to Ollama API
• Simulation orchestrator collects results, metrics, and outputs statistics
Architectural patterns and design principles
• Factory pattern: agent_factory.py
• Template method pattern: researcher_templates.py defines prompt structures
• Layered architecture: clear separation between Data, Core, Agents, Simulation
• Plugin-style extensibility: add new agent types or dataset handlers