How noviscl/ai-researcher works
Project type and purpose
• This is not a web or mobile app but a research‐ideation pipeline built in Python, orchestrated by shell scripts.
• Purpose: Given a research topic, the system retrieves related papers, generates ideas grounded in those papers, deduplicates and refines ideas, expands them into detailed project proposals, ranks and filters proposals, and outputs ranked, high‐quality research proposals.
File structure
• Top level:
• ai_researcher/ – main Python package
• prompts/ – prompt templates (.txt, .json, .py utils)
• scripts/ – bash wrappers to run modules end to end or individually
• src/ – Python modules implementing each stage of the pipeline
• figures/ – illustrative assets (for docs)
• results/ – stored outputs and statistics scripts
• requirements.txt – Python dependencies
• Each src file corresponds to one pipeline step or utility: e.g., lit_review.py, grounded_idea_gen.py, dedup_ideas.py, project_proposal_gen.py, project_proposal_ranking.py, project_proposal_filter.py, plus scoring, analysis, and utility modules.
README for additional insights
• The pipeline has six sequential modules:
1. Related Paper Search
2. Grounded Idea Generation
3. Idea Deduplication
4. Project Proposal Generation
5. Project Proposal Ranking
6. Project Proposal Filtering
• Each can be run standalone or as part of an end‐to‐end flow via scripts/end_to_end.sh
• External services: OpenAI, Semantic Scholar API, Anthropic API, sentence‐transformers embedding models
Elements
• Clear labels on each component: “Related Paper Search,” “Grounded Idea Gen,” etc.
• Blue for core modules
• Green for data stores/cache
• Orange for external services
• Gray for orchestration scripts