How jolovicdev/shandu works
Project type and purpose
• Shandu is a Python-based CLI library/tool, not a web application. It provides an AI-powered, state-based workflow (via LangGraph) for deep, multi-source research—driving search, scraping, content synthesis, and report generation.
File structure
• Top-level:
– setup.py, requirements.txt, MANIFEST.in, LICENSE → packaging & distribution
– cli.py → command-line entry point
– config.py & prompts.py → global configuration and prompt templates
– examples/ → sample usage and output
– tests/ → unit tests for core utilities (citation registry, report generator)
• shandu/agents/ → implements the LangGraph state graph
– graph/builder.py & wrapper.py → builds and executes the state graph
– nodes/ → each file is one node in the research pipeline (initialize, generate_queries, search, source_selection, citations, reflect, report_generation)
– processors/ → post-node processing: content_processor, report_generator
– utils/ → shared helpers (citation_manager, logger, agent_utils)
• shandu/research/ → high-level ResearchGraph API (researcher.py)
• shandu/search/ → AI-powered search adapters (ai_search.py) and generic search interface (search.py)
• shandu/scraper/ → web scraper module with JS rendering (scraper.py)
• shandu/utils/ → logging, caching, rate limiting
README for additional insights
Input (Q, Depth, Breadth) → Deep Research (DR) → SERP Queries (SQ) → Process Results (PR) → Learnings & Directions → decision node → either loop back or output Markdown report (MR).
• Architecture description highlights: state-based workflow, iterative exploration, multi-source synthesis, parallel processing, citation management.