How theblewish/automated-ai-web-researcher-ollama works
Project type and purpose
• This is a Python-based, command-line automated research assistant.
• It orchestrates structured web searches and scraping, drives a locally-hosted LLM (via Ollama), compiles findings into a text file, then produces a summary and Q&A session.
File structure
• Entry point:
– Web-LLM.py (main driver for research sessions)
– Self_Improving_Search.py (implements the iterative “self-improving” cycle)
• Orchestration & focus planning:
– research_manager.py (coordinates research loop, user commands)
– strategic_analysis_parser.py (breaks queries into prioritized focus areas)
• LLM integration:
– llm_wrapper.py (abstracts calls to Ollama’s HTTP API)
– llm_response_parser.py (parses the LLM’s JSON/text responses)
– llm_config.py (configures model name, server URL, temperature, context length)
• Data collection:
– web_scraper.py (performs DuckDuckGo searches, fetches and extracts HTML/text)
• Persistence & output:
– The code writes session logs and research content into timestamped text files in the working directory.
• Configuration & dependencies:
– requirements.txt; .gitignore, LICENSE, README.md
README for additional insights
• Workflow:
1. User issues “@” query in CLI.
2. LLM proposes N focus areas (via strategic_analysis_parser).
3. Loop over focus areas:
a. Formulate search queries
b. Call web_scraper to fetch content and URLs
c. Append results to research file
d. Generate new sub-focus areas (Self_Improving_Search)
4. On quit: aggregate all findings, call LLM for final summary, enter interactive Q&A.
• External dependencies: Ollama (local LLM), DuckDuckGo search API.
• Architecture pattern: modular, layered services—CLI → Orchestrator → Planner / Scraper / LLM → Data store.