# kazdatahelp/automated-ai-web-researcher-ollama architecture

> Architecture diagram of the GitHub repository kazdatahelp/automated-ai-web-researcher-ollama, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kazdatahelp/automated-ai-web-researcher-ollama
- Repository: https://github.com/kazdatahelp/automated-ai-web-researcher-ollama

## Overview

##### Project Type & Purpose
- This is a standalone Python-based automated research assistant that leverages locally running large language models (via Ollama) to conduct real-time, structured online research.
- It automatically breaks down user queries into multiple focused research areas; then, it performs targeted web searches, scrapes results, aggregates findings, and finally produces a comprehensive summary. In addition, it supports a follow-up Q&A conversation mode based on the accumulated research.

##### Analyzing the File Structure
- There is no separate directory hierarchy; rather, the project comprises several Python files at the top level. This suggests a monolithic design for the research automation logic.
- Key files include:
- Web-LLM.py: Likely the main entry point for initiating the research session and handling user interface (CLI-based).
- research_manager.py: Expected to manage the overall research workflow, including managing focus areas, progress tokens, and the orchestration of web search cycles.
- web_scraper.py: Responsible for interacting with external websites to fetch and scrape content based on search queries.
- llm_wrapper.py, llm_response_parser.py, and llm_config.py: These files encapsulate the integration with the local LLM (Ollama), handling configuration, making API calls, and parsing responses.
- Self_Improving_Search.py & strategic_analysis_parser.py: Likely provide additional logic such as refining research queries (self-improving capabilities) and extracting deeper insights from the data.
- Files like requirements.txt, README.md, LICENSE, and .gitignore set up the necessary environment and documentation for the project.

a. Main Components:
- User Interface / CLI Input: Represents where users provide their research queries and commands (triggered by “@”, “s”, “p”, “q”, etc.).
- Research Manager: The central orchestrator that coordinates the overall research flow. It receives user input, distributes tasks, and aggregates results.
- LLM Controller:
- llm_config.py: Contains LLM settings (e.g., model name, temperature, context length) for the Ollama integration.
- llm_wrapper.py and llm_response_parser.py: Abstract and wrap the interactions with the LLM, including sending queries and processing responses.
- Web Scraper / Search Module: Responsible for formulating search URLs, triggering web searches, and scraping and parsing information from selected web pages.
- Self-Improvement / Analysis Module: (via Self_Improving_Search.py and strategic_analysis_parser.py) Further refines research queries and analysis of gathered information.
- Output Generator: Handles the aggregation of research results and generates the final summary report as well as the research session’s text file (documenting content and source URLs).

b. Relationships and Data Flow:
- The User Interface passes the initial research query and commands to the Research Manager.
- The Research Manager decomposes the query into multiple focused areas.
- For each focus area, the Research Manager sends targeted searches to the Web Scraper module.
- Parallelly, the LLM Controller is called (via llm_wrapper) to:
- Query,
- Assist in generating search queries or focus follow-ups,
- Parse responses using llm_response_parser.
- The Web Scraper fetches relevant content from the internet and returns the data to the Research Manager.
- The Self-Improvement Module refines the process by re-evaluating and generating new focus areas if needed.
- Once content is collected, the Research Manager uses the LLM again to synthesize and compile the final summary.
- Finally, the summary and the complete research document are output and (optionally) the system moves to a conversation mode for Q&A.

c. Architectural Patterns & Design Principles:
- Modular design even though all modules live in a single file directory: separating responsibilities (UI, orchestration, LLM integration, web scraping, and output generation).
- Clear service boundaries between the web scraping logic and the LLM interfacing, supporting maintainability and potential future refactoring.
- A feedback loop for self-improvement (query refinement) and dynamic research expansion.
- Use of configuration (in llm_config.py) enabling flexibility with different LLM models and parameters.

d. Relevant Technologies & Frameworks:
- Python (core language) for the entire implementation.
- Ollama as the local LLM engine.
- Standard libraries for web-request handling and text parsing (likely via requests or similar modules internally).
- No explicit web framework appears to be used; the script is driven by CLI inputs.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Main Components
    UI["User Interface / Command Input"]:::ui
    RM["Research Manager / Orchestrator"]:::manager
    OG["Output Generator / Research Document"]:::manager

    %% LLM Controller Subgraph
    subgraph "LLM Controller"
        LLM_Config["llm_config.py"]:::external
        LLM_Wrapper["llm_wrapper.py"]:::external
        LLM_Response["llm_response_parser.py"]:::external
    end

    %% Web Scraper Module
    WS["Web Scraper Module"]:::external

    %% Self-Improvement & Analysis Module Subgraph
    subgraph "Self-Improvement & Analysis Module"
        SI_Search["Self_Improving_Search.py"]:::external
        SA_Parser["strategic_analysis_parser.py"]:::external
    end

    %% Data Flows
    UI -->|"SubmitQuery"| RM
    RM -->|"LLMAnalysis"| LLM_Config
    RM -->|"InitiateSearch"| WS
    WS -->|"ReturnResult"| RM
    RM -->|"FeedbackLoop"| SI_Search
    SI_Search -->|"RefinedFocus"| RM
    RM -->|"GenerateSummary"| OG

    %% Click Events
    click UI "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/Web-LLM.py"
    click RM "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/research_manager.py"
    click LLM_Config "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/llm_config.py"
    click LLM_Wrapper "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/llm_wrapper.py"
    click LLM_Response "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/llm_response_parser.py"
    click WS "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/web_scraper.py"
    click SI_Search "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/Self_Improving_Search.py"
    click SA_Parser "https://github.com/kazdatahelp/automated-ai-web-researcher-ollama/blob/main/strategic_analysis_parser.py"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef manager fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef external fill:#F08080,stroke:#000,stroke-width:2px;
```
