# intelligent-internet/ii-researcher architecture

> Architecture diagram of the GitHub repository intelligent-internet/ii-researcher, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/intelligent-internet/ii-researcher
- Repository: https://github.com/intelligent-internet/ii-researcher

## Overview

##### Main Components
- API Backend (api.py & Docker/Dockerfile.api): This is the service that exposes endpoints (running on http://localhost:8000) to serve both the CLI commands and calls from the frontend.
- Frontend (frontend folder): A Next.js/React application that communicates with the backend API and provides a user interface on http://localhost:3000.
- LiteLLM Server (configured via Docker Compose and litellm_config.yaml in README): Acts as a local LLM proxy server (running on http://localhost:4000) that integrates with external language models.
- Core Research Modules (ii_researcher directory): Contains the pipeline, reasoning modules, and tool implementations. This part encapsulates the primary business logic for processing questions, running deep search workflows, and coordinating between different actions (search, scrape, analyze).
- BAML Modules (baml_client and baml_src directories): These components manage structured operations and outputs. The client library supports both async and sync operations and ties into the overall system, possibly leveraging the BAML (BoundaryML) framework for building structured outputs.
- CLI (cli.py): Provides another user interface mode (command-line) that lets users initiate deep search tasks in either pipeline or reasoning modes.

##### Relationships and Interactions
- The Frontend interacts with the API backend over HTTP, with clear separation of concerns and well-defined endpoints.
- The API backend acts as a coordination hub: receiving requests from the frontend or CLI, then invoking the core research modules (ii_researcher) for processing.
- Within the core research layers, pipelines and reasoning modules interact with various tools provided under ii_researcher/tools (e.g., scrapers and compressors) to execute multi-step searches.
- The BAML client and source libraries are likely used by these core modules to generate, parse, or validate structured outputs.

##### Architectural Patterns and Technologies
- Separation of Concerns: Distinct directories indicate clear layers (frontend vs backend vs core logic).
- Microservices / Containerization: Docker Compose orchestrates the containers for the frontend, API, and LiteLLM server.
- Async Operation: Both asynchronous and synchronous clients are available (visible in the baml_client package) and are used to improve performance during web scraping and API interactions.
- Modular Pipeline & Reasoning Architecture: The ii_researcher package is split into pipeline, reasoning, and tools—each handling different steps of the deep search workflow.
- Use of External APIs: Integration with multiple search and scrape providers, configurable via environment variables.

##### Final Touches
- Every component is clearly labeled.
- Essential elements and data flows as described in the README.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Subgraph: User Interfaces
    subgraph "User Interfaces"
        FE["Frontend (Next.js)"]:::frontend
        CLI["CLI Interface"]:::cli
    end

    %% API Backend
    API["API Backend (Python, FastAPI)"]:::backend

    %% Subgraph: Core Processing
    subgraph "Core Modules (ii_researcher)"
        CR["Core Processing"]:::core
        PM["Pipeline Mode"]:::coreNote
        RM["Reasoning Mode"]:::coreNote
    end

    %% Subgraph: BAML Components
    subgraph "BAML Components"
        BC1["BAML Client"]:::baml
        BC2["BAML Source"]:::baml
    end

    %% External and Supporting Services
    LLM["LiteLLM Server"]:::litellm
    EXT1["Search Providers"]:::external
    EXT2["Scraper Providers"]:::external

    %% Connections from User Interfaces to API Backend
    FE -->|"HTTP_request"| API
    CLI -->|"CLI_command"| API

    %% Connection from API Backend to Core Modules
    API -->|"invoke_processing"| CR

    %% Core Modules interactions with external services and BAML components
    CR -->|"search_API_call"| EXT1
    CR -->|"scrape_request"| EXT2
    CR -->|"uses"| BC1
    CR -->|"uses"| BC2

    %% Bidirectional connection between API Backend and LiteLLM Server
    API <-->|"LLM_req/resp"| LLM

    %% Styling for vertical layout connectors from Core Modules subnotes
    PM --- CR
    RM --- CR

    %% Click events
    click FE "https://github.com/intelligent-internet/ii-researcher/tree/main/frontend"
    click CLI "https://github.com/intelligent-internet/ii-researcher/blob/main/cli.py"
    click API "https://github.com/intelligent-internet/ii-researcher/blob/main/api.py"
    click CR "https://github.com/intelligent-internet/ii-researcher/tree/main/ii_researcher"
    click BC1 "https://github.com/intelligent-internet/ii-researcher/tree/main/baml_client"
    click BC2 "https://github.com/intelligent-internet/ii-researcher/tree/main/baml_src"
    click LLM "https://github.com/intelligent-internet/ii-researcher/blob/main/docker/Dockerfile.litellm"

    %% Class definitions
    classDef frontend fill:#fde2e2,stroke:#b30000,stroke-width:2px;
    classDef cli fill:#e8f1f2,stroke:#31708e,stroke-width:2px;
    classDef backend fill:#e2f7d6,stroke:#3c763d,stroke-width:2px;
    classDef core fill:#d9edf7,stroke:#31708f,stroke-width:2px;
    classDef baml fill:#fcf8e3,stroke:#8a6d3b,stroke-width:2px;
    classDef litellm fill:#f2dede,stroke:#a94442,stroke-width:2px;
    classDef external fill:#eee,stroke:#aaa,stroke-width:2px;
    classDef coreNote fill:#ffffff,stroke:#ccc,stroke-dasharray: 5 5,stroke-width:2px;
```
