# ifitsmanu/ii-researcher architecture

> Architecture diagram of the GitHub repository ifitsmanu/ii-researcher, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ifitsmanu/ii-researcher
- Repository: https://github.com/ifitsmanu/ii-researcher

## Overview

##### Project Type and Purpose
- The project is a full-stack deep search agent that uses custom “BAML” functions to perform intelligent web searches, content scraping, multi-step reasoning, and answer generation. It integrates several pipelines (pipeline mode and reasoning mode) with external APIs and LLM integrations.
- The README highlights rich features such as asynchronous operations, configurable LLMs (through LiteLLM), and multiple search & scraping providers (like Tavily, SerpAPI, Firecrawl, etc). This indicates that the system is designed to combine web-based data retrieval with advanced AI reasoning.

##### Main Components
a. Frontend:
- Next.js application found in the "frontend" directory.
- Interaction with the backend API; annotate that it serves the user interface and communicates with API endpoints (e.g., http://localhost:8000).

b. Backend API:
- API service started with "api.py". This service serves as the bridge between the frontend and core processing logic.
- That it handles incoming requests (e.g., search queries) from the frontend and forwards them to the core pipeline.

c. Core II-Researcher Modules:
- Pipeline components including the action handler modules (action, reflect, search, visit) and the evaluator.
- Reasoning components with sub-modules for agent reasoning, clients (e.g., openai_client), models (action, output, trace), and tools (scrapers, compressors).
- Data flows between components like the “pipeline” and “reasoning” folders, which handle question processing, multi-step reasoning, and report generation.

d. BAML Integration:
- "baml_client" and "baml_src" directories. These are used for interfacing with BAML scripts and functions which structure the outputs. Note that “baml_client” contains both synchronous and asynchronous clients, tracing, and type utilities, while “baml_src” provides the domain-specific BAML scripts used in various tasks.

e. External LLM and API Integrations:
- An external LLM component (LiteLLM in this case) acting as a proxy to underlying AI models (OpenAI’s models, gpt-4o, etc.). It is configured via environment variables and launched either as a service through Docker or with a standalone CLI.
- Mention additional external providers: SerpAPI, Tavily, Firecrawl, which provide web search data and scraping functionalities.
- That these external services feed data into the reasoning and pipeline processing modules.

f. CLI Interface:
- "cli.py" module as a separate entry point that lets users run the system in two modes (pipeline mode and reasoning mode). Indicate that this communicates directly with core processing modules.

g. Docker Deployment:
- Docker components (docker-compose.yml, Dockerfiles under docker and frontend directories) as part of the overall deployment layer. This orchestrates container-based running for the frontend, API backend, and LiteLLM server.

##### Relationships and Data Flow
- From the user interface (frontend) → API server.
- API server routes requests to the core II-Researcher modules.
- Core modules interact with BAML clients and source scripts to process questions.
- Core processing sends requests to external services (search providers, LLMs via LiteLLM) and returns processed results.
- Data flow between CLI invocations and core reasoning components for both pipeline and reasoning modes.
- Link Docker containers together, showing how services (frontend, API, LiteLLM) communicate over defined ports.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global Entities
    User("User"):::presentation

    %% Presentation Layer
    subgraph "Presentation Layer"
        FE("Frontend UI (Next.js)"):::presentation
    end

    %% Business Logic Layer
    subgraph "Business Logic Layer"
        API("Backend API (api.py)"):::business
        subgraph "II-Researcher Core"
            PIPELINE("Pipeline Components"):::business
            REASONING("Reasoning Components"):::business
        end
        CLI("CLI Interface (cli.py)"):::business
    end

    %% Integration Layer
    subgraph "Integration Layer"
        BAML_CLIENT("BAML Client"):::integration
        BAML_SRC("BAML Scripts"):::integration
        LLM_CLIENT("External LLM Client"):::integration
        PROVIDER1("SerpAPI"):::integration
        PROVIDER2("Tavily"):::integration
        PROVIDER3("Firecrawl"):::integration
        LITELLM("LiteLLM Service"):::integration
    end

    %% Infrastructure/Deployment Layer
    subgraph "Infrastructure/Deployment Layer"
        DOCKER_COMPOSE("Docker Compose (docker-compose.yml)"):::infrastructure
        API_DOCKER("API Dockerfile (docker/Dockerfile.api)"):::infrastructure
        FRONT_DOCKER("Frontend Dockerfile (frontend/Dockerfile)"):::infrastructure
        LITELLM_DOCKER("LiteLLM Dockerfile (docker/Dockerfile.litellm)"):::infrastructure
    end

    %% Connections and Data Flow
    User -->|"requests"| FE
    FE -->|"API Call (http://localhost:8000)"| API
    API -->|"routes request"| PIPELINE
    API -->|"routes request"| REASONING
    CLI -->|"invokes"| PIPELINE

    PIPELINE -->|"integrates"| BAML_CLIENT
    PIPELINE -->|"uses"| BAML_SRC
    REASONING -->|"calls"| LLM_CLIENT
    REASONING -->|"fetches data from"| PROVIDER1
    REASONING -->|"fetches data from"| PROVIDER2
    REASONING -->|"fetches data from"| PROVIDER3
    REASONING -->|"communicates with"| LITELLM
    LLM_CLIENT -->|"proxies to"| LITELLM

    DOCKER_COMPOSE -->|"orchestrates"| API_DOCKER
    DOCKER_COMPOSE -->|"orchestrates"| FRONT_DOCKER
    DOCKER_COMPOSE -->|"orchestrates"| LITELLM_DOCKER

    %% Styles
    classDef presentation fill:#AED6F1,stroke:#1F618D,stroke-width:2px,color:#000;
    classDef business fill:#A3E4D7,stroke:#148F77,stroke-width:2px,color:#000;
    classDef integration fill:#F9E79F,stroke:#CA6F1E,stroke-width:2px,color:#000;
    classDef infrastructure fill:#E6B0AA,stroke:#943126,stroke-width:2px,color:#000;

    %% Click Events
    click FE "https://github.com/ifitsmanu/ii-researcher/tree/main/frontend/"
    click API "https://github.com/ifitsmanu/ii-researcher/blob/main/api.py"
    click PIPELINE "https://github.com/ifitsmanu/ii-researcher/tree/main/ii_researcher/pipeline/"
    click BAML_CLIENT "https://github.com/ifitsmanu/ii-researcher/tree/main/baml_client/"
    click BAML_SRC "https://github.com/ifitsmanu/ii-researcher/tree/main/baml_src/"
    click LLM_CLIENT "https://github.com/ifitsmanu/ii-researcher/blob/main/ii_researcher/reasoning/clients/openai_client.py"
    click CLI "https://github.com/ifitsmanu/ii-researcher/blob/main/cli.py"
    click DOCKER_COMPOSE "https://github.com/ifitsmanu/ii-researcher/blob/main/docker-compose.yml"
    click API_DOCKER "https://github.com/ifitsmanu/ii-researcher/blob/main/docker/Dockerfile.api"
    click FRONT_DOCKER "https://github.com/ifitsmanu/ii-researcher/tree/main/frontend/Dockerfile"
    click LITELLM_DOCKER "https://github.com/ifitsmanu/ii-researcher/blob/main/docker/Dockerfile.litellm"
```
