# karust/openserp architecture

> Architecture diagram of the GitHub repository karust/openserp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karust/openserp
- Repository: https://github.com/karust/openserp

## Overview

OpenSERP is a SERP API and CLI that returns structured results from six search engines, with multi-engine search, image search, and optional page extraction. The main path is a CLI or HTTP request through command/API handling to engine-specific search, then normalized results; extraction can fetch clean page content, while megasearch merges and deduplicates results. The tree and excerpts also show shared browser and HTTP-client infrastructure. Some server and runtime wiring is unsampled, so uncertain links are omitted or marked unverified.

Main entry points: [`core/server.go`](https://github.com/karust/openserp/blob/main/core/server.go) (Megasearch), [`core/browser.go`](https://github.com/karust/openserp/blob/main/core/browser.go) (Browser runtime), [`extract/extractor.go`](https://github.com/karust/openserp/blob/main/extract/extractor.go) (Page extraction), [`core/http_client.go`](https://github.com/karust/openserp/blob/main/core/http_client.go) (HTTP clients)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Interfaces"]
  node_main["Process entry<br/>[main.go]"]
  node_cli["CLI search<br/>[search.go]"]
  node_commands["Command setup<br/>[root.go]"]
  node_api["HTTP API<br/>[server.go]"]
end

subgraph group_search["Search orchestration"]
  node_mega["Megasearch<br/>[server.go]"]
  node_clusters["Result clusters<br/>[clusters.go]"]
  node_extract_api["Extraction API<br/>[server.go]"]
  node_extractor["Page extraction<br/>[extractor.go]"]
  node_content["Clean content<br/>[content.go]"]
end

subgraph group_engines["Engine adapters"]
  node_google["Google search<br/>[search.go]"]
  node_yandex["Yandex search<br/>[search.go]"]
  node_baidu["Baidu search<br/>[search.go]"]
  node_bing["Bing search<br/>[search.go]"]
  node_ddg["DuckDuckGo search<br/>[search.go]"]
  node_ecosia["Ecosia search<br/>[search.go]"]
end

subgraph group_runtime["Runtime services"]
  node_browser["Browser runtime<br/>[browser.go]"]
  node_http["HTTP clients<br/>[http_client.go]"]
  node_cache[("Response cache<br/>[cache.go]")]
  node_proxy["Proxy handling<br/>[proxy.go]"]
  node_resilience["Resilient requests<br/>[resilient.go]"]
end

node_user(("CLI/API user"))

node_user -->|"runs search"| node_cli
node_user -->|"sends request"| node_api
node_main -->|"executes"| node_commands
node_user -.->|"starts process"| node_main
node_cli -->|"optionally extracts"| node_extractor
node_api -->|"routes search"| node_mega
node_api -->|"routes extraction"| node_extract_api
node_mega -->|"forms clusters"| node_clusters
node_mega -->|"queries engine"| node_google
node_mega -->|"queries engine"| node_yandex
node_mega -->|"queries engine"| node_baidu
node_mega -->|"queries engine"| node_bing
node_mega -->|"queries engine"| node_ddg
node_mega -->|"queries engine"| node_ecosia
node_google -->|"navigates pages"| node_browser
node_yandex -->|"navigates pages"| node_browser
node_baidu -->|"navigates pages"| node_browser
node_bing -->|"navigates pages"| node_browser
node_ddg -->|"navigates pages"| node_browser
node_ecosia -->|"navigates pages"| node_browser
node_http -->|"uses profile"| node_browser
node_extract_api -.->|"fetches content"| node_extractor
node_extractor -.->|"cleans pages"| node_content
node_api -.->|"checks responses"| node_cache
node_browser -.->|"uses proxy"| node_proxy
node_http -.->|"uses retries"| node_resilience

click node_main "https://github.com/karust/openserp/blob/main/main.go"
click node_cli "https://github.com/karust/openserp/blob/main/cmd/search.go"
click node_commands "https://github.com/karust/openserp/blob/main/cmd/root.go"
click node_api "https://github.com/karust/openserp/blob/main/core/server.go"
click node_mega "https://github.com/karust/openserp/blob/main/core/server.go"
click node_clusters "https://github.com/karust/openserp/blob/main/core/clusters.go"
click node_google "https://github.com/karust/openserp/blob/main/google/search.go"
click node_yandex "https://github.com/karust/openserp/blob/main/yandex/search.go"
click node_baidu "https://github.com/karust/openserp/blob/main/baidu/search.go"
click node_bing "https://github.com/karust/openserp/blob/main/bing/search.go"
click node_ddg "https://github.com/karust/openserp/blob/main/duckduckgo/search.go"
click node_ecosia "https://github.com/karust/openserp/blob/main/ecosia/search.go"
click node_browser "https://github.com/karust/openserp/blob/main/core/browser.go"
click node_http "https://github.com/karust/openserp/blob/main/core/http_client.go"
click node_extract_api "https://github.com/karust/openserp/blob/main/core/server.go"
click node_extractor "https://github.com/karust/openserp/blob/main/extract/extractor.go"
click node_content "https://github.com/karust/openserp/blob/main/extract/content.go"
click node_cache "https://github.com/karust/openserp/blob/main/core/cache.go"
click node_proxy "https://github.com/karust/openserp/blob/main/core/proxy.go"
click node_resilience "https://github.com/karust/openserp/blob/main/core/resilient.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_cli,node_commands,node_api,node_user toneBlue
class node_mega,node_clusters,node_extract_api,node_extractor,node_content toneAmber
class node_google,node_yandex,node_baidu,node_bing,node_ddg,node_ecosia toneMint
class node_browser,node_http,node_cache,node_proxy,node_resilience toneRose
```

## Components

### Interfaces

- [Process entry](https://github.com/karust/openserp/blob/main/main.go): `main.go`
- [CLI search](https://github.com/karust/openserp/blob/main/cmd/search.go): `cmd/search.go`
- [Command setup](https://github.com/karust/openserp/blob/main/cmd/root.go): `cmd/root.go`
- [HTTP API](https://github.com/karust/openserp/blob/main/core/server.go): `core/server.go`

### Search orchestration

- [Megasearch](https://github.com/karust/openserp/blob/main/core/server.go): `core/server.go`
- [Result clusters](https://github.com/karust/openserp/blob/main/core/clusters.go): `core/clusters.go`
- [Extraction API](https://github.com/karust/openserp/blob/main/core/server.go): `core/server.go`
- [Page extraction](https://github.com/karust/openserp/blob/main/extract/extractor.go): `extract/extractor.go`
- [Clean content](https://github.com/karust/openserp/blob/main/extract/content.go): `extract/content.go`

### Engine adapters

- [Google search](https://github.com/karust/openserp/blob/main/google/search.go): `google/search.go`
- [Yandex search](https://github.com/karust/openserp/blob/main/yandex/search.go): `yandex/search.go`
- [Baidu search](https://github.com/karust/openserp/blob/main/baidu/search.go): `baidu/search.go`
- [Bing search](https://github.com/karust/openserp/blob/main/bing/search.go): `bing/search.go`
- [DuckDuckGo search](https://github.com/karust/openserp/blob/main/duckduckgo/search.go): `duckduckgo/search.go`
- [Ecosia search](https://github.com/karust/openserp/blob/main/ecosia/search.go): `ecosia/search.go`

### Runtime services

- [Browser runtime](https://github.com/karust/openserp/blob/main/core/browser.go): `core/browser.go`
- [HTTP clients](https://github.com/karust/openserp/blob/main/core/http_client.go): `core/http_client.go`
- [Response cache](https://github.com/karust/openserp/blob/main/core/cache.go): `core/cache.go`
- [Proxy handling](https://github.com/karust/openserp/blob/main/core/proxy.go): `core/proxy.go`
- [Resilient requests](https://github.com/karust/openserp/blob/main/core/resilient.go): `core/resilient.go`

### Other components

- **CLI/API user**

## Connections

- CLI/API user → CLI search: runs search (evidence: `README.md`)
- CLI/API user → HTTP API: sends request (evidence: `README.md`)
- Process entry → Command setup: executes (evidence: `main.go`)
- CLI/API user → Process entry: starts process
- CLI search → Page extraction: optionally extracts (evidence: `cmd/search.go`)
- HTTP API → Megasearch: routes search (evidence: `core/server.go`)
- HTTP API → Extraction API: routes extraction (evidence: `core/server.go`)
- Megasearch → Result clusters: forms clusters (evidence: `README.md`)
- Megasearch → Google search: queries engine (evidence: `README.md`)
- Megasearch → Yandex search: queries engine (evidence: `README.md`)
- Megasearch → Baidu search: queries engine (evidence: `README.md`)
- Megasearch → Bing search: queries engine (evidence: `README.md`)
- Megasearch → DuckDuckGo search: queries engine (evidence: `README.md`)
- Megasearch → Ecosia search: queries engine (evidence: `README.md`)
- Google search → Browser runtime: navigates pages (evidence: `google/search.go`)
- Yandex search → Browser runtime: navigates pages (evidence: `yandex/search.go`)
- Baidu search → Browser runtime: navigates pages (evidence: `baidu/search.go`)
- Bing search → Browser runtime: navigates pages (evidence: `bing/search.go`)
- DuckDuckGo search → Browser runtime: navigates pages (evidence: `duckduckgo/search.go`)
- Ecosia search → Browser runtime: navigates pages (evidence: `ecosia/search.go`)
- HTTP clients → Browser runtime: uses profile (evidence: `core/http_client.go`)
- Extraction API → Page extraction: fetches content
- Page extraction → Clean content: cleans pages
- HTTP API → Response cache: checks responses
- Browser runtime → Proxy handling: uses proxy
- HTTP clients → Resilient requests: uses retries
