# yelp/nrtsearch architecture

> Architecture diagram of the GitHub repository yelp/nrtsearch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yelp/nrtsearch
- Repository: https://github.com/yelp/nrtsearch

## Overview

nrtSearch is a Lucene-backed search server exposed primarily through gRPC, with optional REST APIs. The user workflow is to create and start an index, register its field schema, stream documents for indexing, and issue searches whose results are returned through the search API. Distinct operational paths include primary-to-replica near-real-time replication and remote backup/restore. The graph preserves major documented search capabilities and plugin extensibility. Some relationships rely on README documentation or SOURCE INDEX references; unsampled implementations and wiring are not asserted.

Main entry points: [`src/main/java/com/yelp/nrtsearch/server/handler/SearchHandler.java`](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/handler/SearchHandler.java) (Search execution), [`src/main/java/com/yelp/nrtsearch/server/handler`](https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/handler) (Request handlers), [`src/main/java/com/yelp/nrtsearch/server/grpc/NrtsearchServer.java`](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/grpc/NrtsearchServer.java) (gRPC server), [`src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java`](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java) (Streaming search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Client API"]
  node_client["gRPC client"]
  node_server["gRPC server"]
  node_handlers["Request handlers"]
end

subgraph group_index["Indexing and Search"]
  node_global["Global index state<br/>[GlobalState.java]"]
  node_fields["Field definitions<br/>[FieldDef.java]"]
  node_fieldfactory["Field definition factory"]
  node_indexstate["Index state<br/>[IndexState.java]"]
  node_shard["Shard and writer<br/>[ShardState.java]"]
  node_searchstream["Streaming search"]
  node_search["Search execution<br/>[SearchHandler.java]"]
  node_searchcontext["Search context<br/>[SearchContext.java]"]
  node_query["Query and retrieval"]
  node_collectors["Facets and collectors"]
  node_highlights["Highlighting"]
end

subgraph group_replication["Replication and Durability"]
  node_primary["Primary replication"]
  node_replicaclient["Replication client"]
  node_nrtdata["NRT data manager"]
  node_remote["Remote storage<br/>[RemoteBackend.java]"]
end

subgraph group_extensions["Extensions and Operations"]
  node_config["Server configuration"]
  node_plugins["Plugin loading"]
  node_metrics["Monitoring"]
end

node_user(("Search client"))

node_user -->|"issues requests"| node_client
node_client -->|"calls gRPC"| node_server
node_server -.->|"routes requests"| node_handlers
node_handlers -->|"accesses state"| node_global
node_handlers -->|"resolves index"| node_indexstate
node_handlers -.->|"registers fields"| node_fieldfactory
node_fieldfactory -->|"creates definitions"| node_fields
node_indexstate -.->|"owns shards"| node_shard
node_searchstream -->|"delegates search"| node_search
node_searchstream -->|"builds context"| node_searchcontext
node_search -.->|"uses context"| node_searchcontext
node_search -.->|"executes queries"| node_query
node_search -.->|"collects results"| node_collectors
node_search -.->|"formats highlights"| node_highlights
node_primary -.->|"sends updates"| node_replicaclient
node_primary -->|"publishes segments"| node_nrtdata
node_nrtdata -->|"uploads index files"| node_remote
node_config -->|"configures plugins"| node_plugins
node_config -.->|"configures service"| node_server
node_plugins -->|"registers metrics"| node_metrics

click node_client "https://github.com/yelp/nrtsearch/blob/main/clientlib/src/main/java/com/yelp/nrtsearch/clientlib/grpc/NrtSearchClient.java"
click node_server "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/grpc/NrtsearchServer.java"
click node_config "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/config/NrtsearchConfig.java"
click node_handlers "https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/handler"
click node_global "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/state/GlobalState.java"
click node_fields "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/field/FieldDef.java"
click node_fieldfactory "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/field/FieldDefCreator.java"
click node_indexstate "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/index/IndexState.java"
click node_shard "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/index/ShardState.java"
click node_searchstream "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java"
click node_search "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/handler/SearchHandler.java"
click node_searchcontext "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/search/SearchContext.java"
click node_query "https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/query"
click node_collectors "https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/search/collectors"
click node_highlights "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/highlights/HighlighterService.java"
click node_primary "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/nrt/NRTPrimaryNode.java"
click node_replicaclient "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/grpc/ReplicationServerClient.java"
click node_nrtdata "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/nrt/NrtDataManager.java"
click node_remote "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/remote/RemoteBackend.java"
click node_plugins "https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/plugins/PluginsService.java"
click node_metrics "https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/monitoring"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_client,node_server,node_handlers,node_user toneBlue
class node_global,node_fields,node_fieldfactory,node_indexstate,node_shard,node_searchstream,node_search,node_searchcontext,node_query,node_collectors,node_highlights toneAmber
class node_primary,node_replicaclient,node_nrtdata,node_remote toneMint
class node_config,node_plugins,node_metrics toneRose
```

## Components

### Client API

- [gRPC client](https://github.com/yelp/nrtsearch/blob/main/clientlib/src/main/java/com/yelp/nrtsearch/clientlib/grpc/NrtSearchClient.java): `clientlib/src/main/java/com/yelp/nrtsearch/clientlib/grpc/NrtSearchClient.java`
- [gRPC server](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/grpc/NrtsearchServer.java): `src/main/java/com/yelp/nrtsearch/server/grpc/NrtsearchServer.java`
- [Request handlers](https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/handler): `src/main/java/com/yelp/nrtsearch/server/handler`

### Indexing and Search

- [Global index state](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/state/GlobalState.java): `src/main/java/com/yelp/nrtsearch/server/state/GlobalState.java`
- [Field definitions](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/field/FieldDef.java): `src/main/java/com/yelp/nrtsearch/server/field/FieldDef.java`
- [Field definition factory](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/field/FieldDefCreator.java): `src/main/java/com/yelp/nrtsearch/server/field/FieldDefCreator.java`
- [Index state](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/index/IndexState.java): `src/main/java/com/yelp/nrtsearch/server/index/IndexState.java`
- [Shard and writer](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/index/ShardState.java): `src/main/java/com/yelp/nrtsearch/server/index/ShardState.java`
- [Streaming search](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java): `src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java`
- [Search execution](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/handler/SearchHandler.java): `src/main/java/com/yelp/nrtsearch/server/handler/SearchHandler.java`
- [Search context](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/search/SearchContext.java): `src/main/java/com/yelp/nrtsearch/server/search/SearchContext.java`
- [Query and retrieval](https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/query): `src/main/java/com/yelp/nrtsearch/server/query`
- [Facets and collectors](https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/search/collectors): `src/main/java/com/yelp/nrtsearch/server/search/collectors`
- [Highlighting](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/highlights/HighlighterService.java): `src/main/java/com/yelp/nrtsearch/server/highlights/HighlighterService.java`

### Replication and Durability

- [Primary replication](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/nrt/NRTPrimaryNode.java): `src/main/java/com/yelp/nrtsearch/server/nrt/NRTPrimaryNode.java`
- [Replication client](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/grpc/ReplicationServerClient.java): `src/main/java/com/yelp/nrtsearch/server/grpc/ReplicationServerClient.java`
- [NRT data manager](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/nrt/NrtDataManager.java): `src/main/java/com/yelp/nrtsearch/server/nrt/NrtDataManager.java`
- [Remote storage](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/remote/RemoteBackend.java): `src/main/java/com/yelp/nrtsearch/server/remote/RemoteBackend.java`

### Extensions and Operations

- [Server configuration](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/config/NrtsearchConfig.java): `src/main/java/com/yelp/nrtsearch/server/config/NrtsearchConfig.java`
- [Plugin loading](https://github.com/yelp/nrtsearch/blob/main/src/main/java/com/yelp/nrtsearch/server/plugins/PluginsService.java): `src/main/java/com/yelp/nrtsearch/server/plugins/PluginsService.java`
- [Monitoring](https://github.com/yelp/nrtsearch/tree/main/src/main/java/com/yelp/nrtsearch/server/monitoring): `src/main/java/com/yelp/nrtsearch/server/monitoring`

### Other components

- **Search client**

## Connections

- Search client → gRPC client: issues requests
- gRPC client → gRPC server: calls gRPC
- gRPC server → Request handlers: routes requests
- Request handlers → Global index state: accesses state (evidence: `src/main/java/com/yelp/nrtsearch/server/handler/DeleteByQueryHandler.java`)
- Request handlers → Index state: resolves index (evidence: `src/main/java/com/yelp/nrtsearch/server/handler/DeleteByQueryHandler.java`)
- Request handlers → Field definition factory: registers fields
- Field definition factory → Field definitions: creates definitions (evidence: `src/main/java/com/yelp/nrtsearch/server/field/IndexableFieldDef.java`)
- Index state → Shard and writer: owns shards (evidence: `src/main/java/com/yelp/nrtsearch/server/index/IndexState.java`)
- Streaming search → Search execution: delegates search (evidence: `src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java`)
- Streaming search → Search context: builds context (evidence: `src/main/java/com/yelp/nrtsearch/server/handler/stream/SearchStreamHandler.java`)
- Search execution → Search context: uses context (evidence: `src/main/java/com/yelp/nrtsearch/server/handler/SearchHandler.java`)
- Search execution → Query and retrieval: executes queries
- Search execution → Facets and collectors: collects results
- Search execution → Highlighting: formats highlights
- Primary replication → Replication client: sends updates
- Primary replication → NRT data manager: publishes segments (evidence: `src/main/java/com/yelp/nrtsearch/server/nrt/NrtDataManager.java`)
- NRT data manager → Remote storage: uploads index files (evidence: `src/main/java/com/yelp/nrtsearch/server/nrt/NrtDataManager.java`)
- Server configuration → Plugin loading: configures plugins (evidence: `src/main/java/com/yelp/nrtsearch/server/plugins/PluginsService.java`)
- Server configuration → gRPC server: configures service
- Plugin loading → Monitoring: registers metrics (evidence: `src/main/java/com/yelp/nrtsearch/server/plugins/PluginsService.java`)