How theproject24/zynsearch works
ZynSearch is a Rust search engine exposed through a CLI, REST and gRPC server, and thin JS, Python, and Go SDKs. Documents are collected from local directories or S3, normalized and analyzed, then inserted into an inverted index; queries are coordinated across shards, retrieved and ranked, and formatted for callers. The graph preserves documented REST, gRPC, streaming, bulk-index, persistence, and SDK capabilities. Several source excerpts are partial or unsampled, so uncertain internal wiring is omitted.
Main entry points: ingestion.rs Source ingestion · engine.rs Search engine · http.rs REST gateway · query_pipeline.rs Query coordinator
Components 22
Clients
- CLI
crates/zynsearch-cli/src/main.rs - Language SDKs
sdks
Document ingestion
- Source ingestion
crates/zynsearch-core/src/ingestion.rs - Document normalization
crates/zynsearch-core/src/document_ingest.rs - Directory crawler
crates/zynsearch-core/src/crawler.rs
Search core
- Search engine
crates/zynsearch-core/src/engine.rs - Text analyzer
crates/zynsearch-core/src/analyzer.rs - Inverted index
crates/zynsearch-core/src/index.rs - Query coordinator
crates/zynsearch-core/src/query_pipeline.rs - Retrieval engine
crates/zynsearch-core/src/searcher.rs - BM25 ranking
crates/zynsearch-core/src/bm25.rs - Top-K collector
crates/zynsearch-core/src/top_k.rs
Server transports
- Server runtime
crates/zynsearch-server/src/main.rs - REST gateway
crates/zynsearch-server/src/http.rs - gRPC contract
proto/zynsearch/v1/zynsearch.proto
Persistence
- Index persistence
crates/zynsearch-core/src/storage.rs
Other components
- Search user
- Local corpus
- S3 objects
- CLI operator
- API caller
- Search results
Connections 32
- CLI operator → CLI: runs
- CLI → Source ingestion: starts ingestion
- Local corpus → Source ingestion: supplies files
- S3 objects → Source ingestion: supplies objects
- Source ingestion → Directory crawler: crawls directory
- Source ingestion → Document normalization: normalizes content
- Source ingestion → Search engine: indexes documents
- Document normalization → Text analyzer: provides text
- Search engine → Text analyzer: analyzes text
- Search engine → Inverted index: updates index
- Search engine → Retrieval engine: executes search
- Retrieval engine → Text analyzer: analyzes query
- Retrieval engine → Inverted index: reads postings
- Retrieval engine → BM25 ranking: scores matches
- Query coordinator → Search engine: dispatches shards
- Query coordinator → Top-K collector: collects results
- Query coordinator → Search results: formats results
- CLI → Query coordinator: submits query
- CLI → Index persistence: loads or saves index
- Source ingestion → Index persistence: persists index
- Search user → Language SDKs: uses client
- Language SDKs → REST gateway: sends REST requests
- API caller → REST gateway: calls API
- API caller → gRPC contract: uses RPC contract
- Server runtime → REST gateway: starts gateway
- Server runtime → Query coordinator: coordinates queries
- REST gateway → Search engine: handles engine actions
- REST gateway → Query coordinator: runs searches
- gRPC contract → Server runtime: defines RPCs
- Server runtime → Index persistence: hydrates or persists
- REST gateway → Search results: serializes response
- gRPC contract → Search results: defines result messages