How quickwit-oss/quickwit works
Quickwit is a cloud-storage-oriented search engine for logs and traces. Users send data through ingestion APIs or configured sources; indexing turns documents into searchable splits, which are published to storage and queried through REST-compatible search APIs. Control-plane services coordinate indexing and ingestion across nodes, while document mapping supplies schema and query interpretation. The README documents integrations and capabilities; sampled source excerpts are partial, so several cross-service links below are documented at product level rather than verified call chains.
Main entry points: indexing_pipeline.rs Indexing pipeline · control_plane.rs Control plane · src REST API handlers · service.rs Search service
Components 24
APIs and search
- REST API handlers
quickwit/quickwit-serve/src - Search service
quickwit/quickwit-search/src/service.rs - ES query parameters
quickwit/quickwit-serve/src/elasticsearch_api/model/search_query_params.rs - Jaeger API
quickwit/quickwit-jaeger/src - OTel API
quickwit/quickwit-serve/src/otlp_api/rest_handler.rs
Ingestion coordination
- Ingest API service
quickwit/quickwit-ingest/src/ingest_api_service.rs - Ingest router
quickwit/quickwit-ingest/src/ingest_v2/router.rs - Shard controller
quickwit/quickwit-control-plane/src/ingest/ingest_controller.rs
Indexing and splits
- Source factory
quickwit/quickwit-indexing/src/source/source_factory.rs - Indexing pipeline
quickwit/quickwit-indexing/src/actors/indexing_pipeline.rs - Document processor
quickwit/quickwit-indexing/src/actors/doc_processor.rs - Indexer
quickwit/quickwit-indexing/src/actors/indexer.rs - Indexing service
quickwit/quickwit-indexing/src/actors/indexing_service.rs - Merge pipeline
quickwit/quickwit-indexing/src/actors/merge_pipeline.rs - Merge planner
quickwit/quickwit-indexing/src/actors/merge_planner.rs - Split publisher
quickwit/quickwit-indexing/src/actors/publisher.rs
Mapping and storage
- Document mapper
quickwit/quickwit-doc-mapper/src/doc_mapper/doc_mapper_impl.rs - Field mapping
quickwit/quickwit-doc-mapper/src/doc_mapper/field_mapping_entry.rs - Query builder
quickwit/quickwit-doc-mapper/src/query_builder.rs - Storage interface
quickwit/quickwit-storage/src/storage.rs - S3-compatible storage
quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs
Distributed operations
- Control plane
quickwit/quickwit-control-plane/src/control_plane.rs - Control-plane metrics
quickwit/quickwit-control-plane/src/metrics.rs
Other components
- Search and ingest clients
Connections 23
- Search and ingest clients → REST API handlers: sends requests
- REST API handlers → Search service: serves searches
- REST API handlers → ES query parameters: parses queries
quickwit/quickwit-serve/src/elasticsearch_api/model/search_query_params.rs - Search and ingest clients → Ingest API service: submits documents
- Search and ingest clients → Jaeger API: sends traces
- Search and ingest clients → OTel API: sends telemetry
- Ingest API service → Ingest router: routes ingest
- Shard controller → Ingest router: coordinates shards
quickwit/quickwit-control-plane/src/ingest/ingest_controller.rs - Control plane → Shard controller: coordinates ingestion
quickwit/quickwit-control-plane/src/control_plane.rs - Control plane → Indexing service: coordinates indexing
quickwit/quickwit-control-plane/src/control_plane.rs - Source factory → Indexing pipeline: supplies sources
- Indexing pipeline → Document processor: spawns actors
quickwit/quickwit-indexing/src/actors/mod.rs - Indexing pipeline → Indexer: spawns actors
quickwit/quickwit-indexing/src/actors/mod.rs - Indexing service → Indexing pipeline: spawns pipelines
quickwit/quickwit-indexing/src/actors/mod.rs - Indexing pipeline → Split publisher: coordinates publishing
quickwit/quickwit-indexing/src/actors/indexing_pipeline.rs - Indexing service → Merge pipeline: manages merges
- Merge pipeline → Merge planner: runs merge planning
quickwit/quickwit-indexing/src/actors/merge_pipeline.rs - Document mapper → Field mapping: uses mappings
- Search service → Query builder: builds queries
- Split publisher → Storage interface: writes splits
- Search service → Storage interface: reads splits
- S3-compatible storage → Storage interface: implements storage
quickwit/quickwit-storage/src/object_storage/s3_compatible_storage.rs - Control plane → Control-plane metrics: records metrics
quickwit/quickwit-control-plane/src/control_plane.rs