How phonepe/solus works
Solus is a Java library for distributed, TTL-based deduplication using sharded Bloom filters. A client configures a storage context and uses SolusEngine to register a deduper, then check, add, or atomically add entities. The data-command layer derives Bloom-filter bit positions and shards entities before delegating reads and updates to the selected backend. Registration metadata and filter data are separate concerns; README capabilities also include batch operations and DC/XDC levels. Backend links reflect documented Aerospike and HBase support; unsampled CRUD and HBase data-store internals limit finer-grained wiring.
Main entry points: SolusEngine.java Solus Engine · DeDuperDataCommands.java Data Commands · IDeDuperMetaStore.java Metadata Store API · IDeDuperDataStore.java Data Store API
Components 20
Library API
- Solus Engine
src/main/java/com/phonepe/solus/SolusEngine.java - CRUD Commands
src/main/java/com/phonepe/solus/commands/impl/DeDuperCrudCommands.java - Data Commands
src/main/java/com/phonepe/solus/commands/impl/DeDuperDataCommands.java
Deduplication Logic
- Deduper Config
src/main/java/com/phonepe/solus/config/DeDuperConfig.java - DC / XDC
src/main/java/com/phonepe/solus/config/DeDuperLevel.java - Deduper State
src/main/java/com/phonepe/solus/DeDuper.java - Shard Calculator
src/main/java/com/phonepe/solus/shard/ShardCalculator.java - Bloom Bit Positions
src/main/java/com/phonepe/solus/filter/impl/EntityWithBitPositions.java
Deduper Metadata
- Metadata Store API
src/main/java/com/phonepe/solus/store/IDeDuperMetaStore.java - Aerospike Metadata
src/main/java/com/phonepe/solus/store/aerospike/AerospikeDeDuperMetaStore.java - HBase Metadata
src/main/java/com/phonepe/solus/store/hbase/HBaseDeDuperMetaStore.java
Storage Backends
- Metadata Backend
- Data Store API
src/main/java/com/phonepe/solus/store/IDeDuperDataStore.java - Aerospike Filter Data
src/main/java/com/phonepe/solus/store/aerospike/AerospikeDeDuperDataStore.java - HBase Filter Data
src/main/java/com/phonepe/solus/store/hbase/HBaseDeDuperDataStore.java - Aerospike Context
src/main/java/com/phonepe/solus/store/context/impl/AerospikeStorageContext.java - HBase Context
src/main/java/com/phonepe/solus/store/context/impl/HBaseStorageContext.java
Other components
- Library Client
- Aerospike
- HBase
Connections 21
- Library Client → Solus Engine: calls
- Solus Engine → CRUD Commands: registers via
- Solus Engine → CRUD Commands: loads deduper
- Solus Engine → Data Commands: runs operations
- Solus Engine → Deduper Config: accepts config
- CRUD Commands → Metadata Store API: manages metadata
- Metadata Store API → Aerospike Metadata: Aerospike implementation
- Metadata Store API → HBase Metadata: HBase implementation
- Aerospike Metadata → Metadata Backend: reads/writes
- HBase Metadata → Metadata Backend: reads/writes
- Data Commands → Shard Calculator: calculates shard
- Data Commands → Bloom Bit Positions: derives bit positions
- Data Commands → Data Store API: reads/updates
- Data Commands → Deduper State: uses configuration
- Data Store API → Aerospike Filter Data: Aerospike implementation
- Data Store API → HBase Filter Data: HBase implementation
- Aerospike Filter Data → Aerospike: reads/writes TTL bins
- HBase Filter Data → HBase: reads/writes filter data
- Aerospike Context → Aerospike Filter Data: configures backend
- HBase Context → HBase Filter Data: configures backend
- Deduper State → DC / XDC: selects level