How halalcloud/kagami-storage works
kagami-storage is a Haystack-inspired small-file storage system. The documented request path centers on proxying client storage requests, directory-based scheduling and metadata, and physical storage on store nodes; ops provides administrative provisioning, while pitchfork monitors store health. README descriptions establish these roles, but sampled source does not show the proxy-to-directory/store call chain, so those links are marked unverified. The graph also shows ops’ evidenced store-admin calls and ZooKeeper-backed cluster-state handling; other auxiliary utilities are omitted.
Main entry points: store_client.py Store admin client · zk_client.py Cluster state client · volume.go Physical store · efs.go Storage proxy
Components 16
Storage services
- Storage proxy
proxy/efs/efs.go - Directory service
directory/directory.go - HBase metadata
directory/hbase/hbase.go - Physical store
store/volume/volume.go - Needle records
store/needle/needle.go - Bucket management
proxy/bucket/bucket.go
Cluster control
- Store monitor
pitchfork/pitchfork.go - ZooKeeper
directory/zk/zk.go - Rebalance service
rebalance/rebalance.go
Administration
- Ops management UI
ops/views/ops.py - Store admin client
ops/commons/store_client.py - Cluster state client
ops/commons/zk_client.py
Ops internals
- Ops configuration
ops/commons/config.py - Ops state
ops/commons/global_var.py - Ops logging
ops/commons/blogging.py
Other components
- Storage client
Connections 15
- Storage client → Storage proxy: sends requests
- Storage proxy → Directory service: routes requests
- Storage proxy → Physical store: proxies storage
- Directory service → HBase metadata: manages metadata
- Store monitor → Physical store: monitors health
- Ops management UI → Store admin client: uses admin client
- Store admin client → Physical store: calls admin API
ops/commons/store_client.py - Cluster state client → ZooKeeper: reads cluster state
ops/commons/zk_client.py - Cluster state client → Ops state: updates state
ops/commons/zk_client.py - Store admin client → Ops state: updates store stats
ops/commons/store_client.py - Cluster state client → Ops configuration: uses settings
ops/commons/zk_client.py - Store admin client → Ops configuration: uses settings
ops/commons/store_client.py - Cluster state client → Ops logging: logs events
ops/commons/zk_client.py - Store admin client → Ops logging: logs failures
ops/commons/store_client.py - Physical store → Needle records: stores records