Skip to content

BlobRedundancyManager: Implement RocksDB Event ListenerΒ #4089

@makr-code

Description

@makr-code

Context

This issue implements the roadmap item 'BlobRedundancyManager: Implement RocksDB Event Listener' for the storage domain. It is sourced from the consolidated roadmap under 🟒 Low Priority β€” Future (v1.9.0+) and targets milestone v1.8.0.

Primary detail section: BlobRedundancyManager: Implement RocksDB Event Listener

Goal

Deliver the scoped changes for BlobRedundancyManager: Implement RocksDB Event Listener in src/storage/ and complete the linked detail section in a release-ready state for v1.8.0.

Detailed Scope

BlobRedundancyManager: Implement RocksDB Event Listener

Priority: Low
Target Version: v1.8.0

blob_redundancy_manager.cpp line 768: "RocksDB listener not implemented" β€” the redundancy manager cannot react to RocksDB compaction events (SST file deletions) to trigger re-replication of blobs that lose their storage backing.

Implementation Notes:

  • [ ] Implement a BlobRedundancyEventListener subclassing rocksdb::EventListener; override OnTableFileDeleted to trigger re-replication of any blobs whose backing SST was deleted.
  • [ ] Register the listener via rocksdb::Options::listeners at database open time.

Priority: High
Target Version: v1.7.0

Raft-based distributed transactions across multiple nodes.

Features:

  • Two-phase commit (2PC) protocol
  • Raft consensus for transaction coordination
  • Cross-shard atomic operations
  • Automatic deadlock detection

API:

DistributedTransactionManager dtx_manager(nodes);
auto tx = dtx_manager.beginDistributedTransaction();

// Write to multiple shards
tx->put("shard1:key1", "value1");
tx->put("shard2:key2", "value2");

// Commit atomically across shards
tx->commit();  // 2PC protocol

Use Cases:

  • Multi-tenant data isolation
  • Geographic data distribution
  • Horizontal scaling

Acceptance Criteria

  • Implement a BlobRedundancyEventListener subclassing rocksdb::EventListener; override OnTableFileDeleted to trigger re-replication of any blobs whose backing SST was deleted.
  • Register the listener via rocksdb::Options::listeners at database open time.
  • Two-phase commit (2PC) protocol
  • Raft consensus for transaction coordination
  • Cross-shard atomic operations
  • Automatic deadlock detection
  • Multi-tenant data isolation
  • Geographic data distribution
  • Horizontal scaling

Relationships

References

  • src/ROADMAP.md
  • src/storage/FUTURE_ENHANCEMENTS.md#blobredundancymanager-implement-rocksdb-event-listener
  • Source key: roadmap:259:storage:v1.8.0:blobredundancymanager-implement-rocksdb-event-listener

Generated from the consolidated source roadmap. Keep the roadmap and issue in sync when scope changes.

Metadata

Metadata

Labels

area:storageStorage layer (RocksDB, persistence)priority:lowLow prioritystatus:openStatus: opentype:featureNew feature or functionality request

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions