[RFC] Multi scan API #13473

anand1976 · 2025-03-19T02:47:36Z

A multi scan API for users to pass a set of scan ranges and have the table readers determine the optimal strategy for performing the scans. This might include coalescing of IOs across scans, for example. The requested scans should be in increasing key order.

Eventually, I envision the MultiScanIterator to be built on top of a producer-consumer queue like container, with RocksDB (producer) enqueueing keys and values into the container and the application (consumer) dequeueing them. Unlike a traditional producer consumer queue, there is no concurrency here. The results will be buffered in the container, and when the buffer is empty a new batch will be read from the child iterators. This will allow the virtual function call overhead to be amortized over many entries.

facebook-github-bot · 2025-03-19T03:43:12Z

@anand1976 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jaykorean · 2025-03-20T15:58:28Z

include/rocksdb/db.h

+      const ReadOptions& /*options*/, const std::vector<ScanDesc>& scans) {
+    std::unique_ptr<Iterator> iter(NewErrorIterator(Status::NotSupported()));
+    std::unique_ptr<MultiScanIterator> ms_iter(
+        new MultiScanIterator(scans, std::move(iter)));


We are passing in the scans as const &, so I assume that we expect these ScanDescriptors to be available during the time of iteration?

I'm treating it like the ReadOptions. Lifetime is not guaranteed for the life of the iterator, so the iterator should cache them.

nit: wondering ScanOptions can be a better name.

jaykorean · 2025-03-20T16:08:01Z

include/rocksdb/multi_scan_iterator.h

+};
+
+// An iterator that returns results from multiple scan ranges. The ranges are
+// expected to be in increasing sorted order. The application on top of RocksDB


"ranges are expected to be in increasing sorted order".

I guess the ScanDescriptors as input don't need to be in sorted order, but MultiScanIterator will give the result in sorted order for the user? Asking because "foo" > "bar".

If the DB has keys ["bar", "baz", "foo", "quux"]

In your example, the iteration will give us "bar" -> "baz" -> "foo" -> "quux" -> "foo" -> "quux"?

Good catch. I'll fix the mistake.

jaykorean · 2025-03-20T16:12:16Z

include/rocksdb/multi_scan_iterator.h

+    NextScan();
+  }
+
+  void Dequeue(Slice& key, PinnableSlice& value) {


We aren't retrieving from queued items, so I feel there could be a better name than Dequeue(), but cannot think of one. Next(key, value) doesn't sound right either. Wondering what @pdillinger thinks.

I welcome any suggestions here. I also thought of using the c++ generator concept and have nested iterators. But the user needs to check status, and that makes it a bit awkward.

We could consider using exceptions in some specific places, not internally but narrowly in exposing results to public APIs.

I'm skeptical that the virtual call overhead in the public API layers is that high, relative to vcall and other costs internally that are needed to get ordered entries across the various levels. I suspect the proposed future buffering could optimize regular iterators as well as these. More detail? Disagree?

Let's think through the aspect of this attempting to maximize opportunities for overlapping scans to reuse each others work. (Even if data ends up in the block cache, there can still be significant seeking work.) Imagine a concurrent tree data structure keeping track of pending latency-tolerant scan requests. We could have select read threads following a kind of elevator algorithm (always forward though) to serve up pending scan requests in key order, reusing work between overlapping requests. There should be more opportunities for reusing results vs. a multiscan API like this, because late breaking requests can still be (in a sense) coalesced with pending requests when doing work at or near their range. The downside of course is concurrency management, feeding results with minimal blocking, etc. Thoughts?

We could consider using exceptions in some specific places, not internally but narrowly in exposing results to public APIs.

Yeah, this would be a more elegant way. Looks like we're doing this already in some places (internally though) - https://github.com/search?q=repo%3Afacebook%2Frocksdb+%22throw+std%22&type=code. Should be fine to do it in public APIs.

I'm skeptical that the virtual call overhead in the public API layers is that high, relative to vcall and other costs internally that are needed to get ordered entries across the various levels. I suspect the proposed future buffering could optimize regular iterators as well as these. More detail? Disagree?

I'm hoping to eliminate the vcall overhead in the internal iterators as well with the proposed buffering. So eventually we shouldn't be paying the vcall cost of Next()/Valid()/status()/key()/value() for every key. My rough idea is to have a buffer embedded in LevelIterator and DBIter, and shared and refilled in batches by the BlockBasedTableIterator and MergingIterator respectively. Might require some prototyping to find the best way to do it and get an idea of the perf improvement.

facebook-github-bot · 2025-03-25T06:25:52Z

@anand1976 has updated the pull request. You must reimport the pull request before landing.

anand1976 · 2025-03-25T16:56:17Z

@jaykorean @pdillinger I updated the PR with a modified MultiScanIterator that uses the c++ input iterator interface. Let me know if this looks cleaner.

anand1976 · 2025-03-25T17:08:45Z

Let's think through the aspect of this attempting to maximize opportunities for overlapping scans to reuse each others work. (Even if data ends up in the block cache, there can still be significant seeking work.) Imagine a concurrent tree data structure keeping track of pending latency-tolerant scan requests. We could have select read threads following a kind of elevator algorithm (always forward though) to serve up pending scan requests in key order, reusing work between overlapping requests. There should be more opportunities for reusing results vs. a multiscan API like this, because late breaking requests can still be (in a sense) coalesced with pending requests when doing work at or near their range. The downside of course is concurrency management, feeding results with minimal blocking, etc. Thoughts?

That's an interesting idea. The synchronization and co-ordination across scans in different threads would add some overhead, as well as potentially increase scan latency. Whether its a net win depends on the workload I guess. That's what I'd be most worried about.

The optimization I'm looking to enable with the multi-scan is for scans not necessarily overlapping in key range, but in pages. For example, 2 scans may include adjacent blocks that share the same 4KB page (or some multiple that we determine would benefit from coalescing). Also, I think the multiscan API is sort of orthogonal to what you propose, since its just a way to capture the user intent.

jaykorean · 2025-03-25T17:13:29Z

include/rocksdb/db.h

+      const ReadOptions& /*options*/, const std::vector<ScanDesc>& scans) {
+    std::unique_ptr<Iterator> iter(NewErrorIterator(Status::NotSupported()));
+    std::unique_ptr<MultiScanIterator> ms_iter(
+        new MultiScanIterator(scans, std::move(iter)));


nit: wondering ScanOptions can be a better name.

jaykorean · 2025-03-25T17:16:45Z

include/rocksdb/multi_scan_iterator.h

+
+    ScanIterator(const std::vector<ScanDesc>& scans, Iterator* db_iter)
+        : scans_(scans), idx_(0), db_iter_(db_iter) {
+      db_iter_->Seek(*scans_[idx_++].range.start);


do we want to throw InvalidArg if idx_ >= scans_.size()?

jaykorean · 2025-03-25T17:20:41Z

include/rocksdb/multi_scan_iterator.h

+
+namespace ROCKSDB_NAMESPACE {
+
+// Descriptor for a RocksDB scan request. Only forward scans for now.


nit: we can add UNDER CONSTRUCTION or EXPERIMENTAL

jaykorean · 2025-03-25T17:22:42Z

include/rocksdb/multi_scan_iterator.h

+
+      class SingleIterator {
+       public:
+        using self_type = SingleIterator;


Question for my learning. What does this using ... do here?

These are part of the iterator_traits, if I'm not mistaken. They provide a uniform interface to access the iterator, so callers can do things like for (auto iter : container), std::find etc.

anand1976 added the WIP Work in progress label Mar 19, 2025

facebook-github-bot added the CLA Signed label Mar 19, 2025

anand1976 changed the title ~~[Draft] Multi scan API~~ [RFC] Multi scan API Mar 19, 2025

anand1976 requested review from pdillinger and jaykorean March 19, 2025 23:58

jaykorean reviewed Mar 20, 2025

View reviewed changes

anand1976 added 3 commits March 24, 2025 23:15

Draft multi scan API

459c1d0

MultiScanIterator variant using C++ iterators

863e609

Rebase

e16fb8f

anand1976 force-pushed the multi_scan branch from ce0f34e to e16fb8f Compare March 25, 2025 06:25

jaykorean reviewed Mar 25, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] Multi scan API #13473

[RFC] Multi scan API #13473

anand1976 commented Mar 19, 2025 •

edited

Loading

facebook-github-bot commented Mar 19, 2025

jaykorean Mar 20, 2025

anand1976 Mar 20, 2025

jaykorean Mar 25, 2025

jaykorean Mar 20, 2025

anand1976 Mar 20, 2025

jaykorean Mar 20, 2025

anand1976 Mar 20, 2025

pdillinger Mar 21, 2025

anand1976 Mar 21, 2025

facebook-github-bot commented Mar 25, 2025

anand1976 commented Mar 25, 2025

anand1976 commented Mar 25, 2025

jaykorean Mar 25, 2025

jaykorean Mar 25, 2025

jaykorean Mar 25, 2025

jaykorean Mar 25, 2025

anand1976 Mar 25, 2025


		namespace ROCKSDB_NAMESPACE {

		// Descriptor for a RocksDB scan request. Only forward scans for now.

[RFC] Multi scan API #13473

Are you sure you want to change the base?

[RFC] Multi scan API #13473

Conversation

anand1976 commented Mar 19, 2025 • edited Loading

facebook-github-bot commented Mar 19, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

facebook-github-bot commented Mar 25, 2025

anand1976 commented Mar 25, 2025

anand1976 commented Mar 25, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anand1976 commented Mar 19, 2025 •

edited

Loading