Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timestamps, cyclic timestamp implementation #348

Open
wants to merge 2 commits into
base: staging
Choose a base branch
from

Conversation

emc2
Copy link

@emc2 emc2 commented Mar 23, 2019

Adds a timestamp trait, as well as an implementation of cyclic timestamps using a rock-paper-scissors counter combined with a traditional integer counter. Unlike a naive integer counter, these these counters do not overflow; rather, they tolerate 2^n-2 concurrent successors, after which the ordering breaks down.

This is useful for implementing versioned values, which are in turn used in many lock-free techniques.

use timestamp::Timestamp;

/// A cyclic timestamp value. For a numeric type having 2^n bits, a
/// CyclicTimestamp guarantees that if a timestamp is less than its
Copy link
Member

@Vtec234 Vtec234 Mar 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if a timestamp is less than its next 2^k - 1 successors, where k = n - 2

then.. what?

#[test]
fn test_u8() {
let values: [CyclicTimestamp<u8>; 9] = [
CyclicTimestamp(0x00),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this and other examples, it would be good to give some explanation of why these specific bit patterns are chosen (or that they're simply random) and maybe give them names if they're significant.

fn new() -> Self;

/// Check `self` is earlier than `other`.
fn earlier(&self, other: &Self) -> bool;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to implement PartialOrd for timestamps instead of custom earlier/later methods?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't observe transitivity.

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! This seems like a nice utility, but I'm not convinced there are practical use cases where it would make life easier.

For example, epochs and indices in all our queues are cyclic timestamps, but they are also all tagged with additional information in the upper/lower bits, which might be a bit of a problem for CyclicTimestamp.

Furthermore, arithmetic on timestamps is (in my experience) not all that complicated:

  • Function succ is t.wrapping_add(1).
  • Function earlier is b.wrapping_sub(a) < a.wrapping_sub(b).
  • Function later is b.wrapping_sub(a) > a.wrapping_sub(b).

@emc2, is there a reason why those functions weren't implemented like this? Could it be that I'm missing something here?

@emc2
Copy link
Author

emc2 commented Apr 23, 2019

The most immediate use case for CyclicTimestamp is Lamport's bakery algorithm. More generally, a common technique in lock-free data structures is to utilize "double-wide" compare-and-set instructions to create versioned values. This is done for atomic snapshots, several lock-free data structures, several solutions to the lock-free memory reclamation problem, and Maged Michael's lock-free malloc. In some of these cases, we compare timestamps for order, not just equality. Finally, some distributed protocols use sequence numbers.

In all these cases where we end up comparing for order, there is an inevitable failure that occurs when integer addition wraps, as this will result in order comparisons incorrectly reporting the earlier timestamps to be later than the ones created after the addition wraps.

CyclicTimestamp eliminates this inevitable failure mode, replacing it with a failure mode that occurs only when the number of concurrently-existing timestamp exceeds a (typically very large) value. In many protocols, there is at most one concurrent timestamp value per pending operation; therefore, failure would require a ridiculously huge number of threads to exist (which, in practice, cannot actually happen). Even when this is not the case, CyclicTimestamp replaces a failure mode which is guaranteed to happen eventually with one that happens with a vanishingly small probability.

The "rock-paper-scissors" technique was taken from one of Herlihy's course lectures, and I believe is described in Herlihy and Shavit's book.

@ghost
Copy link

ghost commented Apr 23, 2019

Thanks for the clarification! I see what rock-paper-scissors is doing now.

I was not able to find this technique in "The Art of Multiprocessor Programming" book. So what I'm failing to understand is where this is really used in practice. It's true that many lock-free algorithms use cyclic timestamps, but is the rock-paper-scissors technique used anywhere?

Another thing I'm not understanding is what are the cases where the following technique is failing, while rock-paper-scissors isn't?

fn earlier(&self, other: &Self) -> bool {
    let a = self.0;
    let b = other.0;
    a.wrapping_sub(b) > b.wrapping_sub(a)
}

fn later(&self, other: &Self) -> bool {
    let a = self.0;
    let b = other.0;
    a.wrapping_sub(b) < b.wrapping_sub(a)
}

fn succ(&self) -> Self {
    let a = self.0;
    let b = a.wrapping_add(1);
    Self(b)
}

@emc2
Copy link
Author

emc2 commented Apr 24, 2019

The official name was something like "trinary counters", or something similar, if I recall correctly. I opted for the "rock-paper-scissors" lingo in the documentation, because it's easier to see what's going on. I recall this technique from one of Herlihy's course lectures, I want to think it was alongside Lamport's Bakery algorithm (that would be a logical place for it), but it's been quite some time.

The biggest problem with your examples using wrapping_sub is that it's not clear what the specification should say. The characteristic wrapping case is where 0xffffffff wraps to 0 (on a 32-bit counter). Should 0xffffffff < 0 be true? If you're considering the call to succ which wraps, then yes. But I can also argue for 0xffffffff > 0.

The trinary counter breaks out of this ambiguity by declaring that we only get valid results when comparing so many successors ahead. Thus, we can write a consistent specification for how earlier and later should behave.

@ghost
Copy link

ghost commented Apr 24, 2019

The trinary counter breaks out of this ambiguity by declaring that we only get valid results when comparing so many successors ahead. Thus, we can write a consistent specification for how earlier and later should behave.

Doesn't the same specification also work with wrapping_sub, i.e. any timestamp is less than it's next 2^k - 1 successors?

If we implemented CyclicTimestamp using wrapping_sub, would we break any of its guarantees?

@bors bors bot force-pushed the staging branch 5 times, most recently from cdc4379 to a5859f3 Compare October 16, 2019 13:28
@bors bors bot force-pushed the staging branch 2 times, most recently from ac3e82e to 4b949c2 Compare July 1, 2022 03:19
@bors bors bot force-pushed the staging branch 2 times, most recently from e733110 to 0fbc9e2 Compare July 23, 2022 08:12
@bors bors bot force-pushed the staging branch 2 times, most recently from 08e880a to 7070018 Compare August 24, 2022 12:45
@bors bors bot force-pushed the staging branch 3 times, most recently from 193e140 to 3e02059 Compare September 10, 2022 15:01
@bors bors bot force-pushed the staging branch 4 times, most recently from 70798d8 to 79168d0 Compare September 29, 2022 03:53
@bors bors bot force-pushed the staging branch 7 times, most recently from 366276a to 42ad5f7 Compare November 22, 2022 06:22
@bors bors bot force-pushed the staging branch 5 times, most recently from 1084184 to 20c973e Compare February 28, 2023 16:19
@bors bors bot force-pushed the staging branch 2 times, most recently from 81ff802 to 94e0400 Compare April 9, 2023 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

2 participants