Add timestamps, cyclic timestamp implementation

[emc2]

Adds a timestamp trait, as well as an implementation of cyclic timestamps using a rock-paper-scissors counter combined with a traditional integer counter. Unlike a naive integer counter, these these counters do not overflow; rather, they tolerate 2^n-2 concurrent successors, after which the ordering breaks down.

This is useful for implementing versioned values, which are in turn used in many lock-free techniques.

[Vtec234]

if a timestamp is less than its next 2^k - 1 successors, where k = n - 2

then.. what?

[Vtec234]

For this and other examples, it would be good to give some explanation of why these specific bit patterns are chosen (or that they're simply random) and maybe give them names if they're significant.

[Vtec234]

Does it make sense to implement PartialOrd for timestamps instead of custom earlier/later methods?

[emc2]

These don't observe transitivity.

[ghost]

Thanks for the PR! This seems like a nice utility, but I'm not convinced there are practical use cases where it would make life easier.

For example, epochs and indices in all our queues are cyclic timestamps, but they are also all tagged with additional information in the upper/lower bits, which might be a bit of a problem for CyclicTimestamp.

Furthermore, arithmetic on timestamps is (in my experience) not all that complicated:

• Function succ is t.wrapping_add(1).
• Function earlier is b.wrapping_sub(a) < a.wrapping_sub(b).
• Function later is b.wrapping_sub(a) > a.wrapping_sub(b).

@emc2, is there a reason why those functions weren't implemented like this? Could it be that I'm missing something here?

[emc2]

The most immediate use case for CyclicTimestamp is Lamport's bakery algorithm. More generally, a common technique in lock-free data structures is to utilize "double-wide" compare-and-set instructions to create versioned values. This is done for atomic snapshots, several lock-free data structures, several solutions to the lock-free memory reclamation problem, and Maged Michael's lock-free malloc. In some of these cases, we compare timestamps for order, not just equality. Finally, some distributed protocols use sequence numbers.

In all these cases where we end up comparing for order, there is an inevitable failure that occurs when integer addition wraps, as this will result in order comparisons incorrectly reporting the earlier timestamps to be later than the ones created after the addition wraps.

CyclicTimestamp eliminates this inevitable failure mode, replacing it with a failure mode that occurs only when the number of concurrently-existing timestamp exceeds a (typically very large) value. In many protocols, there is at most one concurrent timestamp value per pending operation; therefore, failure would require a ridiculously huge number of threads to exist (which, in practice, cannot actually happen). Even when this is not the case, CyclicTimestamp replaces a failure mode which is guaranteed to happen eventually with one that happens with a vanishingly small probability.

The "rock-paper-scissors" technique was taken from one of Herlihy's course lectures, and I believe is described in Herlihy and Shavit's book.

[ghost]

Thanks for the clarification! I see what rock-paper-scissors is doing now.

I was not able to find this technique in "The Art of Multiprocessor Programming" book. So what I'm failing to understand is where this is really used in practice. It's true that many lock-free algorithms use cyclic timestamps, but is the rock-paper-scissors technique used anywhere?

Another thing I'm not understanding is what are the cases where the following technique is failing, while rock-paper-scissors isn't?

fn earlier(&self, other: &Self) -> bool {
    let a = self.0;
    let b = other.0;
    a.wrapping_sub(b) > b.wrapping_sub(a)
}

fn later(&self, other: &Self) -> bool {
    let a = self.0;
    let b = other.0;
    a.wrapping_sub(b) < b.wrapping_sub(a)
}

fn succ(&self) -> Self {
    let a = self.0;
    let b = a.wrapping_add(1);
    Self(b)
}

[emc2]

The official name was something like "trinary counters", or something similar, if I recall correctly. I opted for the "rock-paper-scissors" lingo in the documentation, because it's easier to see what's going on. I recall this technique from one of Herlihy's course lectures, I want to think it was alongside Lamport's Bakery algorithm (that would be a logical place for it), but it's been quite some time.

The biggest problem with your examples using wrapping_sub is that it's not clear what the specification should say. The characteristic wrapping case is where 0xffffffff wraps to 0 (on a 32-bit counter). Should 0xffffffff < 0 be true? If you're considering the call to succ which wraps, then yes. But I can also argue for 0xffffffff > 0.

The trinary counter breaks out of this ambiguity by declaring that we only get valid results when comparing so many successors ahead. Thus, we can write a consistent specification for how earlier and later should behave.

[ghost]

The trinary counter breaks out of this ambiguity by declaring that we only get valid results when comparing so many successors ahead. Thus, we can write a consistent specification for how earlier and later should behave.

Doesn't the same specification also work with wrapping_sub, i.e. any timestamp is less than it's next 2^k - 1 successors?

If we implemented CyclicTimestamp using wrapping_sub, would we break any of its guarantees?