How will we test this?

[alex-dukhno]

Hi guys,

I don't know where is better to start this topic, so let it be here.

I have read Atomic API RFC, and the followed discussion; however, I am not sure that I fully understand everything due to lack of theoretical knowledge and practical experience in writing highly concurrent DS and algorithms, and I want to participate in the development of this crate.

But I have some experience in writing tests with production code. (I don't want to argue what is better test first or test last.) But, I have some thoughts how to use tests in development of crossbeam:

• add an optional section "How we will test this" (or any other name you would like) to the RFCs that propose new or change existed functionality. That section would contain a list of stress test scenarios, maybe with a brief description of the tests and what kind of load should be (and people like me, would be able to build a context around the problem, and maybe bring some new ideas or suggestions)
• include unit tests for single threaded scenarios to implementations of the RFCs. I know this is very personal, but I see some advantages of this. First of all, it is very useful for library developers to taste its API with the tests (It can bring whole new ideas what API should be). Second, it is very convenient to library users to see how they can use API (maybe I am the only one who looks for examples of API usage in tests). And, maybe, the most important that reviewers can spot that some cases weren't covered and needed more attention.

Hi, welcome and thank you for your interest in development of this project! :)

The RFC you're referring to is indeed intended for readers well acquainted with the existing codebase. The text is already fairly long as it is, and sometimes it's a bit difficult to make it digestable to a wider audience. One of the things I'd like to do very soon is collect a bunch of articles/books/papers on lock-free programming so that newcomers can more easily get up to speed.

The RFC has several examples (pushing an element onto a stack, destructing a stack, traversing a linked list), but they're fairly hand-wavy and, like the rest of the text, aimed at already pretty knowledgeable folk.

There is already an open PR implementing the proposal. There are no unit tests, but the code is covered through doctests. The Atomic/Owned/Ptr API is fairly straightforward so it doesn't need much more.

At the moment we're focused on building something very similar to this module, but using a thorough consensus-based approach (RFCs). The atomics API is the first step, and the epoch GC is the next one.

The linked module already contains some tests, so you may take a look at it to get an idea of what we will need. Also take a look at the test script that runs on Travis.

Tests in Crossbeam are a very important topic and we'll need to give it serious thought. Concurrency bugs can be very tricky to provoke and find. Honestly, at the moment I'm not 100% sure how exactly we should organize tests so this is definitely an area where help is very welcome. :) An attempt at testing can be found in Coco, and something like it may be a good start.

[jeehoonkang]

I feel the test methodology itself deserves a proper RFC. AFAICT testing concurrent program is really hard, and someday we need to set a high bar on the implementation. I don't think we need it now, but as the project will become more and more mature it is a must.

[alex-dukhno]

BTW, as @stjepang mentioned a long time ago, that he wants a collection of papers/articles, I'd like to contribute to it with these four videos; that opened my eyes on memory management in concurrent data structures
Erez Petrank — Memory management for concurrent data structures (Part 1)
Erez Petrank — Memory management for concurrent data structures (Part 2)
Erez Petrank — Memory management for concurrent data structures (Part 3)
Erez Petrank — Memory management for concurrent data structures (Part 4)

Thanks, Alex - these are fantastic videos.

I've started a wiki page containing a list of learning resources. Feel free to add more links. :)

https://github.com/crossbeam-rs/rfcs/wiki