Race condition between root span modification and trace sampling

[dgoffredo]

If thread A modifies a trace's root span while another thread B injects one of the root's descendant spans, the root span's SpanData might be concurrently read from (by B) and written to (by A).

I discovered this when thinking about proposed extensions to trace sampling (where spans other than the root span could be considered).

A simple fix would be to replace std::mutex TraceSegment::mutex_ with a std::shared_mutex, and acquire a std::shared_lock on the mutex any time sampling-relevant span data is modified.

Before we commit to that, though, let's revisit the threading guarantees of the library overall. For example, none of nginx, envoy, nor the HTTP example server concurrently access TraceSegment. I hesitate to forbid it outright, but as of now it would make no difference.

Do we want to support multiple threads banging on different spans that are part of the same trace segment? I can imagine fork/join workflows that do this.