[BUG] Thread-level `current stream`

[nan2088]

It appears that, by design, the current stream is intended to be thread-local, consistent with PyTorch's design:

CV-CUDA/python/mod_cvcuda/nvcv/Stream.cpp

Line 524 in d45fa0a

stream.def("__enter__", &Stream::activate, "Activate the CUDA stream as the current stream for this thread.")

However, the implementation uses a global singleton pattern:

CV-CUDA/python/mod_cvcuda/nvcv/Stream.cpp

Line 352 in d45fa0a

StreamStack::Instance().push(*this);

CV-CUDA/python/mod_cvcuda/nvcv/StreamStack.cpp

Line 51 in d45fa0a

static StreamStack stack;

This will cause the current stream to return an incorrect stream in multi-threaded scenarios.

[kgnandanwar]

Bug Confirmation: Thread-Level Current Stream Issue #274

Summary

BUG CONFIRMED ✅ - The current stream in CV-CUDA is NOT thread-local as documented. It uses a global singleton pattern that causes stream interference in multi-threaded scenarios.

Issue Reference: #274

---

The Problem

The documentation claims streams are thread-local:

// python/mod_cvcuda/nvcv/Stream.cpp:524
stream.def("__enter__", &Stream::activate, 
    "Activate the CUDA stream as the current stream for this thread.")

But the implementation uses a global singleton:

// python/mod_cvcuda/nvcv/StreamStack.cpp:51
StreamStack &StreamStack::Instance()
{
    static StreamStack stack;  // ← Global, not thread-local!
    return stack;
}

---

How was it tested

Test Setup

Created a multi-threaded test where:

• Thread 1 activates Stream A (ID: 1)
• Thread 2 activates Stream B (ID: 2)
• Each thread checks Stream.current multiple times

Expected: Each thread sees its own stream
Actual: Both threads see the last activated stream (shared stack)

Test Results

[Thread 1] Created Stream A with ID: 1
[Thread 2] Created Stream B with ID: 2
[Thread 1] Activated Stream A (ID: 1)
[Thread 2] Activated Stream B (ID: 2)

[Thread 1] Iteration 1: Current stream ID = 2  ← Expected 1, got 2!
[Thread 1] ⚠️  WARNING: Expected 1, got 2

[Thread 2] Iteration 1: Current stream ID = 2  ← Correct

[Thread 1] Iteration 2: Current stream ID = 2  ← Still wrong!
[Thread 1] ⚠️  WARNING: Expected 1, got 2

Fatal assertion error  ← Crash during cleanup
Aborted (core dumped)

Analysis

Thread	Activated	Expected	Saw	Result
Thread 1	Stream 1	ID: 1	ID: 2	❌ WRONG
Thread 2	Stream 2	ID: 2	ID: 2	✅ Correct

Thread 1 consistently sees Thread 2's stream because they share the same global StreamStack.

---

Root Cause

In C++, static creates process-wide singletons, not thread-local storage. All threads share the same stack, causing:

1. Thread interference (wrong stream returned)
2. Race conditions (timing-dependent behavior)
3. Crashes (assertion failures during cleanup)

Test Environment: WSL2 Ubuntu, CV-CUDA v0.16.0, Python 3.10.12

---

Reproduce

python3 test_stream_thread_safety.py

The test demonstrates the bug clearly and crashes consistently.
Here is the file test_stream_thread_safety.py

"""
Test if CV-CUDA streams are thread-local or global.
Reproduces bug #274: https://github.com/CVCUDA/CV-CUDA/issues/274

Expected: Each thread sees its own stream
Actual: All threads share the same global StreamStack
"""

import threading
import time
import sys

try:
    import cvcuda
    Stream = cvcuda.Stream
except ImportError as e:
    print(f"Error: {e}\nRun this in WSL2 with CV-CUDA installed")
    sys.exit(1)


class Results:
    def __init__(self):
        self.thread1_streams = []
        self.thread2_streams = []
        self.lock = threading.Lock()


def thread1_worker(results, barrier):
    stream_a = Stream()
    stream_a_id = stream_a.id
    print(f"[Thread 1] Created Stream A with ID: {stream_a_id}")
    
    barrier.wait()
    
    with stream_a:
        print(f"[Thread 1] Activated Stream A (ID: {stream_a_id})")
        
        for i in range(3):
            time.sleep(0.05)
            current_id = Stream.current.id
            
            with results.lock:
                results.thread1_streams.append(current_id)
            
            print(f"[Thread 1] Iteration {i+1}: Current stream ID = {current_id}")
            
            if current_id != stream_a_id:
                print(f"[Thread 1] ⚠️  WARNING: Expected {stream_a_id}, got {current_id}")


def thread2_worker(results, barrier):
    stream_b = Stream()
    stream_b_id = stream_b.id
    print(f"[Thread 2] Created Stream B with ID: {stream_b_id}")
    
    barrier.wait()
    time.sleep(0.025)  # Let Thread 1 activate first
    
    with stream_b:
        print(f"[Thread 2] Activated Stream B (ID: {stream_b_id})")
        
        for i in range(3):
            time.sleep(0.05)
            current_id = Stream.current.id
            
            with results.lock:
                results.thread2_streams.append(current_id)
            
            print(f"[Thread 2] Iteration {i+1}: Current stream ID = {current_id}")
            
            if current_id != stream_b_id:
                print(f"[Thread 2] ⚠️  WARNING: Expected {stream_b_id}, got {current_id}")


def main():
    print("\nCV-CUDA Thread-Local Stream Test")
    print("=" * 80)
    print(f"Default stream ID: {Stream.current.id}\n")
    
    results = Results()
    barrier = threading.Barrier(2)
    
    print("Starting threads...\n")
    t1 = threading.Thread(target=thread1_worker, args=(results, barrier))
    t2 = threading.Thread(target=thread2_worker, args=(results, barrier))
    
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    
    print("\n" + "=" * 80)
    print("RESULTS")
    print("=" * 80)
    print(f"Thread 1 saw: {results.thread1_streams}")
    print(f"Thread 2 saw: {results.thread2_streams}")
    
    # Check if each thread consistently saw its own stream
    thread1_ok = len(set(results.thread1_streams)) == 1
    thread2_ok = len(set(results.thread2_streams)) == 1
    different = results.thread1_streams[0] != results.thread2_streams[0]
    
    if thread1_ok and thread2_ok and different:
        print("\n✓ PASS: Streams are thread-local")
        sys.exit(0)
    else:
        print("\n✗ FAIL: BUG CONFIRMED - Streams are global, not thread-local")
        print("  Code: StreamStack.cpp:51 uses 'static' instead of 'thread_local'")
        sys.exit(1)


if __name__ == "__main__":
    main()