[Bug]: Concurrent `create_collection` + `delete_collection` on the same collection name causes lost update (both return success)

[yihui504]

What happened?

When two concurrent operations — create_collection(name) and delete_collection(name) — race against the same collection name in Chroma's HTTP server mode, in some cases both operations return success. This is a lost update: the create_collection client receives a success response believing the collection exists, but delete_collection has already removed it. The system ends up in an inconsistent state where the create client holds a reference to a collection that no longer exists.

Tested with 10 cycles of concurrent create + delete on the same name: 2 out of 10 cycles exhibited both operations succeeding.

What I expected to happen:

Concurrent create_collection / delete_collection on the same collection name should be serialized — one operation should succeed and the other should receive a deterministic error (e.g. "already exists" or "does not exist"), but never both succeed. The documentation states that collection names must be unique inside a Chroma database, which implies create and delete on the same name must not produce conflicting success results.

Versions

- Chroma server: chromadb/chroma:1.5.9 (Docker image)
- Chroma Python client: 1.5.9 (chromadb.HttpClient)
- Python: 3.12
- OS: Windows 11 (Docker Desktop), server running in Linux container
- Deployment: single-node Docker, IS_PERSISTENT=TRUE

Relevant log output

Minimal reproduction script:


import threading
import chromadb
from collections import Counter

client = chromadb.HttpClient(host="localhost", port=8000)
COLL = "race_test"
results = Counter()

def try_create():
    try:
        client.create_collection(COLL)
        results["create_ok"] += 1
    except Exception as e:
        results[f"create_{type(e).__name__}"] += 1

def try_delete():
    try:
        client.delete_collection(COLL)
        results["delete_ok"] += 1
    except Exception as e:
        results[f"delete_{type(e).__name__}"] += 1

race_issues = 0
for cycle in range(10):
    try:
        client.delete_collection(COLL)
    except Exception:
        pass
    results.clear()
    t1 = threading.Thread(target=try_create)
    t2 = threading.Thread(target=try_delete)
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    if results.get("create_ok", 0) > 0 and results.get("delete_ok", 0) > 0:
        race_issues += 1
        print(f"  cycle {cycle}: BOTH succeeded (lost update): {dict(results)}")
    else:
        print(f"  cycle {cycle}: {dict(results)}")

print(f"\n{race_issues}/10 cycles had concurrent create + delete both succeed")


Example output (chromadb/chroma:1.5.9 Docker):


  cycle 0: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 1: BOTH succeeded (lost update): {'create_ok': 1, 'delete_ok': 1}
  cycle 2: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 3: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 4: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 5: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 6: BOTH succeeded (lost update): {'create_ok': 1, 'delete_ok': 1}
  cycle 7: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 8: {'delete_NotFoundError': 1, 'create_ok': 1}
  cycle 9: {'delete_NotFoundError': 1, 'create_ok': 1}

2/10 cycles had concurrent create + delete both succeed


In the lost-update cycles, `create_collection` returned a valid Collection object and `delete_collection` returned success (None) in the same cycle. The collection's final state is undefined — the create client believes it exists, but it has been deleted.

Related: #4661 reported a similar race condition for `get_or_create_collection` in the sqlite embedded backend (chroma 1.0.10). This issue confirms the race persists in HTTP server mode (`chroma run` / Docker) on 1.5.9 for pure `create_collection` + `delete_collection`.