Skip to content

Commit 5382581

Browse files
committed
Fix for flaky test test_chunks_the_same
This would fail sometimes on Windows / Mac because the clocks used there are less precise and the two [1000, 1001] segments would collide on exactly the same key with LMDB. I just change the test so their content hashes would be different. The test was written to cover an issue with identical index ranges, so it still meets its original purpose. This just changes the tests so that their content hashes differ. Theoretically, this could be a real problem that comes up, and we ought to somehow retry when there is a key clash. But that seems very improbable outside of unit testing, so we leave it as is for now.
1 parent 3e40f14 commit 5382581

File tree

1 file changed

+14
-8
lines changed

1 file changed

+14
-8
lines changed

python/tests/unit/arcticdb/version_store/test_parallel.py

+14-8
Original file line numberDiff line numberDiff line change
@@ -1531,17 +1531,17 @@ def test_chunks_match_at_ends(lmdb_storage, lib_name):
15311531
assert result["a"][-1] == 6
15321532

15331533

1534-
def test_chunks_the_same(lmdb_storage, lib_name):
1534+
@pytest.mark.parametrize("n_runs", range(10))
1535+
def test_chunks_the_same(lmdb_storage, lib_name, n_runs):
15351536
"""Given - we stage chunks with indexes:
15361537
1537-
b:test:0:0xc7ad4135da54cd6e@1739968588832977666[1000,2001]
1538-
b:test:0:0x68d8759aba38bcf0@1739968588832775570[1000,1001]
1539-
b:test:0:0x68d8759aba38bcf0@1739968588832621000[1000,1001]
1538+
b:test:0:h1@1739968588832977666[1000,2001]
1539+
b:test:0:h2@1739968588832775570[1000,1001]
1540+
b:test:0:h3@1739968588832621000[1000,1001]
15401541
15411542
When - We finalize the staged segments
15421543
1543-
Then - We should succeed even though the segments seem to be identical, since they are just covering a duplicated
1544-
index value
1544+
Then - We should succeed even though the segments are covering a duplicated index value
15451545
"""
15461546
lib: Library = lmdb_storage.create_arctic().create_library(
15471547
lib_name,
@@ -1556,7 +1556,7 @@ def test_chunks_the_same(lmdb_storage, lib_name):
15561556
pd.Timestamp(2000),
15571557
]
15581558

1559-
data = pd.DataFrame({"a": len(idx)}, index=idx)
1559+
data = pd.DataFrame({"a": np.arange(len(idx))}, index=idx)
15601560
lib.write("test", data, staged=True)
15611561

15621562
lt = lib._nvs.library_tool()
@@ -1568,7 +1568,13 @@ def test_chunks_the_same(lmdb_storage, lib_name):
15681568
lib.finalize_staged_data("test")
15691569

15701570
df = lib.read("test").data
1571-
assert_frame_equal(df, data)
1571+
try:
1572+
assert_frame_equal(df, data)
1573+
except AssertionError:
1574+
# Where the identical segments end up is arbitrary
1575+
other_order = pd.DataFrame({"a": [2, 3, 0, 1, 4, 5]}, index=idx)
1576+
assert_frame_equal(df, other_order)
1577+
15721578
assert df.index.is_monotonic_increasing
15731579

15741580

0 commit comments

Comments
 (0)