Skip to content

Conversation

@kabo87777
Copy link

Fix Non-Deterministic Behavior in InnerUnseqCompactionWithReadPointPerformerTest

Problem

InnerUnseqCompactionWithReadPointPerformerTest.test was failing non-deterministically under NonDex with 100% failure rate (10/10 runs) due to reliance on HashMap/HashSet iteration order in test setup and file generation.

Way to Reproduce

cd iotdb-core/datanode
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex \
  -Dtest=InnerUnseqCompactionWithReadPointPerformerTest#test \
  -DnondexRuns=10 -Drat.skip=true
# Result: 10/10 failures with varying expected/actual values
# Example: expected:<1149> but was:<2949>

Root Cause

Multiple HashMap/HashSet usages caused non-deterministic iteration order:

  1. Test expectations map (chunkPagePointsNumMerged): HashMap iteration affected assertion order
  2. Measurement paths set (fullPath): HashSet iteration determined TsFile write order
  3. Modification maps (toDeleteTimeseriesAndTime): HashMap affected deletion ordering
  4. Device-measurement grouping in CompactionFileGeneratorUtils: HashMap caused non-deterministic device ordering in generated TsFiles

When NonDex shuffled collection order, measurements/devices were written in different sequences, causing expected chunk/page point counts to mismatch actual values.

Solution

InnerUnseqCompactionWithReadPointPerformerTest.java:

  • Changed Set<String> fullPath from HashSet to LinkedHashSet (line 130)
  • Changed toDeleteTimeseriesAndTime maps from HashMap to LinkedHashMap (lines 313, 356)
  • Changed chunkPagePointsNumMerged from HashMap to LinkedHashMap (line 383)

CompactionFileGeneratorUtils.java:

  • Changed deviceMeasurementMap from HashMap to LinkedHashMap in writeChunkToTsFileWithTimeRange() method

Verification

cd iotdb-core/datanode
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex \
  -Dtest=InnerUnseqCompactionWithReadPointPerformerTest#test \
  -DnondexRuns=20 -Drat.skip=true
# Result: 20/20 passes (100% success rate)

Key changed classes:

  • InnerUnseqCompactionWithReadPointPerformerTest (test changes only)
  • CompactionFileGeneratorUtils (test utility changes only)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant