Skip to content

Comments

YARN-11933. TestAMSimulator fails due to shared node labels store directory.#8264

Open
slfan1989 wants to merge 2 commits intoapache:trunkfrom
slfan1989:YARN-11933
Open

YARN-11933. TestAMSimulator fails due to shared node labels store directory.#8264
slfan1989 wants to merge 2 commits intoapache:trunkfrom
slfan1989:YARN-11933

Conversation

@slfan1989
Copy link
Contributor

@slfan1989 slfan1989 commented Feb 21, 2026 •
edited
Loading

Description of PR

JIRA:YARN-11933. TestAMSimulator fails due to shared node labels store directory.

Problem

TestAMSimulator fails intermittently when running concurrently or repeatedly due to shared Node Labels Store directory conflicts. Multiple test instances write to the same default directory, causing interference and test failures.

Root Cause

The test does not specify an independent Node Labels Store directory via YarnConfiguration.FS_NODE_LABELS_STORE_ROOT_DIR.
All test instances use the default shared path, leading to conflicts when:

  • Tests run in parallel
  • Multiple test executions overlap
  • Concurrent parameterized test cases execute

Solution

Create isolated temporary directories for each test instance:

Setup phase: Create a unique temporary directory for Node Labels Store and configure it via YarnConfiguration.FS_NODE_LABELS_STORE_ROOT_DIR
Teardown phase: Clean up the temporary directory to prevent resource leaks

How was this patch tested?

./mvnw -pl hadoop-tools/hadoop-sls -am -Dtest=org.apache.hadoop.yarn.sls.appmaster.TestAMSimulator test

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.yarn.sls.appmaster.TestAMSimulator
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.071 s -- in org.apache.hadoop.yarn.sls.appmaster.TestAMSimulator
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

@hadoop-yetus
Copy link

hadoop-yetus commented Feb 21, 2026

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 22s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 26m 13s trunk passed
+1 💚 compile 0m 27s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 compile 0m 28s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 checkstyle 0m 32s trunk passed
+1 💚 mvnsite 0m 29s trunk passed
+1 💚 javadoc 0m 29s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 28s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 0m 42s trunk passed
+1 💚 shadedclient 15m 25s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 13s the patch passed
+1 💚 compile 0m 12s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javac 0m 12s the patch passed
+1 💚 compile 0m 14s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 javac 0m 14s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 11s the patch passed
+1 💚 mvnsite 0m 17s the patch passed
+1 💚 javadoc 0m 15s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 15s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 0m 32s the patch passed
+1 💚 shadedclient 14m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 47s hadoop-sls in the patch passed.
+1 💚 asflicense 0m 21s The patch does not generate ASF License warnings.
65m 22s
Subsystem Report/Notes
Docker ClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8264/1/artifact/out/Dockerfile
GITHUB PR #8264
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 1bc16b6ab5cc 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 9e3bbed
Default Java Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8264/1/testReport/
Max. process+thread count 616 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8264/1/console
versions git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@slfan1989
Copy link
Contributor Author

slfan1989 commented Feb 22, 2026

@cnauroth The unit test issue in TestAMSimulator has been fixed. Could you review this PR when you have a moment? Thanks a lot!

@slfan1989 slfan1989 requested a review from cnauroth February 22, 2026 02:21
@hadoop-yetus
Copy link

hadoop-yetus commented Feb 22, 2026

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 21s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 27m 45s trunk passed
+1 💚 compile 0m 24s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 compile 0m 29s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 checkstyle 0m 29s trunk passed
+1 💚 mvnsite 0m 32s trunk passed
+1 💚 javadoc 0m 30s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 29s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 0m 46s trunk passed
+1 💚 shadedclient 15m 17s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 14s the patch passed
+1 💚 compile 0m 12s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javac 0m 12s the patch passed
+1 💚 compile 0m 13s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 javac 0m 13s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 10s the patch passed
+1 💚 mvnsite 0m 14s the patch passed
+1 💚 javadoc 0m 15s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 14s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 0m 34s the patch passed
+1 💚 shadedclient 14m 51s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 51s hadoop-sls in the patch passed.
+1 💚 asflicense 0m 23s The patch does not generate ASF License warnings.
66m 46s
Subsystem Report/Notes
Docker ClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8264/2/artifact/out/Dockerfile
GITHUB PR #8264
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux d2be6d77717a 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 81ddef3
Default Java Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8264/2/testReport/
Max. process+thread count 611 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8264/2/console
versions git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cnauroth cnauroth Awaiting requested review from cnauroth

Assignees

No one assigned

Labels

TOOLS trunk

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@slfan1989 @hadoop-yetus