Experimenting with Cluster in a box by ryon-jensen · Pull Request #17966 · daos-stack/daos

ryon-jensen · 2026-04-10T01:37:28Z

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

github-actions · 2026-04-10T01:37:45Z

Errors are component not formatted correctly,Ticket number prefix incorrect,PR title is malformatted. See https://daosio.atlassian.net/wiki/spaces/DC/pages/11133911069/Commit+Comments,Unable to load ticket data
https://daosio.atlassian.net/browse/Experimenting

daosbuild3 · 2026-04-10T02:35:34Z

Test stage Unit Test completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/5/testReport/

daosbuild3 · 2026-04-10T14:41:56Z

Test stage Functional Cluster Box Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/6/testReport/

daosbuild3 · 2026-04-10T21:13:51Z

Test stage Unit Test completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/11/testReport/

daosbuild3 · 2026-04-17T03:55:31Z

Test stage Functional Cluster Box Medium MD on SSD - Aggregation completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/14/display/redirect

daosbuild3 · 2026-04-17T03:55:41Z

Test stage Functional Cluster Box Medium MD on SSD - Telemetry completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/14/display/redirect

daosbuild3 · 2026-04-17T04:10:55Z

Test stage Functional Cluster Box Medium MD on SSD - Aggregation completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/14/testReport/

daosbuild3 · 2026-04-17T05:42:47Z

Test stage Functional Cluster Box Medium MD on SSD - Aggregation completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/13/execution/node/735/log

daosbuild3 · 2026-04-17T11:45:19Z

Test stage Functional Cluster Box Medium MD on SSD - SnapshotAggregation completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/15/display/redirect

daosbuild3 · 2026-04-17T11:45:20Z

Test stage Functional Cluster Box Medium MD on SSD - TestWithTelemetryNvme completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/15/display/redirect

daosbuild3 · 2026-04-17T12:01:39Z

Test stage Functional Cluster Box Medium MD on SSD - TestWithTelemetryIOBasic completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/15/execution/node/781/log

daosbuild3 · 2026-04-17T12:07:32Z

Test stage Functional Cluster Box Medium MD on SSD - DaosCoreTestRebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/15/execution/node/792/log

daosbuild3 · 2026-04-17T12:11:31Z

Test stage Functional Cluster Box Medium MD on SSD - Rest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/15/execution/node/871/log

daosbuild3 · 2026-04-17T12:18:07Z

Test stage Functional Cluster Box Medium MD on SSD - SnapshotAggregation completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/16/testReport/

daosbuild3 · 2026-04-17T12:27:44Z

Test stage Functional Cluster Box Medium MD on SSD - TestWithTelemetryIOBasic completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/16/execution/node/848/log

daosbuild3 · 2026-04-17T12:35:00Z

Test stage Functional Cluster Box Medium MD on SSD - DaosCoreTestRebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/16/execution/node/859/log

daosbuild3 · 2026-04-17T12:38:04Z

Test stage Functional Cluster Box Medium MD on SSD - Rest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/16/execution/node/939/log

daosbuild3 · 2026-04-17T13:14:21Z

Test stage Functional Cluster Box Medium MD on SSD - SnapshotAggregation completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/17/testReport/

daosbuild3 · 2026-06-06T14:19:25Z

Test stage Functional Cluster Box FTest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/176/display/redirect

daosbuild3 · 2026-06-08T19:57:51Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/177/testReport/

daosbuild3 · 2026-06-09T20:26:50Z

Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17966/179/testReport/

daosbuild3 · 2026-06-10T02:21:58Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/180/display/redirect

daosbuild3 · 2026-06-10T02:22:10Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/180/display/redirect

daosbuild3 · 2026-06-10T08:48:36Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/180/execution/node/2415/log

daosbuild3 · 2026-06-10T12:33:58Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/181/display/redirect

daosbuild3 · 2026-06-10T12:34:10Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/181/display/redirect

daosbuild3 · 2026-06-12T22:07:30Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/183/display/redirect

daosbuild3 · 2026-06-12T22:07:39Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/183/display/redirect

daosbuild3 · 2026-06-12T22:07:56Z

Test stage Functional Cluster Box FTest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/183/display/redirect

daosbuild3 · 2026-06-12T22:08:04Z

Test stage Functional Cluster Box FTest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/183/display/redirect

daosbuild3 · 2026-06-12T23:23:10Z

Test stage Functional Cluster Box FTest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/184/execution/node/735/log

daosbuild3 · 2026-06-13T01:00:26Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/184/execution/node/746/log

daosbuild3 · 2026-06-16T05:49:40Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/186/execution/node/774/log

daosbuild3 · 2026-06-16T07:25:39Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/192/execution/node/772/log

daosbuild3 · 2026-06-16T16:24:43Z

Test stage Functional Cluster Box daos_test Other completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17966/190/display/redirect

- The new "Cluster in a Box" stage is intended to be able to support testing that currently requires hardware resources to run. A Cluster in a Box is a KVM host that hosts VMs that have emulated NVMe and passthrough IB devices, so they appear more like HW systems. - Changed several "PR" tests to run on Cluster Box stages. Some of the tests' configurations were adjusted to be able to run on a VM with less resources. Some tests can take longer to run so timeouts were increased. - As I was testing the CI changes, I noticed that more than "PR" tests were being run. This was because of the gen_commit_pragmas.py functionality, which will include tests if the test file was changed. Because test tags were being changed, the whole file was being run, including non-PR tests. To prevent that, gen_commit_pragmas.py was modified only automatically include tests if actual code changed ... not just comments. - In the Jenkinsfile, added pragma to set build description Signed-off-by: Ryon Jensen <ryon.jensen@hpe.com> Build-description: Testing from git push Cancel-prev-build: false Skip-unit-test: true Skip-unit-test-memcheck: true Skip-nlt: true Skip-fault-injection-test: true Skip-test-el-9-rpms: true Skip-test-leap-15-rpms: true Skip-func-test-el8: true Skip-func-test-el9: true Skip-func-test-leap15: true Skip-func-test-hw-medium: true Skip-func-test-hw-medium-md-on-ssd: true Skip-func-test-hw-medium-vmd: true Skip-func-test-hw-medium-verbs-provider: true Skip-func-test-hw-medium-verbs-provider-md-on-ssd: true Skip-func-test-hw-large: true Skip-func-test-hw-large-md-on-ssd: true Skip-python-bandit-check: true

daosbuild3 · 2026-06-16T20:38:32Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/194/execution/node/783/log

daosbuild3 · 2026-06-16T21:15:05Z

Test stage Functional Cluster Box daos_test Other completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/198/execution/node/706/log

daosbuild3 · 2026-06-16T22:20:29Z

Test stage Functional Cluster Box daos_test Other completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/201/execution/node/706/log

daosbuild3 · 2026-06-16T22:34:39Z

Test stage Functional Cluster Box FTest completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/199/execution/node/780/log

daosbuild3 · 2026-06-16T22:58:45Z

Test stage Functional Cluster Box daos_test Other completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/195/execution/node/706/log

daosbuild3 · 2026-06-16T23:36:04Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/198/execution/node/863/log

daosbuild3 · 2026-06-17T00:42:06Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/201/execution/node/863/log

daosbuild3 · 2026-06-17T00:59:13Z

Test stage Functional Cluster Box daos_test Rebuild completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17966/195/execution/node/863/log

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch 4 times, most recently from 8465579 to b73c868 Compare April 10, 2026 01:58

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch 4 times, most recently from 8e1af9e to b380586 Compare April 10, 2026 20:10

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch from b380586 to 87609f3 Compare April 17, 2026 01:16

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch from 87609f3 to c212f93 Compare April 17, 2026 11:33

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch 2 times, most recently from 98035d4 to 70582b3 Compare April 17, 2026 14:26

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch from 2c0a469 to 60f29b8 Compare June 8, 2026 17:30

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch from 60f29b8 to ecac279 Compare June 9, 2026 23:07

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch from 829ffcc to 3a5b0d3 Compare June 16, 2026 18:14

ryon-jensen force-pushed the ryon-jensen/speed_of_ci branch from 3a5b0d3 to cdf9ce7 Compare June 16, 2026 18:18

Conversation

ryon-jensen commented Apr 10, 2026

Steps for the author:

After all prior steps are complete:

Uh oh!

github-actions Bot commented Apr 10, 2026

Uh oh!

daosbuild3 commented Apr 10, 2026

Uh oh!

daosbuild3 commented Apr 10, 2026

Uh oh!

daosbuild3 commented Apr 10, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Apr 17, 2026

Uh oh!

daosbuild3 commented Jun 6, 2026

Uh oh!

daosbuild3 commented Jun 8, 2026

Uh oh!

daosbuild3 commented Jun 9, 2026

Uh oh!

daosbuild3 commented Jun 10, 2026

Uh oh!

daosbuild3 commented Jun 10, 2026

Uh oh!

daosbuild3 commented Jun 10, 2026

Uh oh!

daosbuild3 commented Jun 10, 2026

Uh oh!

daosbuild3 commented Jun 10, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 12, 2026

Uh oh!

daosbuild3 commented Jun 13, 2026

Uh oh!

daosbuild3 commented Jun 16, 2026

Uh oh!

daosbuild3 commented Jun 16, 2026

Uh oh!

daosbuild3 commented Jun 16, 2026

Uh oh!

daosbuild3 commented Jun 16, 2026

Uh oh!

daosbuild3 commented Jun 16, 2026

Uh oh!

daosbuild3 commented Jun 16, 2026