Make Test failed

SRSLTE make test failed in the following phases!

Can anyone help?
git cloned from original source and then checkout with version 19_12
I'm using USRP B210 and the PC is i7 6700HQ

    Start 581: scell_search_test

581/587 Test #581: scell_search_test ................................ Passed 14.65 sec
Start 582: usim_test
582/587 Test #582: usim_test ........................................ Passed 0.02 sec
Start 583: rrc_reconfig_test
583/587 Test #583: rrc_reconfig_test ................................ Passed 0.02 sec
Start 584: nas_test
584/587 Test #584: nas_test .........................................***Exception: Child aborted 0.25 sec
Start 585: tft_test
585/587 Test #585: tft_test ......................................... Passed 0.04 sec
Start 586: rrc_mobility_test
586/587 Test #586: rrc_mobility_test ................................ Passed 0.05 sec
Start 587: erab_setup_test
587/587 Test #587: erab_setup_test .................................. Passed 0.05 sec

98% tests passed, 9 tests failed out of 587

Total Test time (real) = 252.43 sec

The following tests FAILED:
357 - phy_dl_test-p6-t2-q-m27 (Failed)
362 - phy_dl_test-p6-t3-q-m27 (Failed)
367 - phy_dl_test-p6-t4-q-m27 (Failed)
422 - phy_dl_test-p25-t3-m28 (Failed)
427 - phy_dl_test-p25-t4-m28 (Failed)
437 - phy_dl_test-p25-t2-q-m27 (Failed)
442 - phy_dl_test-p25-t3-q-m27 (Failed)
447 - phy_dl_test-p25-t4-q-m27 (Failed)
584 - nas_test (Child aborted)
Errors while running CTest
Makefile:140: recipe for target 'test' failed
make: *** [test] Error 8

Thanks
Best regards

[TheDuchy]

Same issue here with i7 3612qm (supports AVX)

[ZeroChaos-]

pretty similar here with the 20.04.2 release

92% tests passed, 61 tests failed out of 739

Total Test time (real) = 131.66 sec

The following tests FAILED:
	396 - algebra_2x2_zf_solver_test (Failed)
	472 - phy_dl_test-p6-t3-m0 (Failed)
	473 - phy_dl_test-p6-t3-m7 (Failed)
	474 - phy_dl_test-p6-t3-m14 (Failed)
	475 - phy_dl_test-p6-t3-m21 (Failed)
	476 - phy_dl_test-p6-t3-m28 (Failed)
	492 - phy_dl_test-p6-t3-q-m0 (Failed)
	493 - phy_dl_test-p6-t3-q-m7 (Failed)
	494 - phy_dl_test-p6-t3-q-m14 (Failed)
	495 - phy_dl_test-p6-t3-q-m21 (Failed)
	496 - phy_dl_test-p6-t3-q-m27 (Failed)
	512 - phy_dl_test-p15-t3-m0 (Failed)
	513 - phy_dl_test-p15-t3-m7 (Failed)
	514 - phy_dl_test-p15-t3-m14 (Failed)
	515 - phy_dl_test-p15-t3-m21 (Failed)
	516 - phy_dl_test-p15-t3-m28 (Failed)
	532 - phy_dl_test-p15-t3-q-m0 (Failed)
	533 - phy_dl_test-p15-t3-q-m7 (Failed)
	534 - phy_dl_test-p15-t3-q-m14 (Failed)
	535 - phy_dl_test-p15-t3-q-m21 (Failed)
	536 - phy_dl_test-p15-t3-q-m26 (Failed)
	552 - phy_dl_test-p25-t3-m0 (Failed)
	553 - phy_dl_test-p25-t3-m7 (Failed)
	554 - phy_dl_test-p25-t3-m14 (Failed)
	555 - phy_dl_test-p25-t3-m21 (Failed)
	556 - phy_dl_test-p25-t3-m28 (Failed)
	572 - phy_dl_test-p25-t3-q-m0 (Failed)
	573 - phy_dl_test-p25-t3-q-m7 (Failed)
	574 - phy_dl_test-p25-t3-q-m14 (Failed)
	575 - phy_dl_test-p25-t3-q-m21 (Failed)
	576 - phy_dl_test-p25-t3-q-m27 (Failed)
	592 - phy_dl_test-p50-t3-m0 (Failed)
	593 - phy_dl_test-p50-t3-m7 (Failed)
	594 - phy_dl_test-p50-t3-m14 (Failed)
	595 - phy_dl_test-p50-t3-m21 (Failed)
	596 - phy_dl_test-p50-t3-m28 (Failed)
	612 - phy_dl_test-p50-t3-q-m0 (Failed)
	613 - phy_dl_test-p50-t3-q-m7 (Failed)
	614 - phy_dl_test-p50-t3-q-m14 (Failed)
	615 - phy_dl_test-p50-t3-q-m21 (Failed)
	616 - phy_dl_test-p50-t3-q-m27 (Failed)
	632 - phy_dl_test-p75-t3-m0 (Failed)
	633 - phy_dl_test-p75-t3-m7 (Failed)
	634 - phy_dl_test-p75-t3-m14 (Failed)
	635 - phy_dl_test-p75-t3-m21 (Failed)
	636 - phy_dl_test-p75-t3-m28 (Failed)
	652 - phy_dl_test-p75-t3-q-m0 (Failed)
	653 - phy_dl_test-p75-t3-q-m7 (Failed)
	654 - phy_dl_test-p75-t3-q-m14 (Failed)
	655 - phy_dl_test-p75-t3-q-m21 (Failed)
	656 - phy_dl_test-p75-t3-q-m27 (Failed)
	672 - phy_dl_test-p100-t3-m0 (Failed)
	673 - phy_dl_test-p100-t3-m7 (Failed)
	674 - phy_dl_test-p100-t3-m14 (Failed)
	675 - phy_dl_test-p100-t3-m21 (Failed)
	676 - phy_dl_test-p100-t3-m28 (Failed)
	692 - phy_dl_test-p100-t3-q-m0 (Failed)
	693 - phy_dl_test-p100-t3-q-m7 (Failed)
	694 - phy_dl_test-p100-t3-q-m14 (Failed)
	695 - phy_dl_test-p100-t3-q-m21 (Failed)
	696 - phy_dl_test-p100-t3-q-m27 (Failed)

[andrepuschmann]

Hey guys,
thanks for the hint. The above mentioned tests should all be fixed in the next release. Since they are not critical (only local to the test code itself), they won't be part of any hot-fix release for 19.12 or 20.04. Before the official 20.10 release, we will have a pre-release. It would be great if you could double check those issues are indeed gone for you. We'll keep you posted. Thanks

[andrepuschmann]

We pushed a release candidate for 20.10 to the next branch of this repo. We would appreciate if you could run your tests again and report here. Besides the scell_search_test which has some randomness involved that may fail the test, all above mentioned TCs should work. I am closing the ticket until I hear back from you. Thanks for your help

[ZeroChaos-]

next branch built with gcc 10 on gentoo

92% tests passed, 60 tests failed out of 793

Label Time Summary:
long      = 392.14 sec*proc (253 tests)
phy       = 392.14 sec*proc (253 tests)
srsenb    =  76.30 sec*proc (5 tests)
srsue     =  25.57 sec*proc (1 test)

Total Test time (real) = 117.42 sec

The following tests FAILED:
	508 - phy_dl_test-p6-t3-m0 (Failed)
	509 - phy_dl_test-p6-t3-m7 (Failed)
	510 - phy_dl_test-p6-t3-m14 (Failed)
	511 - phy_dl_test-p6-t3-m21 (Failed)
	512 - phy_dl_test-p6-t3-m28 (Failed)
	528 - phy_dl_test-p6-t3-q-m0 (Failed)
	529 - phy_dl_test-p6-t3-q-m7 (Failed)
	530 - phy_dl_test-p6-t3-q-m14 (Failed)
	531 - phy_dl_test-p6-t3-q-m21 (Failed)
	532 - phy_dl_test-p6-t3-q-m27 (Failed)
	548 - phy_dl_test-p15-t3-m0 (Failed)
	549 - phy_dl_test-p15-t3-m7 (Failed)
	550 - phy_dl_test-p15-t3-m14 (Failed)
	551 - phy_dl_test-p15-t3-m21 (Failed)
	552 - phy_dl_test-p15-t3-m28 (Failed)
	568 - phy_dl_test-p15-t3-q-m0 (Failed)
	569 - phy_dl_test-p15-t3-q-m7 (Failed)
	570 - phy_dl_test-p15-t3-q-m14 (Failed)
	571 - phy_dl_test-p15-t3-q-m21 (Failed)
	572 - phy_dl_test-p15-t3-q-m26 (Failed)
	588 - phy_dl_test-p25-t3-m0 (Failed)
	589 - phy_dl_test-p25-t3-m7 (Failed)
	590 - phy_dl_test-p25-t3-m14 (Failed)
	591 - phy_dl_test-p25-t3-m21 (Failed)
	592 - phy_dl_test-p25-t3-m28 (Failed)
	608 - phy_dl_test-p25-t3-q-m0 (Failed)
	609 - phy_dl_test-p25-t3-q-m7 (Failed)
	610 - phy_dl_test-p25-t3-q-m14 (Failed)
	611 - phy_dl_test-p25-t3-q-m21 (Failed)
	612 - phy_dl_test-p25-t3-q-m27 (Failed)
	628 - phy_dl_test-p50-t3-m0 (Failed)
	629 - phy_dl_test-p50-t3-m7 (Failed)
	630 - phy_dl_test-p50-t3-m14 (Failed)
	631 - phy_dl_test-p50-t3-m21 (Failed)
	632 - phy_dl_test-p50-t3-m28 (Failed)
	648 - phy_dl_test-p50-t3-q-m0 (Failed)
	649 - phy_dl_test-p50-t3-q-m7 (Failed)
	650 - phy_dl_test-p50-t3-q-m14 (Failed)
	651 - phy_dl_test-p50-t3-q-m21 (Failed)
	652 - phy_dl_test-p50-t3-q-m27 (Failed)
	668 - phy_dl_test-p75-t3-m0 (Failed)
	669 - phy_dl_test-p75-t3-m7 (Failed)
	670 - phy_dl_test-p75-t3-m14 (Failed)
	671 - phy_dl_test-p75-t3-m21 (Failed)
	672 - phy_dl_test-p75-t3-m28 (Failed)
	688 - phy_dl_test-p75-t3-q-m0 (Failed)
	689 - phy_dl_test-p75-t3-q-m7 (Failed)
	690 - phy_dl_test-p75-t3-q-m14 (Failed)
	691 - phy_dl_test-p75-t3-q-m21 (Failed)
	692 - phy_dl_test-p75-t3-q-m27 (Failed)
	708 - phy_dl_test-p100-t3-m0 (Failed)
	709 - phy_dl_test-p100-t3-m7 (Failed)
	710 - phy_dl_test-p100-t3-m14 (Failed)
	711 - phy_dl_test-p100-t3-m21 (Failed)
	712 - phy_dl_test-p100-t3-m28 (Failed)
	728 - phy_dl_test-p100-t3-q-m0 (Failed)
	729 - phy_dl_test-p100-t3-q-m7 (Failed)
	730 - phy_dl_test-p100-t3-q-m14 (Failed)
	731 - phy_dl_test-p100-t3-q-m21 (Failed)
	732 - phy_dl_test-p100-t3-q-m27 (Failed)

[andrepuschmann]

Can you post your entire cmake output, please? Also are you running with memcheck, i.e. valgrind? Parallel tests? Please post also the stdout of one of the failed tests.

[ZeroChaos-]

I would, but the next branch appears to have disappeared.... you want me to build master?

[andrepuschmann]

Ah, yes. We've just updated master. Please use this branch. Thanks

[ZeroChaos-]

The requested info is a bit large, so I had to put it here https://dev.gentoo.org/~zerochaos/net-wireless:srslte-9999:20201028-144208.log.gz

[andrepuschmann]

I can't access the file I am afraid. Just post the stdout of one of the failed tests and CMake output. I believe the reason is the same.

[ZeroChaos-]

My apologies, you can access it now

[andrepuschmann]

I believe it's because of the parallel execution of tests (see ctest -j 12 --test-load 12). Perhaps a race with loading the fftw_wisdom.

Could you provide this file cat ./Testing/Temporary/LastTest.log from the main build folder, please?

[ZeroChaos-]

Updated for 20.10.1

92% tests passed, 60 tests failed out of 793

Label Time Summary:
long      = 492.36 sec*proc (253 tests)
phy       = 492.36 sec*proc (253 tests)
srsenb    = 115.82 sec*proc (5 tests)
srsue     =  32.59 sec*proc (1 test)

Total Test time (real) = 173.28 sec

The following tests FAILED:
	508 - phy_dl_test-p6-t3-m0 (Failed)
	509 - phy_dl_test-p6-t3-m7 (Failed)
	510 - phy_dl_test-p6-t3-m14 (Failed)
	511 - phy_dl_test-p6-t3-m21 (Failed)
	512 - phy_dl_test-p6-t3-m28 (Failed)
	528 - phy_dl_test-p6-t3-q-m0 (Failed)
	529 - phy_dl_test-p6-t3-q-m7 (Failed)
	530 - phy_dl_test-p6-t3-q-m14 (Failed)
	531 - phy_dl_test-p6-t3-q-m21 (Failed)
	532 - phy_dl_test-p6-t3-q-m27 (Failed)
	548 - phy_dl_test-p15-t3-m0 (Failed)
	549 - phy_dl_test-p15-t3-m7 (Failed)
	550 - phy_dl_test-p15-t3-m14 (Failed)
	551 - phy_dl_test-p15-t3-m21 (Failed)
	552 - phy_dl_test-p15-t3-m28 (Failed)
	568 - phy_dl_test-p15-t3-q-m0 (Failed)
	569 - phy_dl_test-p15-t3-q-m7 (Failed)
	570 - phy_dl_test-p15-t3-q-m14 (Failed)
	571 - phy_dl_test-p15-t3-q-m21 (Failed)
	572 - phy_dl_test-p15-t3-q-m26 (Failed)
	588 - phy_dl_test-p25-t3-m0 (Failed)
	589 - phy_dl_test-p25-t3-m7 (Failed)
	590 - phy_dl_test-p25-t3-m14 (Failed)
	591 - phy_dl_test-p25-t3-m21 (Failed)
	592 - phy_dl_test-p25-t3-m28 (Failed)
	608 - phy_dl_test-p25-t3-q-m0 (Failed)
	609 - phy_dl_test-p25-t3-q-m7 (Failed)
	610 - phy_dl_test-p25-t3-q-m14 (Failed)
	611 - phy_dl_test-p25-t3-q-m21 (Failed)
	612 - phy_dl_test-p25-t3-q-m27 (Failed)
	628 - phy_dl_test-p50-t3-m0 (Failed)
	629 - phy_dl_test-p50-t3-m7 (Failed)
	630 - phy_dl_test-p50-t3-m14 (Failed)
	631 - phy_dl_test-p50-t3-m21 (Failed)
	632 - phy_dl_test-p50-t3-m28 (Failed)
	648 - phy_dl_test-p50-t3-q-m0 (Failed)
	649 - phy_dl_test-p50-t3-q-m7 (Failed)
	650 - phy_dl_test-p50-t3-q-m14 (Failed)
	651 - phy_dl_test-p50-t3-q-m21 (Failed)
	652 - phy_dl_test-p50-t3-q-m27 (Failed)
	668 - phy_dl_test-p75-t3-m0 (Failed)
	669 - phy_dl_test-p75-t3-m7 (Failed)
	670 - phy_dl_test-p75-t3-m14 (Failed)
	671 - phy_dl_test-p75-t3-m21 (Failed)
	672 - phy_dl_test-p75-t3-m28 (Failed)
	688 - phy_dl_test-p75-t3-q-m0 (Failed)
	689 - phy_dl_test-p75-t3-q-m7 (Failed)
	690 - phy_dl_test-p75-t3-q-m14 (Failed)
	691 - phy_dl_test-p75-t3-q-m21 (Failed)
	692 - phy_dl_test-p75-t3-q-m27 (Failed)
	708 - phy_dl_test-p100-t3-m0 (Failed)
	709 - phy_dl_test-p100-t3-m7 (Failed)
	710 - phy_dl_test-p100-t3-m14 (Failed)
	711 - phy_dl_test-p100-t3-m21 (Failed)
	712 - phy_dl_test-p100-t3-m28 (Failed)
	728 - phy_dl_test-p100-t3-q-m0 (Failed)
	729 - phy_dl_test-p100-t3-q-m7 (Failed)
	730 - phy_dl_test-p100-t3-q-m14 (Failed)
	731 - phy_dl_test-p100-t3-q-m21 (Failed)
	732 - phy_dl_test-p100-t3-q-m27 (Failed)

Just to test the parallel execution theory, I went ahead and tested with "ctest -j1" and it failed the same 60 tests, but slower.

LastTest.log is 368M (54M compressed), but if you want it you can download it here (from the -j1 run, just in case it matters):
https://pentoo.org/~zero/LastTest.log.xz