Inconsistent pipe parallel performance

[MerelyLogical]

14 Feb:

zw4215, 26, For 1M input, check time for P1 (10.51) is more than 1.6 x P2 (5.30), PASS
zw4215, 27, For 1M input, check time for P1 (10.51) is more than 2.5 x P4 (2.66), PASS
zw4215, 28, For 1M input, check time for P1 (10.51) is more than 4.5 x P8 (2.19), PASS
zw4215, 29, For 1M input, check time for P1 (10.51) is more than 7.0 x P16 (1.07), PASS
zw4215, 30, For 1M input, check time for P1 (10.51) is more than 10.0 x P32 (0.58), PASS

With no modification in the code, on the next day:
15 Feb:

zw4215, 26, For 1M input, check time for P1 (10.51) is more than 1.6 x P2 (5.32), PASS
zw4215, 27, For 1M input, check time for P1 (10.51) is more than 2.5 x P4 (4.32), FAIL
zw4215, 28, For 1M input, check time for P1 (10.51) is more than 4.5 x P8 (2.19), PASS
zw4215, 29, For 1M input, check time for P1 (10.51) is more than 7.0 x P16 (1.09), PASS
zw4215, 30, For 1M input, check time for P1 (10.51) is more than 10.0 x P32 (0.59), PASS

I hope this doesn't suggest that the coursework marking this time will be random as well...

[ThunderMikey]

The same observation from mine on the P1 and P4 test.

[dl5415]

I observe the same problem with P1 and P4

[jjd06]

I was going to put this down to background process noise, especially since @MerelyLogical's is the first that gets run after I start an instance, but it sounds like there's something more systematic going on. There's another test run happening now -- let's see what that one results in. Have no fear: if this continues to be an issue, it will be fixed before the marking is complete.

[ruhi-c]

I am also facing this issue.

In particular, tests 26 - 30 (the bash script pipeline tests) passed for me on all of JD's testing runs (from 2019-02-13 to 2019-02-18) but for the assessment I suddenly failed tests 26 and 27.

I haven't changed the code at all so I don't think I'm at fault here?

For completeness, here are the results of testing from all runs:

jjd06_logs_2019-02-13_19-28-40

arc13, 26, For 1M input, check time for P1 (10.51) is more than 1.6 x P2 (5.35), PASS
arc13, 27, For 1M input, check time for P1 (10.51) is more than 2.5 x P4 (3.46), PASS
arc13, 28, For 1M input, check time for P1 (10.51) is more than 4.5 x P8 (2.18), PASS
arc13, 29, For 1M input, check time for P1 (10.51) is more than 7.0 x P16 (1.08), PASS
arc13, 30, For 1M input, check time for P1 (10.51) is more than 10.0 x P32 (0.58), PASS

jjd06_logs_2019-02-14_18-09-13

arc13, 26, For 1M input, check time for P1 (10.47) is more than 1.6 x P2 (5.31), PASS
arc13, 27, For 1M input, check time for P1 (10.47) is more than 2.5 x P4 (2.66), PASS
arc13, 28, For 1M input, check time for P1 (10.47) is more than 4.5 x P8 (1.36), PASS
arc13, 29, For 1M input, check time for P1 (10.47) is more than 7.0 x P16 (1.09), PASS
arc13, 30, For 1M input, check time for P1 (10.47) is more than 10.0 x P32 (0.58), PASS

jjd06_logs_2019-02-15_16-16-08

arc13, 26, For 1M input, check time for P1 (10.48) is more than 1.6 x P2 (5.35), PASS
arc13, 27, For 1M input, check time for P1 (10.48) is more than 2.5 x P4 (2.65), PASS
arc13, 28, For 1M input, check time for P1 (10.48) is more than 4.5 x P8 (1.35), PASS
arc13, 29, For 1M input, check time for P1 (10.48) is more than 7.0 x P16 (1.06), PASS
arc13, 30, For 1M input, check time for P1 (10.48) is more than 10.0 x P32 (0.59), PASS

jjd06_logs_2019-02-18_07-12-07

arc13, 26, For 1M input, check time for P1 (10.57) is more than 1.6 x P2 (5.31), PASS
arc13, 27, For 1M input, check time for P1 (10.57) is more than 2.5 x P4 (2.71), PASS
arc13, 28, For 1M input, check time for P1 (10.57) is more than 4.5 x P8 (1.34), PASS
arc13, 29, For 1M input, check time for P1 (10.57) is more than 7.0 x P16 (1.12), PASS
arc13, 30, For 1M input, check time for P1 (10.57) is more than 10.0 x P32 (0.59), PASS

jjd06_logs_2019-02-18_19-31-45

arc13, 26, For 1M input, check time for P1 (10.50) is more than 1.6 x P2 (5.31), PASS
arc13, 27, For 1M input, check time for P1 (10.50) is more than 2.5 x P4 (3.36), PASS
arc13, 28, For 1M input, check time for P1 (10.50) is more than 4.5 x P8 (1.69), PASS
arc13, 29, For 1M input, check time for P1 (10.50) is more than 7.0 x P16 (0.69), PASS
arc13, 30, For 1M input, check time for P1 (10.50) is more than 10.0 x P32 (0.60), PASS

jjd06_summative

arc13, 26, For 1M input, check time for P1 (10.53) is more than 1.6 x P2 (7.19), FAIL
arc13, 27, For 1M input, check time for P1 (10.53) is more than 2.5 x P4 (4.26), FAIL
arc13, 28, For 1M input, check time for P1 (10.53) is more than 4.5 x P8 (2.15), PASS
arc13, 29, For 1M input, check time for P1 (10.53) is more than 7.0 x P16 (1.10), PASS
arc13, 30, For 1M input, check time for P1 (10.53) is more than 10.0 x P32 (0.59), PASS

[jjd06]

Sigh. So despite my best efforts, some noise still affected some testing. I have rerun your assessment, @ruhi-c.

[kin-nex]

Would it be possible to re-run mine too? I'm experiencing similar results to Ruhi

[ruhi-c]

Thank you very much @jjd06 =)

[thomasjallerton]

I also saw inconsistent results for test 27, with no change of code;

2019-02-18_19-31-45

tja115, 26, For 1M input, check time for P1 (10.50) is more than 1.6 x P2 (5.38), PASS
tja115, 27, For 1M input, check time for P1 (10.50) is more than 2.5 x P4 (2.71), PASS
tja115, 28, For 1M input, check time for P1 (10.50) is more than 4.5 x P8 (1.87), PASS
tja115, 29, For 1M input, check time for P1 (10.50) is more than 7.0 x P16 (1.11), PASS
tja115, 30, For 1M input, check time for P1 (10.50) is more than 10.0 x P32 (0.57), PASS

2019-02-18_07-12-07

tja115, 26, For 1M input, check time for P1 (10.56) is more than 1.6 x P2 (5.35), PASS
tja115, 27, For 1M input, check time for P1 (10.56) is more than 2.5 x P4 (2.70), PASS
tja115, 28, For 1M input, check time for P1 (10.56) is more than 4.5 x P8 (1.34), PASS
tja115, 29, For 1M input, check time for P1 (10.56) is more than 7.0 x P16 (1.09), PASS
tja115, 30, For 1M input, check time for P1 (10.56) is more than 10.0 x P32 (0.59), PASS

2019-02-15_16-16-08

tja115, 26, For 1M input, check time for P1 (10.49) is more than 1.6 x P2 (5.32), PASS
tja115, 27, For 1M input, check time for P1 (10.49) is more than 2.5 x P4 (2.70), PASS
tja115, 28, For 1M input, check time for P1 (10.49) is more than 4.5 x P8 (1.36), PASS
tja115, 29, For 1M input, check time for P1 (10.49) is more than 7.0 x P16 (1.12), PASS
tja115, 30, For 1M input, check time for P1 (10.49) is more than 10.0 x P32 (0.59), PASS

2019-02-14_18-09-13

tja115, 26, For 1M input, check time for P1 (10.56) is more than 1.6 x P2 (5.27), PASS
tja115, 27, For 1M input, check time for P1 (10.56) is more than 2.5 x P4 (4.32), FAIL
tja115, 28, For 1M input, check time for P1 (10.56) is more than 4.5 x P8 (2.17), PASS
tja115, 29, For 1M input, check time for P1 (10.56) is more than 7.0 x P16 (1.10), PASS
tja115, 30, For 1M input, check time for P1 (10.56) is more than 10.0 x P32 (0.60), PASS

Summative

tja115, 26, For 1M input, check time for P1 (10.49) is more than 1.6 x P2 (5.28), PASS
tja115, 27, For 1M input, check time for P1 (10.49) is more than 2.5 x P4 (4.33), FAIL
tja115, 28, For 1M input, check time for P1 (10.49) is more than 4.5 x P8 (2.19), PASS
tja115, 29, For 1M input, check time for P1 (10.49) is more than 7.0 x P16 (0.69), PASS
tja115, 30, For 1M input, check time for P1 (10.49) is more than 10.0 x P32 (0.59), PASS

[dl5415]

I have experienced inconsistent results as well in the summative assessment.

[jjd06]

Rerun for @ESalih-Dev, @thomasjallerton and @dl5415.

[kin-nex]

Thanks :)

[JinYeeNg]

I also experienced the inconsistent results in my summative test. Is it possible to rerun my test also?

[rmurai0610]

I have also experience the inconsistent results. Would it be possible to rerun my test as well.

[jjd06]

Rerun for @JinYeeNg and @rmurai0610 as well.

[JinYeeNg]

Thank you for the rerun ;-). @jjd06