Enhancement Proposal
Python's default output buffering behavior can prevent print() statements from reaching stdout immediately in non-interactive contexts. As a results, the stdout of a Checkbox suite or test may not accurately reflect what is currently running if underlying test is implemented as a Python script (especially for longer runtime tests, where it can be delayed by hours).
For instance, memory stress-ng tests are implemented in the memory_stress_ng.py Python script. The print() statements execute as expected, but their output does not reach stdout immediately in non-interactive contexts due to Python's default buffering behavior. This test has a long enough total runtime across subtests such that without the intermediate print() statements between subtests reaching stdout when triggered, the output_timeout of a Testflinger agent can be exceeded on such devices before the buffer is finally flushed at the script end.
This can be resolved by setting the environment variable PYTHONUNBUFFERED=1 and ensuring it propagates to these scripts, or adding -u to python3 invocations.
This can also be set in the [environment] section of launchers as needed, which is what I'm doing as a workaround, but I wanted to gather thoughts on whether this could be applied globally / semi-globally in Checkbox, or whether there are side effects or constraints that block this.
Enhancement Proposal
Python's default output buffering behavior can prevent
print()statements from reaching stdout immediately in non-interactive contexts. As a results, the stdout of a Checkbox suite or test may not accurately reflect what is currently running if underlying test is implemented as a Python script (especially for longer runtime tests, where it can be delayed by hours).For instance, memory stress-ng tests are implemented in the memory_stress_ng.py Python script. The
print()statements execute as expected, but their output does not reach stdout immediately in non-interactive contexts due to Python's default buffering behavior. This test has a long enough total runtime across subtests such that without the intermediateprint()statements between subtests reaching stdout when triggered, theoutput_timeoutof a Testflinger agent can be exceeded on such devices before the buffer is finally flushed at the script end.This can be resolved by setting the environment variable
PYTHONUNBUFFERED=1and ensuring it propagates to these scripts, or adding-utopython3invocations.This can also be set in the
[environment]section of launchers as needed, which is what I'm doing as a workaround, but I wanted to gather thoughts on whether this could be applied globally / semi-globally in Checkbox, or whether there are side effects or constraints that block this.