Skip to content

Log build steps in the outcome file #129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+112 −2
Open

Log build steps in the outcome file #129

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0560e56
Record build steps in the outcome file
gilles-peskine-arm Jun 18, 2024
ef8d12d
Ignore weirdness from cmake
gilles-peskine-arm Jun 20, 2024
a5a28ed
Write an outcome file line for every component
gilles-peskine-arm Oct 30, 2024
4ce986e
Don't log `make -q` calls in the outcome file
gilles-peskine-arm Oct 30, 2024
a11f2eb
Outcome lines from make: indicate when using a different makefile
gilles-peskine-arm Oct 31, 2024
e71d3b3
Less misleading outcome file line for every component
gilles-peskine-arm Jan 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions scripts/all-core.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,9 @@ pre_initialize_variables () {
# Specify character collation for regular expressions and sorting with C locale
export LC_COLLATE=C

# Make the location of the root directory available to reporting scripts
export MBEDTLS_TEST_ROOT="$PWD"

: ${MBEDTLS_TEST_OUTCOME_FILE=}
: ${MBEDTLS_TEST_PLATFORM="$(uname -s | tr -c \\n0-9A-Za-z _)-$(uname -m | tr -c \\n0-9A-Za-z _)"}
export MBEDTLS_TEST_OUTCOME_FILE
Expand Down Expand Up @@ -790,6 +793,19 @@ pre_prepare_outcome_file () {
fi
}

## write_outcome_line SUITE CASE RESULT [CAUSE]
## Write a line in the outcome file if enabled.
## Report $MBEDTLS_TEST_PLATFORM, $MBEDTLS_TEST_CONFIGURATION and the
## supplied arguments.
write_outcome_line () {
if [ -z "$MBEDTLS_TEST_OUTCOME_FILE" ]; then
return
fi
printf '%s;%s;%s;%s;%s;%s\n' >>"$MBEDTLS_TEST_OUTCOME_FILE" \
"$MBEDTLS_TEST_PLATFORM" "$MBEDTLS_TEST_CONFIGURATION" \
"$1" "$2" "$3" "${4-}"
}

pre_print_configuration () {
if [ $QUIET -eq 1 ]; then
return
Expand Down Expand Up @@ -983,6 +999,12 @@ run_component () {
fi
fi

if [ $component_status -eq 0 ]; then
write_outcome_line "all.sh" "$current_component" "PASS"
else
write_outcome_line "all.sh" "$current_component" "FAIL" "$component_status"
fi

# Restore the build tree to a clean state.
cleanup
unset current_component
Expand Down
87 changes: 87 additions & 0 deletions scripts/quiet/make
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,90 @@ NO_SILENCE=" --version | test "
TOOL="make"

. "$(dirname "$0")/quiet.sh"
EXIT_STATUS=$?

log_outcome () {
# Options passed to make indicating that a different makefile is used
modifier=
# Space-separated list of targets
targets=
# Temporary variable used in the loop after an option that takes an argument
skip=

for arg in "$@"; do
if [ -n "$skip" ]; then
if [ "$skip" = "modifier" ]; then
modifier="$modifier$arg "
fi
skip=
continue
fi

case $arg in
--dry-run|--help|--just-print|--question|--recon|--version|-n|-q|-v)
# do-nothing option
return;;
--directory=*|--file=*|-C?*|-f?*) # Modifier option with its argument
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are both "--directory=somedir" and "--directory somedir" required?

For example would passing "--directory" and nothing else assert skip=1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With --directory=..., we want to remember the option and move on to the next argument which is another option or a non-option argument. With --directory, we want to skip the next argument, which is the argument of the --directory option.

make --directory is invalid. I don't know what the code here does in this case, but I don't think this wrapper needs to care about invalid usage of make.

modifier="$modifier$arg ";;
--directory|--file|-C|-f) # Modifier option with separate argument
skip=modifier
modifier="$modifier$arg ";;
--assume-new|--assume-old| \
--include-dir|--makefile|--new-file|--old-file|--what-if| \
-I|-W|-k|-o) # Boring option with separate argument
skip=1; continue;;
-*) continue;; # Other option (assumed boring)
*=*) continue;; # Variable assignment
*[!-+./0-9@A-Z_a-z]*) # Target with problematic character
targets="$targets ${arg%%[!-+./0-9@A-Z_a-z]*}...";;
*) # Normal target
targets="$targets $arg";;
esac
done

if [ -n "$MBEDTLS_TEST_ROOT" ] && [ "$MBEDTLS_TEST_ROOT" != "$PWD" ]; then
# Record that `make` was run in a different directory.
# This is only accurate when run by all.sh (or more generally
# if $MBEDTLS_TEST_ROOT is set in the environment).
case "$PWD" in
"$MBEDTLS_TEST_ROOT/"*) modifier="-C ${PWD#"${MBEDTLS_TEST_ROOT}/"} $modifier";;
*) modifier="-C $PWD $modifier";;
esac
fi

if [ -n "$targets" ]; then
targets=${targets# }
else
# Assume that the default target is "all". This is true for
# the toplevel Makefile and when using CMake, but might not be
# true when using a different makefile.
targets=all
fi

# We have a single pass/fail status. This is not accurate when there are
# multiple targets: it's possible that some passed and some failed.
# To figure out which targets passed when the overall result is a failure,
# we'd have to do some complex parsing of logs.
if [ $EXIT_STATUS -eq 0 ]; then
result=PASS
else
result=FAIL
fi
cause= # Identifying failure causes would be nice, but difficult

for target in $targets; do
case "$target" in
"clean"|"neat") continue;; # Boring
cmTC_*) continue;; # Weirdness from CMake
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are temporary target files that will be cleaned up. So the logic is sound. It is not obscure but standard even though unfortunately undocumented behavior from cmake. ;(

esac
echo >>"${MBEDTLS_TEST_OUTCOME_FILE}" \
"${MBEDTLS_TEST_PLATFORM};${MBEDTLS_TEST_CONFIGURATION};${TOOL};${target};${result};${cause}"
done
}

if [ -n "${MBEDTLS_TEST_OUTCOME_FILE}" ] &&
[ -n "${MBEDTLS_TEST_CONFIGURATION}" ]; then
log_outcome "$@"
fi

exit $EXIT_STATUS
5 changes: 3 additions & 2 deletions scripts/quiet/quiet.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ fi

if [[ " $@ " =~ $NO_SILENCE || -n "${VERBOSE_LOGS}" ]]; then
# Run original command with no output supression
exec "${ORIGINAL_TOOL}" "$@"
"${ORIGINAL_TOOL}" "$@"
else
# Run original command and capture output & exit status
TMPFILE=$(mktemp "quiet-${TOOL}.XXXXXX")
Expand All @@ -75,5 +75,6 @@ else
rm "${TMPFILE}"

# Propagate the exit status
exit $EXIT_STATUS
set_status () { return $1; }
set_status $EXIT_STATUS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the indirection?

If we want to preserve the status, could't we just return "EXIT_STATUS" ? What am I missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make includes (“sources”) quiet.sh with the . command. Calling exit inside quiet.sh would exit the whole program. This was ok until now because make didn't want to do anything else in this case. But now make wants to run additional logging code. So we have to return from quiet.sh, and communicate the status.

Note that in sh, there is no language feature to return early from a sub-script sourced with the . command. In ksh, bash and zsh, you can do it with the return builtin, but that extension is not available in a basic sh shell such as dash.

fi