Skip to content

Commit 6e022f5

Browse files
committed
fix: split buildtest into a unit-test pass and a separate benchmark pass
Mirrors credfeto-global-pre-commit's src/scripts/buildtest design: the main dotnet test run now always excludes every benchmark test project (keeping --long-running/--parallel-algorithm aggressive, safe now that benchmarks aren't mixed in), and any benchmark projects found are run individually afterwards, one dotnet test invocation per project, without those two flags - some benchmark projects' test host rejects them as invalid arguments. Also fixes the -b/--no-benchmmarks flag's broken unquoted TEST_BENCHMARKS assignment, which is now load-bearing for skipping the new benchmark phase. Refs #679
1 parent a0bf8ee commit 6e022f5

2 files changed

Lines changed: 47 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ Please ADD ALL Changes to the UNRELEASED SECTION and not a specific release
4949
- check: guard against empty file list before checking — prevents false-positive success when no scripts are found
5050
- git/ignore-changelog: skip push hooks when pushing to target repositories
5151
- git/fetch and git/switchtomain now warn and skip a repo on error instead of aborting the whole run, so remaining repos still get processed.
52-
- development/buildtest no longer passes --long-running/--parallel-algorithm flags to dotnet test, since some benchmark test projects' test host rejects them as invalid arguments (exit code 5, zero tests ran) instead of running
52+
- development/buildtest now runs unit tests first, always excluding benchmark test projects, then runs any benchmark projects found individually without the --long-running/--parallel-algorithm flags, since some benchmark projects' test host rejects them as invalid arguments (exit code 5, zero tests ran) instead of running
5353
### Changed
5454
- Replace raw echo with standard output helpers (die/info/success) in github/cancel-workflows
5555
- Replace raw echo with standard output helpers (die/info/success) in git/update-repos-personal

development/buildtest

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ while [ $# -gt 0 ]; do
6767
shift # past argument
6868
;;
6969
-b|--no-benchmmarks)
70-
TEST_BENCHMARKS=--filter-not-namespace "*.BenchMark.Tests" --filter-not-namespace "*.BenchMark.Tests.*"
70+
TEST_BENCHMARKS=1
7171
shift # past argument
7272
;;
7373
-i|--no-integratopn)
@@ -161,12 +161,27 @@ dotnet build \
161161
-nodeReuse:False || die "Build Failed"
162162

163163
info "Testing..."
164+
# Every benchmark test project is always excluded here, regardless of the
165+
# -b/--no-benchmmarks flag; benchmarks (if any) are run separately below, one
166+
# project per dotnet test invocation, since some benchmark projects' test
167+
# host rejects --long-running/--parallel-algorithm as invalid arguments when
168+
# passed to a combined run.
169+
#
170+
# Keep this split in sync with src/scripts/buildtest in
171+
# credfeto/credfeto-global-pre-commit - that repo is the source of truth for
172+
# this design; this file is a local fork/vendored copy, not consumed
173+
# directly from there (see credfeto/scripts#679, credfeto/scripts#723).
164174
# shellcheck disable=SC2086
165175
dotnet test \
166176
--configuration Release \
167177
--no-restore \
168178
--no-build \
169-
$TEST_BENCHMARKS \
179+
--long-running 10 \
180+
--parallel-algorithm aggressive \
181+
--filter-not-namespace "*.Benchmark.Tests" \
182+
--filter-not-namespace "*.Benchmark.Tests.*" \
183+
--filter-not-namespace "*.BenchMark.Tests" \
184+
--filter-not-namespace "*.BenchMark.Tests.*" \
170185
$TEST_INTEGRATION \
171186
--ignore-exit-code 8 \
172187
"-p:ApiCompatGenerateSuppressionFile=true" \
@@ -177,6 +192,35 @@ dotnet test \
177192
"-p:Version=0.0.0.1-test" \
178193
|| die "Tests Failed"
179194

195+
if [ -z "$TEST_BENCHMARKS" ]; then
196+
BENCH_PROJECTS=$(find "$SOURCE_DIR" -type f -iname "*.csproj" -not -path "*/obj/*" -not -path "*/bin/*" | while IFS= read -r proj; do
197+
bn=$(basename "$proj")
198+
printf '%s\n' "$bn" | grep -qiE '(^|[.])bench(mark)?[.]tests?[.]csproj$' && printf '%s\n' "$proj"
199+
done)
200+
201+
if [ -n "$BENCH_PROJECTS" ]; then
202+
while IFS= read -r bench_project; do
203+
[ -n "$bench_project" ] || continue
204+
205+
info "Running benchmark tests: $bench_project"
206+
dotnet test "$bench_project" \
207+
--configuration Release \
208+
--no-restore \
209+
--no-build \
210+
--ignore-exit-code 8 \
211+
"-p:ApiCompatGenerateSuppressionFile=false" \
212+
"-p:ContinuousIntegrationBuild=true" \
213+
"-p:IsProduction=true" \
214+
"-p:Optimize=true" \
215+
"-p:SuppressNETCoreSdkPreviewMessage=true" \
216+
"-p:Version=0.0.0.1-test" \
217+
|| die "Benchmark Tests Failed ($bench_project)"
218+
done <<BENCH_EOF
219+
$BENCH_PROJECTS
220+
BENCH_EOF
221+
fi
222+
fi
223+
180224
if [ "$PACK" = "1" ]; then
181225
info "Packing..."
182226
dotnet pack \

0 commit comments

Comments
 (0)