Issue running compliance check in GitHub Actions CI

[ptsouchlos]

Describe the bug
I've been trying to setup a UCI compliance check as part of my CI setup for my chess engine but it consistently fails on step 12 of the compliance check. The odd thing is that I cannot reproduce the behavior when running the compliance check locally.

Currently I'm building fastchess from source in the CI using clang and then using that built binary for the compliance check.

I'm trying to figure out if this is a bug in the check, an issue with the GitHub runner, or with my engine.

To Reproduce
See my PR link below.

Expected behavior
Would be good for the check to match what I see locally.

Environment (please complete the following information):

• OS: Fedora 42 (Local)
• CLI Command: fastchess --compliance byte-knight
• Version: fastchess alpha 1.5.0

Additional context
See this PR for the actions runs that failed ptsouchlos/byte-knight#113

[Disservin]

mh this is very weird.. can you try with the release binary?
https://github.com/Disservin/fastchess/releases/download/v1.6.0-alpha/fastchess-linux-x86-64.tar

[ptsouchlos]

OK I tried with the release binary and it still fails at the same place...

Very strange
https://github.com/DeveloperPaul123/byte-knight/actions/runs/18065945761/job/51408602451?pr=113

[Disservin]

Mh gonna have to think about it a bit and how to best debug this..cause the logger isn't configured for this at that time. So probably a modified fastchess I'll try to get to it tomorrow maybe

[ptsouchlos]

I could try to piece together how logging is configured when I get some time. If you beat me to it that's cool as well. No pressure, appreciate the help!

[Disservin]

I think #925 could be the cause of the issue here, github ci runners are probably slower than your dev machine so a non exact score is reported

[Disservin]

can you try with master again ? I can think you can update your ci to this

gh run download --repo Disservin/fastchess -p "fastchess-linux-x86-64";
mkdir fast-chess && mv fastchess-linux-x86-64 fast-chess
./fast-chess/fastchess-linux-x86-64/fastchess -v

[ptsouchlos]

Thanks for the patch. I just gave this a try and it failed at the same step during the compliance check.

https://github.com/DeveloperPaul123/byte-knight/actions/runs/18430808532/job/52517857640?pr=113

Your snippet for the GitHub action worked though and fastchess reports this as its version:

fastchess alpha 1.6.0 compiled for ubuntu-22.04 20251011-8b32639 (CI)

Also, I was looking through #925 and it made me wonder what I do for low control times. I checked locally, and it seems byte-knight doesn't print out an info line with depth information for just a short control time:

uciok
position startpos
go wtime 100
info string searching rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
info string searching max depth 128 start_time Instant { tv_sec: 9127, tv_nsec: 125789028 } soft_timeout 5ms hard_timeout 20ms
bestmove b1a3

So I'm a bit confused as to how the check passes locally but not in the CI?

[Disservin]

you can log the output like this during compliance check

./fastchess --compliance bash "wrapper.sh ../Stockfish/src/stockfish"

#!/usr/bin/env bash

engine="$1"
shift

logfile="engine.log"

: > "$logfile"

"$engine" "$@" 2>&1 | tee -a "$logfile"

[ptsouchlos]

Thanks Disservin, I've been doing so more testing and it seems that the compliance test fails in the same spot on my laptop. I'll have to check again on my desktop - but it seems there may be a bug in how I output info lines with short time controls.

This was the output in the engine log for a failed run:

 _         _           _        _      _   _   
| |__ _  _| |_ ___ ___| |___ _ (_)__ _| |_| |_ 
| '_ \ || |  _/ -_)___| / / ' \| / _` | ' \  _|
|_.__/\_, |\__\___|   |_\_\_||_|_\__, |_||_\__|
      |__/                       |___/         

byte-knight 3.0.0 by Paul T. (DeveloperPaul123) <developer.paul.123@gmail.com>
option name Hash type spin default 16 min 1 max 1024
option name Threads type spin default 1 min 1 max 1
id name byte-knight
id author Paul T. (DeveloperPaul123)
uciok
readyok
option name Hash type spin default 16 min 1 max 1024
option name Threads type spin default 1 min 1 max 1
id name byte-knight
id author Paul T. (DeveloperPaul123)
uciok
option name Hash type spin default 16 min 1 max 1024
option name Threads type spin default 1 min 1 max 1
id name byte-knight
id author Paul T. (DeveloperPaul123)
uciok
readyok
readyok
readyok
info string searching 3r2k1/p5n1/1pq1p2p/2p3p1/2P1P1n1/1P1P2pP/PN1Q2K1/5R2 w - - 0 27
info string searching max depth 128 start_time Instant { tv_sec: 11400, tv_nsec: 472516387 } soft_timeout 5ms hard_timeout 20ms
bestmove g2g1

[Disservin]

Okay :D I've since merged the PR to show your engines output so that you don't need a wrapper script in case something fails.
I'll close this issue feel free to comment once you fixed it or ran into something else

#932

[ptsouchlos]

So I tried again on my desktop (Windows 11) and used the pre-built binary from here on the same branch/commit for my engine and things pass

I will test again using Linux on my desktop as well to confirm the difference.

[ptsouchlos]

Tested again on my Linux desktop and the result is the same, it passes:

[Disservin]

Hm.. weird, I guess it's very time sensitive, I think you can simply try to rerun your CI workflow and it will pickup the latest master version which should show the engine output.

[ptsouchlos]

Alright I finally patched the issue and my engine passed now! ptsouchlos/byte-knight#113

Here was the fix: ptsouchlos/byte-knight#150