When you run tests/ssl-opt.sh on a PSK-only configuration, it automatically detects the requirements of some test cases, e.g. a test case that uses tickets=1 is skipped when MBEDTLS_SSL_TICKET_C. Furthermore, the script detects that this configuration is PSK-only, with two consequences: test cases that intrinsically require a certificate are skipped, and test cases that don't require a certificate are adapted to work with pre-shared keys (e.g. replacing -cert by -psk in the client and server calls).

In several places, we rely on shell globs to analyze a string, and we use [^bar] to match any character other than b, a or r. This is not portable: it should be [!bar] (this is a difference between shell globs and the usual regular expression syntaxes). Using ^ to negate a character range happens to work in some shells, but in others ^ is just an ordinary chararacter. On shells where ^ is an ordinary characters, in many configurations that have less than the default feature set, test cases that should be skipped are instead executed and fail. There may also be undue skips, although I haven't identified any so far.

ssl-opt.sh runs under /bin/sh, which has different implementations on different platforms. On the CI, we use Ubuntu, where /bin/sh is dash, and different versions of dash treat ^ differently. As far as I can tell, all the components where it matters run on our default Linux platform, which is Ubuntu 16.04, where ^ is a negation (same on all Ubuntu 18.04, 20.04 and 22.04). Thus this is currently not a problem on the CI (but it's a ticking bomb). It is a problem on many developer machines, since we tend to use recent Ubuntu (those of us who don't use a Mac), and Ubuntu ≥ 24.04 is affected.

Typical symptom: on Ubuntu 24.04 or other systems where /bin/sh is not bash, in a configuration with certain features disabled (especially a PSK-only configuration on 2.28 — on 3.6 PSK-only detection works fine), some test cases run even though you'd expect them to be skipped.

Workaround: run bash tests/ssl-opt.sh instead of tests/ssl-opt.sh. (If it's invoked via all.sh, you have to edit the invocation of ssl-opt.sh in all.sh or components-*.sh.)