`fips.enable.tls` test failing on rhel-9.8

[HuijingHei]

fips.enable.tls kola test is failing on rhel-9.8, maybe also rhel-10.2.

Test is passed locally, also passed on prow CI (see #189), maybe there is something special in CI env.

See log:

[    7.818050] ignition[1074]: Ignition 2.25.1
[    7.818996] ignition[1074]: Stage: files
[    7.819966] ignition[1074]: reading system config file "/usr/lib/ignition/base.d/00-core.ign"
[    7.823491] ignition[1074]: no config dir at "/usr/lib/ignition/base.platform.d/qemu"
[    7.833154] ignition[1074]: Adding "FIPS TLS Test Server" to list of CAs
[    7.843742] ignition[1074]: files: ensureUsers: op(1): [started]  creating or modifying user "core"
[    7.867301] ignition[1074]: files: ensureUsers: op(1): [finished] creating or modifying user "core"
[    7.869169] ignition[1074]: files: ensureUsers: op(2): [started]  adding ssh keys to user "core"
[    7.871879] ignition[1074]: wrote ssh authorized keys file for user: core
[    7.873218] ignition[1074]: files: ensureUsers: op(2): [finished] adding ssh keys to user "core"
[    7.875998] ignition[1074]: files: createFilesystemsFiles: createFiles: op(3): [started]  writing file "/sysroot/etc/ignition-machine-config-encapsulated.json"
[    7.879904] ignition[1074]: files: createFilesystemsFiles: createFiles: op(3): [finished] writing file "/sysroot/etc/ignition-machine-config-encapsulated.json"
[    7.883033] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): [started]  writing file "/sysroot/var/resource/https-fips"
[    7.885909] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET https://34.172.244.189:8443/index.html: attempt #1
[    7.892748] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET error: Get "https://34.172.244.189:8443/index.html": dial tcp 34.172.244.189:8443: connect: connection refused
[    8.096648] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET https://34.172.244.189:8443/index.html: attempt #2
[    8.099571] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET error: Get "https://34.172.244.189:8443/index.html": dial tcp 34.172.244.189:8443: connect: connection refused
[    8.503260] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET https://34.172.244.189:8443/index.html: attempt #3
[    8.506431] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET error: Get "https://34.172.244.189:8443/index.html": dial tcp 34.172.244.189:8443: connect: connection refused
[    9.310866] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET https://34.172.244.189:8443/index.html: attempt #4
[    9.313929] ignition[1074]: files: createFilesystemsFiles: createFiles: op(4): GET error: Get "https://34.172.244.189:8443/index.html": dial tcp 34.172.244.189:8443: connect: connection refused

[jbtrystram]

Test is passed locally, also passed on prow CI (see #189), maybe there is something special in CI env.

Any outbound connection from the ITUP cluster must be allowed by an egress rule. @marmijo made that in the past IIRC.
Edit: i gave the doc link to @angelcerveraroldan to update the egress rule.

[marmijo]

Any outbound connection from the ITUP cluster must be allowed by an egress rule.

That's right! We have to explicitly allow every outbound connection. @HuijingHei is there a reason we're using port 8443 here instead of 443? We're allowed to use port 443 for HTTPS connections but need to request and justify usage of any other port.
See: https://github.com/coreos/coreos-assembler/blob/e265089704cec18983b2996e807db30777319c48/mantle/kola/tests/fips/fips.go#L200

[HuijingHei]

Thank you @marmijo and @jbtrystram , hold on, there are on going some discussion for strategy of the test over in coreos/coreos-assembler#4477 (comment)