How to pass connectivity check without default gateway?

[mcanevet]

Context

I'm running Talos on OpenNebula (but the issue exists also for other platforms I suppose) in a network environment where we don't have a default gateway route (0.0.0.0/0). The network configuration is managed through the machine config, and connectivity goes through a corporate proxy.

Issue

Talos seems to be stuck waiting for connectivity, and from looking at the code (internal/app/machined/pkg/controllers/network/status.go), I can see that ConnectivityReady requires either:

• A default gateway route, OR
• Successfully configured network probes

Since we don't have a default route, I'm trying to understand how to use network probes.

What I Found

I discovered that there's a probe mechanism in Talos that can validate connectivity by checking TCP endpoints. Looking at the code, I can see:

1. The PlatformConfigSpec has a Probes field
2. The metal platform can read probe configuration from META partition (key 0xa)
3. Example probe format:

probes:
  - interval: 1s
    tcp:
      endpoint: proxy.example.com:3128
      timeout: 10s

Questions

1. Is there a way to configure network probes on OpenNebula platform? Or is this feature only available for metal platform?
2. Is there any workaround to pass the connectivity check without a default gateway? Could I configure this through machine config or some other method?
3. Can I use talos.platform=metal on OpenNebula? Would this allow me to use META partition for probe configuration and machine configuration, or would it break other things?

I'd appreciate any guidance on how to handle this scenario. Thanks!

[protonjhow]

I can appreciate that a default route is pretty common scenario, but it doesn't inherently mean there is connectivity either.

Flannel for example has a similar approach, and they had to offer --iface and --iface-can-reach because chasing a default isn't ideal for a number of reasons.

I am happy to produce a PR to make this more configurable, if there is agreement that this can be revised?

[protonjhow]

ref to the config options on flannel that got lost to md: (https://github.com/flannel-io/flannel/blob/master/Documentation/configuration.md#key-command-line-options)

[smira]

We probably need to expose network probes in the machine configuration to make it easier to configure, and in general we can make connectivity check configurable. This shouldn't be painful.

[mcanevet]

Here is a patch to expose the probes in machine configuration: #12606

[mcanevet]

This could also work for us (ability to declare a blackhole as default gateway): #12608