k3s systemd file is using After=network-online.target

[oak-tree]

Environmental Info:
K3s Version:

k3s version v1.20.0+k3s2 (2ea6b163)
go version go1.15.5

Node(s) CPU architecture, OS, and Version:

Running on Jetson Xavier, Ubuntu 18.04. Jetpack 4.4

Cluster Configuration:

Single cluster node 

Describe the bug:

When powering the machine the system sometimes run k3s and sometimes not. It seems to be related to networking issue? 

We cannot assume the system have network when its boot. 

Steps To Reproduce:

Poweron a machine without aninternet access or with a weak wifi Signal 

Expected behavior:

Run k3s and its pods of the last deplyoment every powerup of the IOT device 

Actual behavior:

Sometimes the k3s does not start 

Additional context / logs:

It seems to be related to `k3s` `systemd` file. For some reason it asks for `After=network-online.target`. Why not run `k3s` after `After=network.target`. Then be flexable about the network it self. Since we are talking about IOT device, It makes more sense to be flexable about networking .

[network.target vs network-online.target](https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/)

[oak-tree]

"network-online.target is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind.[...]
: for example network server software should generally not pull this in (since server software generally is happy to accept local connections even before any routable network interface is up), its primary purpose is network client software that cannot operate without network*[...]"

source

it might be reasonable to treat k3s as the network server in the above example. what do you think?

[brandond]

its primary purpose is network client software that cannot operate without network

This is K3s. We need a routable IP address (with a default route present on one of the interfaces) in order to configure listeners and determine what addresses to advertise to other nodes in the cluster.

[Shaked]

@brandond

Just to clarify, does it mean that the device's network should work but it doesn't have to be connected to WiFi/ethernet/WAN? Especially when there's only one cluster with only one node - the cluster itself.

Regardless, do you have any idea why k3s won't start after a device restart or local network temporary failure? Or better, what to look for when that happens

[brandond]

I believe that kubernetes itself will fail to start if it doesn't find an interface with a default route. In the past, I've seen folks work around this in airgap environments by setting up a dummy interface and adding a static default route to it. If you need do this, you are of course welcome to customize the k3s systemd unit to start at whatever you feel is the right time for your environment.

This isn't really k3s-specific, but you might look at the output of systemd-analyze critical-chain k3s to figure out why it's not hitting the required target on your node.