clarification of cloud config user instantiation

[clanktron]

In the spirit of kairos being immutable, and considering I don't see /etc/passwd included in the COS_PERSISTENT partition, I'm assuming users/groups are reconstructed after every boot. This is great, what I'm confused about is how user lifecycle management is support to work declaratively.

There seems to be 4 ways to instantiate a user:

1. top level users entry in your cloud-config file
2. users entry in one of the stages layers of your cloud-config
3. an ensure_entities statement at any of the stages layers
4. baked into the container image

A few questions:

• When a user is declared at the top level of a cloud-config file, is this applied at every boot or just the initial install?

The docs state that if you want to add a user post-install that you should add a users entry in the initramfs stage. This to me implies that all stages are run at every boot, and that the top-level declarations are only pertinent for the initial install. If this is the case, why not skip the top-level entirely and only put the users entry in your chosen stage? I initially did this but it seems that not having a top-level users declaration isn't valid syntax (kairos-agent validate states the rendered json is invalid).

• What takes precedence in cloud config, users or ensure_entities? If both have overlapping entries how are these handled?

Say I have a users entry and an ensure_entities entry in the same "step" of a stage. Or say I have a user in the initramfs stage and an ensure_entities user in the fs stage. Is any sort of merging done here or is this just undefined/unsupported? In the docs only passwd and ssh_authorized_keys are evaluated if the user already exists, but this would only cover the situation where ensure_entities is used before users and not the other way around. I'm not necessarily interested in doing this but I think the behavior here should be clearly defined.

• When can I expect users declared at the top level to be available?

I've tried doing some things as a certain user (with su or just changing some directory/file permissions etc) in the stages section. But it seems that none of my created users are available on the system at this point? My users are available by the time I can log in via console/ssh so that's not the issue.

For example, the following error is a step in my cloud config that runs at the fs stage.

Oct 12 04:46:06 nas kairos-agent[1791]: ERRO[2023-10-12T04:46:06Z] chown: invalid user: ‘clayton:clayton’
Oct 12 04:46:06 nas kairos-agent[1791]: : failed to run chown -R clayton:clayton /scratch/dotfiles && su clayton -c "HOME=/home/clayton /scratch/dotfiles/install" >

I assumed that fs was the "final" stage and that I could use my users by this point/stage (I assumed I could use them at any stage to be honest). When can I assume they'll be available? If I want do user related things do I have to add an ensure_entities or users in a prior stage (or at least before the step on the same stage) as opposed to a top-level instantiation?

---

This next question could almost be categorized as a separate issue, but it seems mildly related so I thought I'd include it here. Is more of the "real" cloud init spec going to be supported in the future? Namely user fields like sudo:, doas:, expiredate:, etc.

[Itxaka]

Hi @clanktron

The users at the top level is mainly for backwards compatibility with cloud config. Because yip supports a subset of cloud-config, is possible to set it at the top level.

IMO we should have dropped that a long time ago and only support stages but it has to be maintained due to backwards compatibility.

The docs state that if you want to add a user post-install that you should add a users entry in the initramfs stage. This to me implies that all stages are run at every boot, and that the top-level declarations are only pertinent for the initial install. If this is the case, why not skip the top-level entirely and only put the users entry in your chosen stage?

Indeed stages are run on each boot.
Top level stanzas are ALSO run on each boot, but they are all lumped on the "boot" stage (except hostname stanza which is done during "initramfs"). You can see yip loading the cloud-config and creating the stage on the fly here: https://github.com/mudler/yip/blob/master/pkg/schema/loader_cloudinit.go#L109

I initially did this but it seems that not having a top-level users declaration isn't valid syntax (kairos-agent validate states the rendered json is invalid).

Its only a warning, as its optional to set the users there, we emit a warning but not having users in there should be ok. In fact our test suite does not have the users at the top level: https://github.com/kairos-io/kairos/blob/master/tests/assets/autoinstall.yaml

What takes precedence in cloud config, users or ensure_entities? If both have overlapping entries how are these handled?

Good question, I have no idea :) Let me look that up and come back to you.

When can I expect users declared at the top level to be available?

As mentioned above they are created on the "boot" stage.

The following stages are available and in the following order:

• rootfs: During initramfs but run over initramfs system, so the real root is under /sysroot
• initramfs: During initramfs, but chrooted into what its going to be the final system (so runs under /sysroot)
• boot: after switch_root, so you are on the final system and initramfs has gone away
• fs: same as boot, but you have guaranteeed the existance of COS_OEM and COS_PERSISTENT
• network: Once network is avialable
• reconcile: 5 minutes after boot and then every 60 minutes

So your top level users will be available during the boot stage. If you want to use them then your configs should be done on that stage or later and after the user creation (lexicographic order)

That is why on our own stuff we create them during initramfs and usually as soon as possible: https://github.com/kairos-io/packages/blob/main/packages/static/kairos-overlay-files/files/system/oem/10_accounting.yaml

Let me come back at you with the users/ensure_entities answer

[Itxaka]

What takes precedence in cloud config, users or ensure_entities? If both have overlapping entries how are these handled?

No merging, users uses the ensure_entities lib underneath, and they both check if the user exists before doing anything, so no matter what, they will not overlap.

Then it comes to the ordering of the plugins, in our case, ensure_entities goes first then Users https://github.com/kairos-io/kairos-agent/blob/main/pkg/cloudinit/cloudinit.go#L42

Then it also comes to the files and ordering of the files and stages and steps. If your user stanza is on initramfs it will be created before a entitites in a fs stage for example. And if your user stanza is in a file called 01_user.yaml it will be run before a entitites on a files called 02_users.yaml. And if they are in the same files but on different steps, the first step will be first, no matter if using users or entitites

So the only issue is if you have them in the same file, same stage, same step. In that case, entitites will go first and then users, but they wont be recreated.

So after all user is a more user-friendly way of denifing users, while entities is a more complex and complete system to generate /etc/passwd , /etc/shadow and /etc/groups files.

[Itxaka]

@clanktron moved this to a discussion which seems more appropiate :)

[clanktron]

@Itxaka thanks for the comprehensive explanation! I was still having that issue with the clayton user not being available in the fs stage despite instantiating it at the top level (so boot stage), but I'll be retrying that again soon so I'll report back.

On another note, is more of the official cloud-init user spec planned on being supported? Namely some of the fields I mentioned like expiredate:, doas:, etc. From what I've seen the project really only supports a subset of it (for easy migration purposes?), while leaning more into the stages stuff for more robust config.

[Itxaka]

Yes indeed, you are correct in your analysis, the subset is only supported for an easy transition from cloud-init to yip and nothing is planned to support a bigger subset. For more complex needs we recommend just dropping the cloud-config syntax and fully using the whole yip stuff.

Note that there is nothing preventing us from supporting more syntax nor we have anything against it so if you are interested in expanding it, PRs are welcome and will be accepted for sure :)