Add docs about execution order of Cloud-init files, steps, and substeps #2294

jjqq2013 · 2025-07-08T02:36:02Z

Just fix typos, add some explanation about execution order of Cloud-init files, steps, and substeps, which are not that obvious to users.

m-ildefons · 2025-07-29T15:12:52Z

docs/content/en/docs/Reference/cloud_init.md

+- The execution order of multiple `substeps` (such as `files`, `commands`, `environment` etc.) within a single `step` is not guaranteed. 
+  Therefore, when you have multiple `substeps` that related to each other, please do not define them in a single `step`.
+  Instead, you can split the `substeps` into multiple `steps` which are ensured to be executed by the definition order.
+- The name of `steps` and output of `substeps` will be output to system journal log which can be viewed by the command `journalctl -u 'elemental*'`.
+- It is highly recommended to declare the `name` property of *steps*, so that it will be easier to investigate the output of `journalctl -u 'elemental*'` by the name.


The wording here is pretty confusing.
In other places in the documentation, it's referred to as stage and action:

stages: fs: # <-- this is a "stage" - name: "foobar" # <-- this is an "action" command: echo foobar

the fs is obviously an stage.
The properties in it seems is called as "steps" in other documents, instead of action, e.g., https://rancher.github.io/elemental-toolkit/docs/customizing/stages/

And https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/

How do you think?

Should we change steps to actions?

The point I'd like to clarify is the substeps which is not explicitly described anywhere.

stages: fs: # <-- this is a "stage" - name: "foobar" # <-- this is an "step title" commands: # <-- I'd like to call this as a substep. - echo foobar - command2 files: # <-- I'd like to call this as a substep. - path: .. content: ... - path: .. content: ...

It is not clear about if the commands are executed before files or reverse.

(But of course inside the same substep, such as commands, it is an array, each item in it is executed by the definition order).

m-ildefons · 2025-07-29T15:16:36Z

docs/content/en/docs/Customizing/stages.md

 - During upgrades, installation, deployments  , and resets

-Cloud-init files in `/system/oem`, `/oem` and `/usr/local/oem` are applied in 5 different phases: `boot`, `network`, `fs`, `initramfs` and `reconcile`. All the available cloud-init keywords can be used in each stage. Additionally, it's possible also to hook before or after a stage has run, each one has a specific stage which is possible to run steps: `boot.after`, `network.before`, `fs.after` etc.
+Cloud-init files in `/system/oem`, `/oem`, `/usr/local/oem`, and kernel boot args are applied in 5 different phases: `boot`, `network`, `fs`, `initramfs` and `reconcile`. All the available cloud-init keywords can be used in each stage. Additionally, it's possible also to hook before or after a stage has run, each one has a specific stage which is possible to run steps: `boot.after`, `network.before`, `fs.after` etc.


The reconcile phase is special, because it's executed repeatedly on a timer. The other phases are executed during system boot.

Thanks for pointing the reconcile phase, but that is not my modification.
My modification is only to add the kernel boot args.

BTW, here you can see, the term phases is used for stages such as fs. Should we unify them? But probably it is not that confusing.

I would agree to change phases to stages in the above examples 👍

@frelon Changed to stages, but only for above case.

m-ildefons · 2025-07-29T15:21:58Z

docs/content/en/docs/Reference/cloud_init.md

+Each cloud-config file is loaded and executed only at the appropriate stage, this allows further components to emit their own stages at the desired time.
+
+_Note_:
+- The execution order of multiple `substeps` (such as `files`, `commands`, `environment` etc.) within a single `step` is not guaranteed. 


As far as I can tell, this is not true. The execution order within a stage should be as-listed in each file and each group of actions in each file should be executed according to the alphabetical order of filenames.

This is what I'd like to clarify.
The order is not the order defined in the yaml, this is mainly because the definition order of properties in yaml(or json) is not respected.

stages: fs: - name: test environment_file: /tmp/env_file1 environment: ENV1: this is ENV1 files: - path: /tmp/file1 content: "this is file1\n" directories: - path: /tmp/dir1 users: user1: passwd: password commands: - | bash -x >> /tmp/log 2>&1 <<'EOF' ls -lFd /tmp/env_file1 printenv ENV1 ls -lFd /tmp/file1 ls -lFd /tmp/dir1 id user1 EOF

You might expect that when the commands get executed, all things defined above it has been executed, such as the user1 has been created.
But it is not. As of now, the output (/tmp/log) is:

+ ls -lFd /tmp/env_file1 ls: cannot access '/tmp/env_file1': No such file or directory + printenv ENV1 this is ENV1 + ls -lFd /tmp/file1 ---------- 1 root root 14 Aug 6 07:13 /tmp/file1 + ls -lFd /tmp/dir1 d--------- 2 root root 40 Aug 6 07:13 /tmp/dir1/ + id user1 id: ‘user1’: no such user

Hi!

The execution order of steps in a stage is defined here. This means that the files(EnsureFiles plugin) is always executed before the commands plugin for example.

@fferrann @m-ildefons Thanks for pointing out the ref. Yes, I know where is the definition, but most users don't have time to check source code for such things. And there is no obvious logical reason that why it is implemented in such a way, it is just happened to be that order. So I think the document should explicitly tell users about the order is not like what they might think.

The order might change in the future, such as when adding a new substeps, then where is should be put? and who is going to determine the order? Who can know the order just by the substep name without thinking deeply or checking source code?

m-ildefons · 2025-07-29T15:23:09Z

docs/content/en/docs/Reference/cloud_init.md

+  Therefore, when you have multiple `substeps` that related to each other, please do not define them in a single `step`.
+  Instead, you can split the `substeps` into multiple `steps` which are ensured to be executed by the definition order.


IMO this is pretty hard to understand. An example would help.

updated with an example. https://github.com/rancher/elemental-toolkit/pull/2294/files#r2256131749

jjqq2013 · 2025-08-06T05:34:29Z

@m-ildefons Thank you so much for taking a look into this. I will modify the document. Your comments are very helpful.

jjqq2013 · 2025-08-06T07:24:05Z

I have updated some description.

jjqq2013 · 2025-08-06T07:24:42Z

docs/content/en/docs/Reference/cloud_init.md

+  ```
+
+  You might expect that when the `commands` get executed, all things defined above it has been executed, such as the user1 has been created.
+  But it is not.  As of now, the output (/tmp/log) is:


frelon

Some comments, also I would remove any mention of substeps, yip works using stages and steps.

docs/content/en/docs/Customizing/stages.md

frelon · 2025-08-06T07:27:35Z

docs/content/en/docs/Reference/cloud_init.md

+            EOF
+  ```
+
+  You might expect that when the `commands` get executed, all things defined above it has been executed, such as the user1 has been created.


As described before the execution ordering is specified when loading the plugins in

elemental-toolkit/pkg/cloudinit/cloudinit.go

Line 52 in d8450e0

executor.WithPlugins(

Let me add the reference to the document. Welcome to correct me.

@frelon Added at https://github.com/rancher/elemental-toolkit/pull/2294/files#r2256178003

jjqq2013 · 2025-08-06T07:37:21Z

docs/content/en/docs/Reference/cloud_init.md

+  ```
+
+  The internal order is depending on the implementation, see https://github.com/rancher/elemental-toolkit/blob/d8450e01b9119b76670421880070d2cc7fa5f936/pkg/cloudinit/cloudinit.go#L52.
+  You can see that the `plugins.User` is behind `plugins.Commands`.


jjqq2013 · 2025-08-06T07:51:27Z

docs/content/en/docs/Reference/cloud_init.md

+            ls -lFd /tmp/file1
+            ls -lFd /tmp/dir1
+            id user1
+            EOF


mark. See above "step1" and "step2". Multiple steps defined in a stage will be executed by the definition order,
but inside each step, there are multiple plugin in it, such as files, 'commands`, what should we call them? @frelon

jjqq2013 · 2025-08-06T07:52:19Z

Some comments, also I would remove any mention of substeps, yip works using stages and steps.

@frelon Welcome to give it a better name. What would you call the properties under same step? Could you please check https://github.com/rancher/elemental-toolkit/pull/2294/files#r2256228693 ?

docs/content/en/docs/Reference/cloud_init.md

… and substeps Signed-off-by: jjqq2013 <[email protected]>

jjqq2013 requested a review from a team as a code owner July 8, 2025 02:36

jjqq2013 force-pushed the james-doc branch 6 times, most recently from 1b965ae to 1e6610d Compare July 8, 2025 04:53

jjqq2013 mentioned this pull request Jul 8, 2025

What is the execution order of substeps? #2295

Open

jjqq2013 changed the title ~~Fix typos, add docs about execution order of Cloud-init stages and steps~~ Fix typos, add docs about execution order of Cloud-init files, steps, and substeps Jul 8, 2025

jjqq2013 force-pushed the james-doc branch from 1e6610d to 530f55d Compare July 8, 2025 04:55

jjqq2013 changed the title ~~Fix typos, add docs about execution order of Cloud-init files, steps, and substeps~~ Fix typos. Add docs about execution order of Cloud-init files, steps, and substeps Jul 8, 2025

jjqq2013 force-pushed the james-doc branch from 530f55d to 0369d4b Compare July 8, 2025 06:07

jjqq2013 force-pushed the james-doc branch from 0369d4b to 37b2bfc Compare July 22, 2025 04:42

jjqq2013 changed the title ~~Fix typos. Add docs about execution order of Cloud-init files, steps, and substeps~~ Add docs about execution order of Cloud-init files, steps, and substeps Jul 22, 2025

jjqq2013 mentioned this pull request Jul 29, 2025

[BUG] A fundamental unbelievable issue about execution order of /oem/*.yaml harvester/harvester#8759

Open

m-ildefons reviewed Jul 29, 2025

View reviewed changes

jjqq2013 force-pushed the james-doc branch 3 times, most recently from 0fa17c0 to 9f21fa8 Compare August 6, 2025 07:23

jjqq2013 commented Aug 6, 2025

View reviewed changes

frelon reviewed Aug 6, 2025

View reviewed changes

jjqq2013 force-pushed the james-doc branch from 9f21fa8 to ea272e6 Compare August 6, 2025 07:36

jjqq2013 commented Aug 6, 2025

View reviewed changes

jjqq2013 force-pushed the james-doc branch 3 times, most recently from b58c393 to 05167d0 Compare August 6, 2025 07:49

jjqq2013 commented Aug 6, 2025

View reviewed changes

frelon reviewed Aug 6, 2025

View reviewed changes

docs/content/en/docs/Reference/cloud_init.md Outdated Show resolved Hide resolved

jjqq2013 force-pushed the james-doc branch from 05167d0 to 8cbc028 Compare August 6, 2025 16:45

Fix typos. Add docs about execution order of Cloud-init files, steps,…

e818b90

… and substeps Signed-off-by: jjqq2013 <[email protected]>

jjqq2013 force-pushed the james-doc branch from 8cbc028 to e818b90 Compare August 6, 2025 16:49

		Therefore, when you have multiple `substeps` that related to each other, please do not define them in a single `step`.
		Instead, you can split the `substeps` into multiple `steps` which are ensured to be executed by the definition order.

Add docs about execution order of Cloud-init files, steps, and substeps #2294

Are you sure you want to change the base?

Add docs about execution order of Cloud-init files, steps, and substeps #2294

Uh oh!

Conversation

jjqq2013 commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjqq2013 Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjqq2013 Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjqq2013 Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjqq2013 Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjqq2013 commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjqq2013 commented Aug 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

frelon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjqq2013 commented Aug 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jjqq2013 commented Jul 8, 2025 •

edited

Loading

jjqq2013 Aug 6, 2025 •

edited

Loading

jjqq2013 Aug 6, 2025 •

edited

Loading

jjqq2013 Aug 6, 2025 •

edited

Loading

jjqq2013 Aug 6, 2025 •

edited

Loading

jjqq2013 commented Aug 6, 2025 •

edited

Loading