fix(agent): separate download and upload paths for agent package transfer - #1166
fix(agent): separate download and upload paths for agent package transfer#1166aewtemp wants to merge 37 commits into
Conversation
Update changelogs and docs upon release
Documentation update and build-related changes
Release 7.2.2
Update changelogs and docs upon release
Bumps [black](https://github.com/psf/black) from 26.3.0 to 26.3.1. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](psf/black@26.3.0...26.3.1) --- updated-dependencies: - dependency-name: black dependency-version: 26.3.1 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com>
Bump black from 26.3.0 to 26.3.1
Release 7.2.3
Update changelogs and docs upon release
Update testing for Checkmk 2.5.
NORELEASE - Bump Checkmk versions and dependencies
NORELEASE - Bump Checkmk versions and dependencies#1063
Release 7.3.0
Update changelogs and docs upon release
Release 7.3.1
Update changelogs and docs upon release
Release 7.3.2
Update changelogs and docs upon release
Release 7.4.0
Update changelogs and docs upon release
Release 7.5.0
Update changelogs and docs upon release
Release 7.6.0
Update changelogs and docs upon release
No Release: Fix Ansible tests, bump Checkmk versions and update build dependencies
Release 7.7.0
Update changelogs and docs upon release
Release 7.8.0
Update changelogs and docs upon release
Release 8.0.0
Update changelogs and docs upon release
Release 8.1.0
Update changelogs and docs upon release
Release 8.2.0
Update changelogs and docs upon release
Release 8.3.0
Update changelogs and docs upon release
|
I have read the CLA Document and I hereby sign the CLA or my organization already has a signed CLA. 1 out of 2 committers have signed the CLA. |
…sfer Fixes Checkmk#741 The `win_copy` and `copy` modules always expect their source file to be located either on the Ansible controller, or on the target host itself when `remote_src: true` is set. They cannot transparently bridge a file between two different remote hosts. Previously, a single path variable (`__checkmk_agent_agent.file.*`) was used both as the destination for the initial package download (which may run on a delegated host, e.g. `localhost`, depending on `checkmk_agent_delegate_download`) and as the source/destination for the subsequent copy task onto the actual target host. When the download was delegated to a host different from the target, this caused the copy task to fail, since the file did not exist at the expected location on the target. This change splits the previously shared path into two explicit, independent path sets: - `__checkmk_agent_agent.file.download.*`: where the package is downloaded to (on the delegated/download host). - `__checkmk_agent_agent.file.upload.*`: where the package is copied/uploaded to on the target host, and subsequently installed from. All download tasks (`get_url`/`win_get_url` via `checkmk_agent_delegate_download`) now write to `file.download.*`, while the `copy`/`win_copy` tasks explicitly use `file.download.*` as `src` and `file.upload.*` as `dest`. Installation tasks (`package`, `zypper`, `win_package`) now consistently reference `file.upload.*`, since that is the final location on the target host. Additionally: - `checkmk_agent_delegate_download` is changed from a hostname-based variable (defaulting to `inventory_hostname`) to a boolean, making the delegation intent explicit rather than implicit through hostname comparison. - `__checkmk_agent_host_tmp_dir` (used for the download destination) is now defined centrally in `vars/main.yml`, while OS-specific `__checkmk_agent_lin_tmp_dir` / `__checkmk_agent_win_tmp_dir` variables are introduced for the upload destination on Linux and Windows targets respectively. Signed-off-by: Alexander Temp <alexander.temp@sit.nrw>
|
@aewtemp the name/email you made your commits with, does not match the one in your GitHub account. Can you either add the name or make the commit with a name, that matches yours? Worst case you need to copy the files and create a new commit and PR. Let me know, if I can help with that. We do appreciate your contribution though! I am currently reviewing, so if you need to re-create the PR, maybe wait for my feedback to land. |
robin-checkmk
left a comment
There was a problem hiding this comment.
I like the idea of splitting the file path into two sets:
__checkmk_agent_agent.file.download.*: where the package is downloaded to (on the delegated/download host). __checkmk_agent_agent.file.upload.*: where the package is copied/uploaded to on the target host, and subsequently installed from.
What I have doubts and concerns about:
checkmk_agent_delegate_download is changed from a hostname-based variable (defaulting to inventory_hostname) to a boolean, making the delegation intent explicit rather than implicit through hostname comparison.
This is a breaking change, which will affect every single user, who uses the delegation to this day. And while I understand the idea, I would ask you to revert to the previous behavior. Also consider my more detailed comments on this matter.
__checkmk_agent_host_tmp_dir (used for the download destination) is now defined centrally in vars/main.yml, while OS-specific __checkmk_agent_lin_tmp_dir / __checkmk_agent_win_tmp_dir variables are introduced for the upload destination on Linux and Windows targets respectively.
I do not see merrit in this change, why is it necessary?
|
|
||
| ## Delegation | ||
| checkmk_agent_delegate_api_calls: 'localhost' | ||
| checkmk_agent_delegate_download: "{{ inventory_hostname }}" |
There was a problem hiding this comment.
This is a breaking change. Can we somehow avoid it?
Coming back to this after reviewing further: I think we can avoid this, as the conditionals you introduced further down are unnecessary, if you keep this filed reserved for the hostname to delegate to.
There was a problem hiding this comment.
The problem here is, that this configuration option leaves the impression one can randomly choose a delegate to have the files downloaded and then copied from to the target host.
This violates ansibles own assumption for the modules ansible.builtin.copy and ansible.windows.win_copy, whose expectations are that the file to copy has to reside either on the controller or on the target host.
One can only download files directly to the target host, or the controller, to then be able to copy the files to their destination.
So I don't think it would be good to leave the logic behind the delegation as is.
There was a problem hiding this comment.
I think I see your point and I remember considering the option to actually delegate things to a third party host, so it can act as an intermediary. But I am still very hesitent to introduce such a breaking change. Especially, as the issue you describe has not surficed yet. I would rather extend on documentation, instead of an unnecessary breaking change.
There was a problem hiding this comment.
I fully understand your point.
But hear me out: Just because nobody has encountered the error yet, doesn't mean it's not there. And I am the first person then to have hit that problem.
I am targeting 1700+ hosts with this Playbook. Windows and Linux hosts. I tried delegating the operations for Windows hosts to a third (not controller nor target) Windows machine. It fails.
This is my approach to fix this misconception and the resulting error.
To me this wont break any setup already using this collection in their setups. Because they have not had this error yet.
There is no way this would be working if you choose a third host as delegate for the download and copy operation. Try it yourself.
Besides that Problem, if the controller is a Linux host, you only will be able to transfer files to Linux hosts. The current state does not work when cross-transferring files from Linux to Windows and vice versa.
This will also be fixed with my approach.
I'll fix and commit the outstanding things.
I would appreciate it if you could reconsider this once more.
| - name: "Preflight - Make sure facts for the delegate Host (Controller or Target Host) are present." | ||
| ansible.builtin.setup: | ||
| delegate_to: "{{ 'localhost' if checkmk_agent_delegate_download else inventory_hostname }}" | ||
| delegate_facts: true | ||
| tags: | ||
| - download-package | ||
|
|
||
| - name: "Preflight - Set facts for the delegate Host (Controller or target Host)" | ||
| ansible.builtin.set_fact: | ||
| __checkmk_agent_delegate_is_windows: >- | ||
| {{ hostvars['localhost']['ansible_facts']['os_family'] == 'Windows' }} | ||
| tags: | ||
| - download-package | ||
|
|
||
| - name: "{{ ansible_facts['system'] }}: Include agent package download tasks (Linux)." | ||
| ansible.builtin.include_tasks: "Linux-files.yml" | ||
| when: not __checkmk_agent_delegate_is_windows | ||
| tags: | ||
| - include-system-tasks | ||
| - download-package | ||
|
|
||
| - name: "{{ ansible_facts['system'] }}: Include agent package download tasks (Windows)." | ||
| ansible.builtin.include_tasks: "Win32NT-files.yml" | ||
| when: __checkmk_agent_delegate_is_windows |
There was a problem hiding this comment.
Why is this necessary? This feels awefully complex and seems to do the same as the original code.
There was a problem hiding this comment.
I just came to notice that I left the original inclusion of os_family specific variables above this block.
I'll check if it works without explicitly loading the needed variables for the controller host.
Fixes #741
The
win_copyandcopymodules always expect their source file to be located either on the Ansible controller, or on the target host itself whenremote_src: trueis set. They cannot transparently bridge a file between two different remote hosts.Previously, a single path variable (
__checkmk_agent_agent.file.*) was used both as the destination for the initial package download (which may run on a delegated host, e.g.localhost, depending oncheckmk_agent_delegate_download) and as the source/destination for the subsequent copy task onto the actual target host. When the download was delegated to a host different from the target, this caused the copy task to fail, since the file did not exist at the expected location on the target.This change splits the previously shared path into two explicit, independent path sets:
__checkmk_agent_agent.file.download.*: where the package is downloaded to (on the delegated/download host).__checkmk_agent_agent.file.upload.*: where the package is copied/uploaded to on the target host, and subsequently installed from.All download tasks (
get_url/win_get_urlviacheckmk_agent_delegate_download) now write tofile.download.*, while thecopy/win_copytasks explicitly usefile.download.*assrcandfile.upload.*asdest. Installation tasks (package,zypper,win_package) now consistently referencefile.upload.*, since that is the final location on the target host.Additionally:
checkmk_agent_delegate_downloadis changed from a hostname-based variable (defaulting toinventory_hostname) to a boolean, making the delegation intent explicit rather than implicit through hostname comparison.__checkmk_agent_host_tmp_dir(used for the download destination) is now defined centrally invars/main.yml, while OS-specific__checkmk_agent_lin_tmp_dir/__checkmk_agent_win_tmp_dirvariables are introduced for the upload destination on Linux and Windows targets respectively.Pull request type
Check the type of change your PR introduces:
What is the current behavior?
The role "agent" currently is build upon the belief that setting a target_name on "checkmk_agent_delegate_download" would delegate the download and copy transaction of the agent to that host. But the module "windows.win_copy" and "builtin.copy" expect the file to be copied either on the controller or the target host.
Issue Number: #741
What is the new behavior?
The changes introduced by this request allow the controller to either be linux or windows based Operating systems.
The "checkmk_agent_delegate_download" variable was changed to a boolean. If true, the download of the file will be handled by the controller. Otherwise the download will be done on the target.
Other information