[WIP] Add two roles for Azure ARC Onboarding

[Klaas-]

SUMMARY

As prep for doing more arc stuff in ansible we need to be able to setup arc agent on non-azure servers.
Best way to do this should be by using the packages.microsoft.com repository and then connecting using a token.
This is my first idea of how to do this, locally I am only using it on RHEL, so this is naturally the first OS I implemented.

ISSUE TYPE

• New Role Pull Request

COMPONENT NAME

roles/azure_arc
roles/azure_repo_microsoft_prod

ADDITIONAL INFORMATION

No tests, just sharing my first idea

[p3ck]

This is great! BTW - I've been working on a role to do this as well.

https://github.com/p3ck/cloud.azure_ops/blob/configure_archost/playbooks/ARCHOST.md

The pull request is here: redhat-cop/cloud.azure_ops#111

I can see some improvements I can make from your work and vice versa..

[p3ck]

I updated my PR to include a lot of the updates from your version. (I added you to the author list)

I guess the question is does Azure/Microsoft want roles added to this collection? Or should we continue to expand the cloud.azure_ops repo?

[jeremycline]

I updated my PR to include a lot of the updates from your version. (I added you to the author list)

I guess the question is does Azure/Microsoft want roles added to this collection? Or should we continue to expand the cloud.azure_ops repo?

Hi! I have been chatting with @alt-key and @stefanoochoa from the Azure Arc team about adding roles to this collection for Arc, so this is great timing.

I've done some light contributions to this collection, but I'm not an expert by any stretch of the imagination, so out of curiosity why might we not want to include this in the Azure collection? I've not run across https://github.com/redhat-cop/cloud.azure_ops before, how is it intended to differ from the roles in this collection?

Also, I (and I imagine the Arc folks as well) are happy to help test this out and expand support to additional distros.

[Klaas-]

@jeremycline maybe you can start by getting proper docs on the arc agent, some assumptions I made in this PR are unverified because of lack of documentation:

azure/roles/azure_arc/tasks/main.yml

Lines 52 to 53 in 4ecb6af

	# It is currently unclear if this is a valid indicator for "azure arc registration is already done" a query about this is open towards the ARC team
	when: azure_arc_show.status != 'Connected'

is the prime example :)

Other examples are exit codes, they are currently undocumented.

azure/roles/azure_arc/tasks/main.yml

Line 44 in 4ecb6af

changed_when: azure_arc_proxy_set.rc == 0

Another thing you could help with is

azure/roles/azure_arc/tasks/main.yml

Lines 55 to 63 in 4ecb6af

	- name: Get a token using the ARC service principal
	# Run on localhost, so that service principal secrets don't leave control node
	delegate_to: localhost
	# This command produces output that includes an access token
	no_log: true
	azure.azcollection.azure_rm_accesstoken_info:
	scopes:
	- https://management.azure.com/.default
	register: azure_arc_access_token

Currently there is no way to issue single-use-arc-onboard-only tokens. From a security perspective it's a bad idea to give even a short lived secret to a target node. The best idea is to create a service principal that is only allowed to onboard arc systems. But even then giving a 60 minute token is not as good as doing single use tokens

Another thing you can help with is provide resources to CI test ARC modules. I've started with #2065 - but at the moment you can't test it in CI because there are no ARC machines there.
Also have a look at the module, there are several API side issues documented in there, tags don't update on create or update API call or autoUpgradeMinorVersion does not work
( https://learn.microsoft.com/en-us/rest/api/hybridcompute/machine-extensions/create-or-update?view=rest-hybridcompute-2025-01-13&tabs=HTTP )

I have several issues open with MSFT support about the API, and I only really looked at it for like a day ... :)

[Klaas-]

I've incorporated a lot of the changes @p3ck made, this should be working, if you guys get a non-azure system into CI I'll also write some integration tests :)

[stefanoochoa]

Hi @Klaas-! I'm Stefano, an engineer on the Azure Arc team. Firstly, thank you for kicking of support for Arc onboarding in this collection; this is really exciting to see and super helpful for Arc users.

We've been reviewing your PR internally and had a few ideas/suggestions around how the installation of the Arc agent could be handled.

We're wondering if you're open to using the install script which we publish (https://aka.ms/azcmagent) as the mechanism for configuring the packages.microsoft.com repo and installing the Arc agent package. A couple reasons this might be beneficial:

• This script already handles arc agent installation for all distros Azure Arc support, which could simplify the role and remove the need for any distro-specific logic.
• This approach aligns with how we guide installation in other scenarios (e.g. Azure Portal), so it helps keep the experience consistent for users.

Curious what you think about this. We're absolutely open to collaborating; I'm happy to discuss details, open a design-discussion issue if that helps, and contribute changes to your branch if you're comfortable with that.

Thanks again for getting this work started!

[Klaas-]

Hi @Klaas-! I'm Stefano, an engineer on the Azure Arc team. Firstly, thank you for kicking of support for Arc onboarding in this collection; this is really exciting to see and super helpful for Arc users.

We've been reviewing your PR internally and had a few ideas/suggestions around how the installation of the Arc agent could be handled.

We're wondering if you're open to using the install script which we publish (https://aka.ms/azcmagent) as the mechanism for configuring the packages.microsoft.com repo and installing the Arc agent package. A couple reasons this might be beneficial:

* This script already handles arc agent installation for all distros Azure Arc support, which could simplify the role and remove the need for any distro-specific logic.

I think you (as in MSFT generally) should stop doing that and get this properly standardized ( this is issue microsoft/linux-package-repositories#247 -- maybe put some internal pressure there to get this rolling).

* This approach aligns with how we guide installation in other scenarios (e.g. Azure Portal), so it helps keep the experience consistent for users.

Yeah, I agree. The solution should always be "setup repository" and then just "${package-manger} install packagename".
By briefly looking at your script I believe this is basically what you are doing.
I noticed a failure immediately though:

function check_physical_memory {
    size=$(grep MemTotal /proc/meminfo | tr -s ' ' | cut -d ' ' -f2)
    unit=$(grep MemTotal /proc/meminfo | tr -s ' ' | cut -d ' ' -f3)
    if [ $unit == "kB" ]; then
        echo "Total physical memory: ${size} ${unit}"
    fi
}

This does not actually show physical memory. That's also wrong in the azure arc implementation though ... :) I have complained about that in past :) Also I would say you should not do that here, but rather inside your packages. There may always be situations where this script is not used (updates for example if you ever change your requirements). Speaking for RPM based installations, I'd guess you can check for this in a rpm prescript, not sure if it's a really good idea though :)

Curious what you think about this. We're absolutely open to collaborating; I'm happy to discuss details, open a design-discussion issue if that helps, and contribute changes to your branch if you're comfortable with that.

I am very open for collaboration here. The first step we need for this would be a way to test it. Means you should talk with the MSFT people/contractors that manage the ansible azure-collection on how to get this into the CI pipeline and then we can write some tests for it.

Thanks again for getting this work started!

[stefanoochoa]

The first step we need for this would be a way to test it

I went through the documentation in CONTRIBUTING.md and was able to put together a simple integration test.

Would it be alright if I raised a PR to your branch to add the integration test?

The solution should always be "setup repository" and then just "${package-manger} install packagename"

Agreed, that is absolutely the ideal state.

In practice, one of the challenges we must balance is the breadth of platforms Arc supports and the overhead of keeping that support matrix correctly implemented and in sync across multiple entry points.

That's where our install script comes in today; it centralizes the "setup repo & install package" logic for all Arc-supported OSes so that each consumer (e.g Azure Portal, Ansible) doesn't need to re-encode distro and version-specific behavior.

Conceptually, it's still doing exactly what you described, "setup repo & install package", just with the OS detection and branching abstracted away into one place.

Happy to iterate on what the right balance looks like here once we have tests in place and can reason about the behavior more concretely.