Add --all-projects flag to support cross-project zone management via X-Auth-All-Projects header

[abhijith-darshan]

Problem

When the Designate service user is scoped to a project that does not own the DNS zones (e.g. scoped to project-a but zones live in project-b), the webhook returns 0 zones and cannot manage any records.

This is a common OpenStack setup where DNS zones are owned by a dedicated admin project, while service users authenticate against a different project.

Expected Behaviour

The webhook should support an --all-projects flag that sets the X-Auth-All-Projects: true header on all Designate API requests, allowing the service user to see and manage zones across all projects.

Proposed Implementation

In internal/designate/client/client.go, add the header when constructing the service client:

func NewDesignateClient(allProjects bool) (DesignateClientInterface, error) {
    serviceClient, err := createDesignateServiceClient()
    if err != nil {
        return nil, err
    }
    if allProjects {
        serviceClient.MoreHeaders = map[string]string{
            "X-Auth-All-Projects": "true",
        }
    }
    return &designateClient{serviceClient}, nil
}

In cmd/webhook/main.go, expose the flag:

pflag.BoolVar(&allProjects, "all-projects", false, "Manage all zones and recordsets regardless of the project the service user resides in")

[frittentheke]

@abhijith-darshan using all-projects implies an admin user scoped token right?
Why not add the use of x-auth-sudo-project-id to just select project the zone(s) live in?

I don't even know if the management of records would work later, even if the zone was listed due to all-projects=true.

For reference, this is how the OpenStackSDK implemented the corresponding options: openstack/openstacksdk@e7bdb02

Edit: Additionally I'd argue that using a properly scoped application credential token would actually be the "right" solution?

[abhijith-darshan]

@abhijith-darshan using all-projects implies an admin user scoped token right? Why not add the use of x-auth-sudo-project-id to just select project the zone(s) live in?

I don't even know if the management of records would work later, even if the zone was listed due to all-projects=true.

For reference, this is how the OpenStackSDK implemented the corresponding options: openstack/openstacksdk@e7bdb02

Edit: Additionally I'd argue that using a properly scoped application credential token would actually be the "right" solution?

@frittentheke thank you for the quick response.

Yes all-projects indicates admin scoped token, app credentials are by default scoped to a project and this won't be an issue.

However, there are setups where service users are provisioned centrally via automation with an admin-scoped token, needing to manage zones across all projects. We at SAP have a similar setup with a custom controller and are looking to migrate to external-dns + this webhook.

Would you be willing to consider this implementation #85?

I'm happy to document the consequences and caveats around X-Auth-All-Projects usage.

I have tested this in our environment and it works but I can also look further into x-auth-sudo-project-id as well, if it satisfies our use case

[abhijith-darshan]

also combining it with #84 will greatly improve performance

[abhijith-darshan]

x-auth-sudo-project-id will only manage for a single project but we would need to manage DNS entries in multiple projects and zones

[frittentheke]

x-auth-sudo-project-id will only manage for a single project but we would need to manage DNS entries in multiple projects and zone

And all this with a single instance of external-dns and this webhook then?
Would this then work with the current state the webhook is in? All the API calls to e.g. add records would need to be using the all-projects header, see https://docs.openstack.org/api-ref/dns/dns-api-v2-index.html#id87.

While this would just allow what OpenStack allows ... it still feels wrong to encourage these kind of configurations.
If we were talking about e.g. domain scoped tokens (https://docs.openstack.org/keystone/latest/admin/tokens-overview.html#domain-scoped-tokens) it might make sense ... but OpenStack cloud-wide access to all zones given to a single external-dns instance seems kinda wrong.

[abhijith-darshan]

x-auth-sudo-project-id will only manage for a single project but we would need to manage DNS entries in multiple projects and zone

And all this with a single instance of external-dns and this webhook then? Would this then work with the current state the webhook is in? All the API calls to e.g. add records would need to be using the all-projects header, see https://docs.openstack.org/api-ref/dns/dns-api-v2-index.html#id87.

While this would just allow what OpenStack allows ... it still feels wrong to encourage these kind of configurations. If we were talking about e.g. domain scoped tokens (https://docs.openstack.org/keystone/latest/admin/tokens-overview.html#domain-scoped-tokens) it might make sense ... but OpenStack cloud-wide access to all zones given to a single external-dns instance seems kinda wrong.

I understand and I am not disputing this at all.

At SAP we have a special Admin tenant / domain which needs this kind of a setup only in the admin domain controlled by the infrastructure team. It would be really useful for us to use this. This would never work in a normal domain + projects...

We would very much appreciate the support on this 🙏

[abhijith-darshan]

@frittentheke any thoughts on my request?

[frittentheke]

Sorry @abhijith-darshan about the delay.
In the end I agree more flexibility in using this webhook does not hurt.

But I was just wondering if Gophercloud would not be the better place to handle the "all projects"
switch and injection of the header? Or if it doesn't already have this capability?

Why use an SDK if I am still adding various headers instead of having them abstracted away?
see e.g. https://github.com/gophercloud/gophercloud/blob/315483ea3e4905ae2b62786327963e93c795a43f/openstack/dns/v2/zones/requests.go#L181-L216

[abhijith-darshan]

@frittentheke thank you for the suggestion. I think gophercloud's DNS support for AllProjects is only available in ListSharesOpts (for the zone shares endpoint), not in ListOpts for zones.List().

Recordset operations also have no equivalent abstraction.

Zone shares would require the zone owner to explicitly share zones with each project beforehand, which doesn't fit the admin tenant use case.

I could open an issue there but in the meantime, would you consider going with MoreHeaders as a pragmatic solution? I can actively maintain this part and once there is capability from gopher cloud, I would contribute the switch.

@kayrus is also helping out in fixing the clouds.yaml coalesce issue here as well - gophercloud/gophercloud#3738 which can be used to handle cross project tech user auth in this webhook

[frittentheke]

I could open an issue there but in the meantime, would you consider going with MoreHeaders as a pragmatic solution? I can actively maintain this part and once there is capability from gopher cloud, I would contribute the switch.

@abhijith-darshan Please raise and issue or create a PR to add this capability Gophercloud. I am all for pragmatism, but also for fixing things in the right place, to the benefit of all using the Gophercloud SDK.
With your immediate contact to a dev and the potential to bring this small (while already existing elsewhere ) feature in the next release of Gophercloud, all that is left is bumping the dependency in this webhook.

But I gladly accept your "--all-projects" feature as proposed here once the situation around Gophercloud is clear / resolved.

[frittentheke]

@abhijith-darshan did you you create an issue with Gophercloud yet? If so please don't miss linking it here (or reference this issue there).

I'd really like to go forward with the cross-project capability for this external-dns webhook, but again I'd rather see this be something happening in the SDK.

[abhijith-darshan]

@frittentheke not yet… will do it

[frittentheke]

@frittentheke not yet… will do it

@abhijith-darshan any update?

[abhijith-darshan]

Hey @frittentheke - I have created the issue and also opening a PR there
gophercloud/gophercloud#3800