external-dns annotations to support dynamic defined --ingress-class and --aws-zone-type

[rcazzatoApk]

What would you like to be added:
To set a Split-Brain DNS (Split-Horizon) solution for AWS EKS / Route53 Public/Private Hosted Zones I deployed 2 external-dns instances:

• one using ingress-class = public (internet-facing ingress controller) and records in Public Hosted Zone (--ingress-class=internet-facing, --aws-zone-type=public)

• one using ingress-class = private (internal ingress controller) and records in Private Hosted Zone (--ingress-class=internal, --aws-zone-type=private)

It works and this is an acceptable solution even if 2 external-dns instances must be managed.

For some cases though I need to set some public IPs (internet-facing ingress-class) in the private hosted zone. As I know the only viable solution should be to deploy another external-dns instance with mixed: --ingress-class=internet-facing and --aws-zone-type=private but specific zone (domains) filters should be adopted to avoid race condition on zones management and also I will have another instance to manage.

Can external-dns be enhanced to support "ingress annotations" to introduce more flexible use of "--ingress-class" and " --aws-zone-type" per ingress instead of for external-dns instance ?

Why is this needed:
Less external-dns instances deployment an management.
More flexibility with configuration tied to ingress/service settings instead of external-dns instance wide setting.

[ivankatliarchuk]

Share your kubernetes manifests with spec section included, and what outcome is expected.

[rcazzatoApk]

Below container args for an instance (I call external-dns-public) managing DNS records for AWS route53 Public Hosted Zone with an internet facing ingress-class and that will map hostname "abc.example.com" to "Public IPs". This also allow Let's Encrypt HTTP-01 challenge to work (cert-manager deployment) and set a kubernetes secret with valid "certificate" for hostname "abc.example.com":

containers:
        - args:
            - '--policy=sync'
            - '--provider=aws'
            - '--registry=txt'
            - '--interval=1m'
            - '--txt-owner-id=...'
            - '--txt-prefix=extdns-'
            - '--ingress-class=haproxy-public'
            - '--source=service'
            - '--source=ingress'
            - '--aws-api-retries=3'
            - '--aws-zone-type=public'
            - '--aws-batch-change-size=1000'
            - '--aws-prefer-cname'

Below container args for another instance (I call external-dns-internal) managing DNS records for AWS route53 Private Hosted Zone with an internal ingress-class to map SAME hostname "abc.example.com" to "Private IPs". This allow "DNS Split Brain" (also named "DNS Split Horizon") solution and is great.
With specific configuration on AWS Private Hosted Zone (nearest DNS Server resolving query) will be resolved first (and completely mask out) the Public one with same domain (Authoritative DNS Server) and for certificate side, internal ingress can be configured to use kubernetes secret created and maintained by public ingress with let's encrypt):

containers:
        - args:
            - '--policy=sync'
            - '--provider=aws'
            - '--registry=txt'
            - '--interval=1m'
            - '--txt-owner-id=...'
            - '--txt-prefix=extdns-'
            - '--ingress-class=haproxy-internal'
            - '--source=service'
            - '--source=ingress'
            - '--aws-api-retries=3'
            - '--aws-zone-type=private'
            - '--aws-batch-change-size=1000'
            - '--aws-prefer-cname'

As I know, with external-dns there is no configuration based on service/ingress level (e.g. annotation) that will drive Hostname/IP mapping so I need 2 external-dns completely separate instances to achieve Split Brain solution (one with "--ingress-class=haproxy-public" and "--aws-zone-type=public" and another with "--ingress-class=haproxy-internal" and "--aws-zone-type=private"). See also mappings like example below:

1. Ingress A with records created in AWS Public hosted zone using ingress-public (pub hosted zone = abc.example.com => Public IPs)
2. Ingress B with records created in AWS Public hosted zone using ingress-internal (pub hosted zone = abc.example.com => Private IPs)
3. Ingress C with records created in AWS Private Hosted Zone using ingress-internal (pri hosted zone = abc.example.com => Ptivate IP)
4. Ingress D with records created in AWS Private Hosted Zone using ingress-public (pri hosted zone = abc.example.com => Public IP)

As per Split Brain private hosted zone (or in general nearest DNS Server resolving DNS query) mask out completely public one (or in general Authoritative DNS Server) for the same domain, sometimes cases can arise to use cases 2 or 4 so, service/ingress level annotations to combine what "--ingress-class" IPs (e.g. AWS NLB internal/public IPs) are used for specific --aws-zone-type" (internal/public), if feasible, could allow more flexibility and a single external-dns instance.