Interactive onboarding command: dnscontrol init

[cafferata]

I briefly ran this idea past @TomOnTime today in a short video call and mentioned that an upcoming holiday period should give me time to pick this work back up. Before going further I want to surface it here so other maintainers and long time contributors can weigh in. Feedback on scope, UX, the metadata schema and naming is very welcome.

In my own experience helping developers try DNSControl (and from my own start back in November 2023), every newcomer hits the same wall:
how and where do I start? @Aartsie was one of those developers, and like most newcomers they arrived with:

I want to import my existing DNS records and domains into DNSControl so I can experience its real strength: DNS as code, reviewable diffs, safe previews before anything goes live, a full change history in git and the freedom to switch providers later without rewriting my zones. Help me get there quickly.

Today that path involves reading the docs, tracing the provider Go source for the exact field names, hand writing
creds.json and on top of that building a
dnsconfig.js from scratch while having to learn an entirely new JavaScript DSL. This proposal is about removing that cold start.

Pinging the maintainers listed in
OWNERS. Most of you look after these providers as fellow users rather than on behalf of the vendor, so a smoother onboarding directly helps the people who come after you. Feedback and pushback both welcome:

@arnoschoon @atrull @blackshadev @bytemain @chicks-net @costasd @D3luxee @das7pad @dnscale-ops @e-im @edglynes @hamptonmoore @hedger @hnrgrgr @huihuimoe @imlonghao @ishanjain28 @jbelien @jpbede @juliusrickert @KaiSchwarz-cnic @kallsyms @KlettIT @koesie10 @kordianbruck @masterzen @matthewmgamble @mikenz @MisterErwin @onlyhavecans @patschi @pgaskin @pierre-emmanuelJ @PJEilers @ppmathis @pragmaton @rblenkinsopp @riku22 @riyadhalnur @SimenBai @SphericalKat @SukkaW @systemcrash @tlimoncelli @tomfitzhenry @tresni @ttkzw @vatsalyagoel @vojtad @willpower232 @xddxdd @zupolgec

---

Context

A new user of DNSControl has to hand write creds.json before they can run preview or
push. The information needed to fill that file correctly is scattered across at least three places:

• The provider doc page in documentation/provider/
• The provider Go source where field names are read
• .github/workflows/pr_integration_tests.yml where CI credentials are mapped

Field names are inconsistent across providers (apitoken, api_token, KeyId,
AccountName, and so on) and the portal URL where you create an API token is not discoverable from the CLI. For a first time user that means a round trip through the docs before they can get to
preview.

Proposal

Add an interactive command:

dnscontrol init

It walks the user through:

1. Pick a DNS provider (NONE to defer).
2. If the DNS provider also acts as a registrar, offer to reuse the same account. Otherwise pick a registrar (
   NONE is available).
3. Open the provider's API settings page in the default browser on request.
4. Prompt per credential field, with password masking for secrets and $EDITOR for multi line values (PEM blocks).
5. Ask for one or more domains for a starter dnsconfig.js.
6. Optionally call
   dnscontrol get-zones --format=nameonly and show which configured domains exist at the provider, which are only in the config and which are only at the provider.
7. Optionally run dnscontrol preview as a final sanity check.

Driving it from the provider

To keep the onboarding data next to the provider code, each provider would register its metadata from the same
init() where it already calls
RegisterMaintainer. The TransIP registration is a good showcase because it covers the expressive end of the API (two auth methods, a multi line PEM field and a second URL for domain management):

providers.RegisterCredsMetadata("TRANSIP", providers.CredsMetadata{
    DisplayName: "TransIP",
    Kind:        providers.KindDNS,
    DocsURL:     "https://docs.dnscontrol.org/provider/transip",
    PortalURL:   "https://www.transip.nl/cp/account/api/",
    DomainsURL:  "https://www.transip.nl/cp/domein-hosting/",
    Notes:       "TransIP supports two auth methods: a short lived access token, or an account name paired with a long lived private key.",
    Fields: []providers.CredsField{
        {
            Key:      "_authMethod",
            Label:    "Which authentication method do you want to use?",
            Choices:  []string{"Access token", "Account name + private key"},
            Required: true,
            Internal: true,
        },
        {
            Key:      "AccessToken",
            Label:    "Access token",
            Required: true,
            Secret:   true,
            ShowIf:   map[string]string{"_authMethod": "Access token"},
        },
        {
            Key:      "AccountName",
            Label:    "Account name",
            Required: true,
            ShowIf:   map[string]string{"_authMethod": "Account name + private key"},
        },
        {
            Key:       "PrivateKey",
            Label:     "Private key (opens $EDITOR)",
            Required:  true,
            Multiline: true,
            ShowIf:    map[string]string{"_authMethod": "Account name + private key"},
        },
    },
})

Fields can declare Required, Secret, Multiline, Choices, EnvVar, Default, plus an Internal selector with
ShowIf branching. That lets providers with multiple auth methods (TransIP as shown above, or Route53 static keys versus assume role) ask the right question first and only prompt for the relevant fields. Simpler providers reduce to just
PortalURL and a few Fields entries.

Providers without registered metadata still work: the init flow falls back to a generic key/value loop and prints the doc URL.

Sample transcript: Cloudflare with a single account

? Which DNS service provider do you want to configure? CLOUDFLAREAPI
? Use the same Cloudflare account as your registrar? Yes

== DNS provider: Cloudflare ==

API settings for Cloudflare: https://dash.cloudflare.com/profile/api-tokens
? Open the API settings page in your browser? Yes
? API Token (required) **********
? Account ID (optional) 0123456789abcdef
? creds.json entry name for this provider cloudflare
? First domain name for dnsconfig.js example.com
? Write these files? Yes

Done.
? Compare domains in dnsconfig.js with zones at Cloudflare? Yes

Zones at Cloudflare compared with dnsconfig.js:
  In both          : example.com
  Only in config   : (none)
  Only at provider : other.com

? Run `dnscontrol preview` now? Yes

Sample transcript: TransIP with two auth methods

? Which DNS service provider do you want to configure? TRANSIP

== DNS provider: TransIP ==

API settings for TransIP: https://www.transip.nl/cp/account/api/
Manage domains at TransIP: https://www.transip.nl/cp/domein-hosting/
TransIP supports two auth methods: a short lived access token, or an
account name paired with a long lived private key.
? Which authentication method do you want to use? Account name + private key
? Account name my-account
? Private key (opens $EDITOR) [Enter to launch editor]
...

Open questions

1. Rollout: a generic fallback means metadata can land per provider in separate PRs. Would a tracking issue with a checkbox per provider help maintainers claim theirs?
2. Documentation: should the same metadata drive a generated "credentials reference" page in
   documentation/? Or do we remove this information from the per provider doc pages entirely and make
   init (plus the metadata registered in Go) the single source of truth?

I have a working draft locally but would like to gauge interest and catch pushback before opening a PR. Feedback on the metadata schema, UX and rollout is very welcome.

interactive-onboarding-command-dnscontrol-init-demo.mp4

[blackshadev]

Great idea, any idea to lower the bar of entry to dnscontrol is a good idea in my books.

Some feedback about the general idea

Regarding the implementation, I think your approach is very flexibel, all be it maybe a bit heavy on the core side. The fact that you can have choices and dependent questions in your provider seems very extensive, but from a user perspective very nice and thoughtful. If it where me I would not give an option to choice your authentication method and just default to one. Since you already coded it, it seems fine.

Regarding the data provided in the onboarding process, I do think the PortalURL and DomainsURL are a bit excessive. I like the URL to dnscontrol and an url to the provider (www.transip.nl) seems fine, but both seems a bit weird. Also those are pretty specific to transip, for other providers the domains are calles zones and you do not have one portal, for instance for AWS it is dependent on your region.

The notes as a separate text field for more info about the authentication is a very good idea.

Your questions

1. Yeah one issue with checkboxes where we link the PRs works for me
2. I would not remove the documentation, either keep it as is or generate it from the provided metadata. The reasoning for it is an UX one. If I am an existing user of DNSControl I would not think to run the init commando more than once, my project has already been initialized. So I would expect the docs to be sufficient for me to add new providers. Also after already having setup an provider, and when the remote provider deprecates the authentication method (or it expires) I would still look in the docs for solutions. Not running an init commando. Lastly, my least joyful reason: thank about the AI, those like to consume text to make magic happen (or not happen, depending on their mood).

[tomfitzhenry]

Yes, I think onboarding could be improved with a wizard. I'm glad to see this is being worked on!

It might be easier to develop/review/ship/use if it has a smaller scope, e.g.

• just using NewRegistrar("dohdefault"); for now, rather than provider-specific registrars
• not having a generic fallback. Users can always fallback to the existing doc-based approach. I expect the very popular providers will quickly get working wizard implementations anyway.
• just supporting a single auth mechanism per provider

All of those could be added later. I'm sure lots will be learned from shipping a simpler version.

If you do decide to support multiple auth methods, that could be a field like:

AuthMechanisms: map[string]providers.AuthMechanism{
    "access_token": providers.AuthMechanism{
        description: "Authenticate with an access token",
        fields: []providers.AuthField{providers.AuthField{name: "access_token"}},
    },
    "private_key": providers.AuthMechanism{
        description: "Authenticate with your username and a private key",
        fields: []providers.AuthField{providers.AuthField{name: "username", public: true}, providers.AuthField{name: "private_key"}},
    },
}

This maps a bit closer to the relationship between the fields, and so avoids awkward fields like Required, ShowIf, and Internal. There might be some duplication, but not too much.

[cafferata]

Thanks for the careful read, @blackshadev. The DomainsURL point lands, my framing was too TransIP shaped. I will drop it and keep just PortalURL.

A short note on the rest of your feedback so others know how I plan to fold it in:

• Auth method picker: leaving it in. The Internal plus ShowIf plumbing already supports it cleanly and providers without multiple methods just skip it.
• Notes: keeping as a free text field, useful as you noted.
• Rollout: agreed, one tracking issue with a checkbox per provider so PRs can link back to it.
• Documentation: agreed, the per provider docs stay. Generating the credentials section from the same metadata is a nice idea but I will park it for a later PR rather than expand the initial scope.

---

Thanks @tomfitzhenry, the scope reductions are good food for thought.

• Provider specific registrars: keeping this in. The wizard already defaults to NONE so users who do not want a registrar accept it with a single keystroke. For those who do, the picker is one short list, so the simple case stays cheap.
• Generic fallback: agree, dropping that for the first version of the PR. Providers without registered metadata simply do not appear in the picker, with a pointer to the doc page instead. Cleaner surface and a clearer signal to maintainers that adding metadata is the way in.
• Single auth mechanism: I would like to leave it in for now and see how it scales. TransIP (token versus account name + private key) and Route53 (static keys versus assume role) are the early test cases. I will put the PR up when it is ready and would value your technical review on whether the approach holds up.

[SukkaW]

When I first used DNSControl, I wished to have some kind of ready-to-use template so that I could immediately run dnscontrol check or dnscontrol preview to get my hands on.

And since then, I have actually made one: https://github.com/SukkaW/dnscontrol-gitops-template

As for the provider, registering more metadata (like where to create a new API token, where to find the DNS service provider's own documentation, etc.) fields LGTM. We may even improve the DNSControl's error message a bit, i.e., instead of dnscontrol preview/push simply throwing the Missing authentication method-like message, the CLI can also print a few useful documentation links (about how to edit creds.json, the docs of the provider, where to create an API token based on providers' metadata, etc.).

[chicks-net]

I like the.direction this is going in. The definition of the fields seemed a bit over-designed with the if-conditions, but it makes sense to me now. I can see that it would be nice to add this to the docs automatically similar to #4173 in a future PR.

I also like SukkaW's example repo. It makes me wonder if I've missed any others. I'd also like to plug https://github.com/fini-net/fini-coredns-example as a working example and onboarding path. This raises some tangentially related questions so I've started #4206 for those questions.

[systemcrash]

Anything to ease onboarding, but as always: read the manual.

[SukkaW]

Anything to ease onboarding, but as always: read the manual.

And as I mentioned, dnscontrol may include a few docs link in the error message output to help users better find the docs they need: #4205 (comment)

i.e., instead of dnscontrol preview/push simply throwing the Missing authentication method-like message, the CLI can also print a few useful documentation links (about how to edit creds.json, the docs of the provider, where to create an API token based on providers' metadata, etc.).

[cafferata]

First implementation is now open as #4208 for technical review. The PR description includes the full context, sample transcripts, the GitBook documentation preview, and the test plan. Earlier feedback from this thread (drop DomainsURL, keep the auth method picker, drop the generic fallback, keep the per provider docs) is reflected in the code.

Looking forward to your reviews there. 👀