FEATURE: Create zones during preview (Disable with --populate-on-preview=false) #3337
Conversation
|
Initially I was planning on invoking Side note: The race-condition as fixed in #3328 is no longer an issue; the improved cache handling when creating zones is still worth landing. |
There was a problem hiding this comment.
Code: looks good
Testing: I'll get around to that later this week or early next week.
Design alternative suggestion: Maybe Zone caching should be done globally instead? I see each provider has a mutex-protected cache. Expecting every provider to get this right seems like a risk. Any time we make all providers do the same kind of work, it's better to move that code to the main DNSControl system.
That is... Maybe a better interface would be to treat ZoneNames like we do other things with GetFOO() and GetFOOCorrections().
What would this look like?
Providers would provide 2 functions:
- GetZoneNames() which returns a []string. No mutexes, etc. just do the work and return a list of strings.
- GetZoneNamesCorrections(). We send it a []string of zones to be created; it returns []Corrections that create those zones.
The main DNSControl code would call those functions, store the data, handling all mutexes, etc. as needed.
Thoughts?
|
Thanks for the feedback, Tom! Multiple drivers need details from the zone listing for internal usage. A few examples off the top of my head:
Drivers need access to that information from various external and internal methods, so they need a place to "cache" this structured information to avoid duplicate provider API calls. Idea: Go-Generics are GA now, we could provide a simple caching primitive that drivers can use: func NewZoneCache[Zone any](fetchAll func() (map[string]Zone, error)) ZoneCache[Zone]
type ZoneCache [Zone any] struct {
mu sync.Mutex
cache map[string]Zone
fetchAll func() (map[string]Zone, error)
}
func (z *ZoneCache) GetZone(name string) (Zone, error)
func (z *ZoneCache) GetZoneNames() ([]string, error)
func (z *ZoneCache) AddZone(name string, zone Zone)
type myZone struct {
ID string
TTL int
}
type myDriver struct {
zoneCache ZoneCache[myZone]
}
func (d *myDriver) fetchZones (map[string]myZone, error)
func NewDriver() {
d := myDriver{}
d.zoneCache = NewZoneCache(d.fetchZones)
}With that said, I like your suggestion for refactoring the |
|
Good point! Providers do need more than just a []string of zone names. I hadn't thought of that. I like your idea of using generics! Let's give that a try. Some thoughts (feel free to ignore any of these)
Thanks! |
|
I've added
I've tested the changes together with the other PRs for Cloudflare and Hetzner: I ran the integration tests for both (CF: worker/redirect tests skipped). I've manually tested pushing with a single new zone first and then an existing plus a new zone. All working as expected 🎉 |
|
Hi @das7pad Please rebase and I'll try to merge this before we get more conflicts (oops!) How does this relate to the other PRs you've submitted recently? sorry but I've gotten a little lost and I'm not sure what order to merge things. |
Any corrections for creating zones will be displayed/executed in a separate loop before the main loop for gathering corrections kicks off. This zone-check loop is silent unless any provider has corrections. Zones will get created on preview already to show a complete picture right away (opt-out via --populate-on-preview=false).
das7pad
force-pushed
the
create-zone-order
branch
from
aa65c67 to
ddc6f85
Compare
No worries. I've rebased the changes and squashed the copy change for the flag.
This PR ensures that zones get created as the first step of processing zones. Compatible drivers [1] can then be used with Here, a "compatible driver" [1] is a provider without a zone cache, or with a zone cache implementation that either populates the cache with any newly creates zones; or (properly) resets the cache after creating zones. The Does that help? |
|
Yes! Very helpful! Sounds like I can merge these in any order (which is brilliant of you!). However the optimal order is this PR and the ZoneCache PR (I don't see a PR related to https://github.com/das7pad/dnscontrol/tree/zone-cache... has it been created?). Then the other PRs can be rewritten to use ZoneCache and be merged independently. Did I get that right? (I hope so!) Tom |
tlimoncelli
changed the title
|
Awesome 🎉 I've opened #3365 with the |
This PR is changing the order for creating zones to happen before any data is gathered for the zone (i.e.
driver.GetNameservers/driver.GetZoneRecordscalls).Any corrections for creating zones will be displayed/executed in a separate loop before the main loop for gathering corrections kicks off. This zone-check loop is silent unless any provider has corrections.
Zones will get created on preview already to show a complete picture right away (opt-out via
--populate-on-preview=false).With these changes, a single
dnscontrol pushinvocation is sufficient for bootstrapping a zone with HETZNER (and most other providers) 🎉Demo with HETZNER and two zones
Before
A lot of errors and nothing is happening.
A lot of errors, the zone gets created in the 1st pass. The 2nd pass actually creates resources in the zone.
After
No errors! The first pass creates the zone. The 2nd pass is just the preview, no noise from zone checking.
A single pass is sufficient for creating the zone, figuring out nameservers and zone records. 🎉
Closes #3007