fix: reduce hosts plugin refresh interval

[saewoni]

Summary

• reduce the AKS LocalDNS hosts setup timer refresh interval from 15 minutes to 10 seconds
• tighten timer accuracy from 1 minute to 1 second so the shorter cadence is honored
• update the timer comments to describe the new refresh behavior

Validation

• git diff --check
• systemd-analyze verify parts/linux/cloud-init/artifacts/aks-localdns-hosts-setup.timer
• docker run --rm -v "/home/sakwa/agentbaker:/src" shellspec-docker --shell bash --format d spec/parts/linux/cloud-init/artifacts/cse_config_spec.sh

[Copilot]

Pull request overview

This PR changes the systemd timer that periodically runs the AKS LocalDNS hosts setup job, reducing the refresh cadence so /etc/localdns/hosts is updated much more frequently.

Changes:

• Reduce aks-localdns-hosts-setup.timer refresh interval from 15 minutes to 10 seconds.
• Tighten timer scheduling accuracy from 1 minute to 1 second.
• Update inline timer comments to match the new intended behavior.

[yewmsft]

why 10s? we went from 15min to 10s — that's ~90x more frequent. some concerns before this lands:

1. load math. on a 1000-node cluster × ~10s period × N critical FQDNs = ~100N dig qps against upstream. RandomizedDelaySec=5s only spreads across 5s of a 10s window, so peak smoothing is ~50%. what's the typical critical-FQDN count, and have we measured this against VNet DNS / 168.63.129.16 headroom? agentbaker fix: run aptmarkwalinuxagent hold operation on foreground #7797 is the precedent — provisioning-timing changes that affect upstream load need a production metric + canary before fleet rollout.

2. OnUnitInactiveSec vs OnUnitActiveSec. you also changed the semantics. OnUnitInactiveSec=10s fires 10s after the script completes, not every 10s. If aks-localdns-hosts-setup.sh takes 3–5s (multiple dig calls with timeout 3, can retry across upstream servers), the effective period is ~13–15s. Either is fine, but the inline comment still says "10 seconds after each run completes" — OnUnitActiveSec would actually deliver the "every ~10s" reading more people expect. Pick one and make the comment match the semantics.

3. AccuracySec=1s. default is 1min, which lets systemd coalesce wake-ups across timers. dropping to 1s defeats that batching — small per-node cost but real at fleet scale and not free on battery-constrained / low-end SKUs. is 1s accuracy actually load-bearing here when the wall-clock target is ~10s and the script itself takes longer than that?

4. why 10s specifically? the prior comment said "15 minutes balances freshness against unnecessary DNS traffic" — that's gone now. what data drove 10s vs 30s vs 1min? if there was a specific stale-IP incident, link the ICM so the next person reading this .timer knows what 10s is calibrated against and when it can be relaxed again.

5. e2e / perf signal. plan to canary on a small region and watch (a) upstream DNS qps, (b) node CPU/journald write rate from the per-run script logs, (c) any drift in aks-localdns-hosts-setup.service failures before rolling everywhere?

[yewmsft]

why 10s? we went from 15min to 10s — that's ~90x more frequent. some concerns before this lands:

1. load math. on a 1000-node cluster × ~10s period × N critical FQDNs = ~100N dig qps against upstream. RandomizedDelaySec=5s only spreads across 5s of a 10s window, so peak smoothing is ~50%. what's the typical critical-FQDN count, and have we measured this against VNet DNS / 168.63.129.16 headroom? agentbaker fix: run aptmarkwalinuxagent hold operation on foreground #7797 is the precedent — provisioning-timing changes that affect upstream load need a production metric + canary before fleet rollout.
2. OnUnitInactiveSec vs OnUnitActiveSec. you also changed the semantics. OnUnitInactiveSec=10s fires 10s after the script completes, not every 10s. If aks-localdns-hosts-setup.sh takes 3–5s (multiple dig calls with timeout 3, can retry across upstream servers), the effective period is ~13–15s. Either is fine, but the inline comment still says "10 seconds after each run completes" — OnUnitActiveSec would actually deliver the "every ~10s" reading more people expect. Pick one and make the comment match the semantics.
3. AccuracySec=1s. default is 1min, which lets systemd coalesce wake-ups across timers. dropping to 1s defeats that batching — small per-node cost but real at fleet scale and not free on battery-constrained / low-end SKUs. is 1s accuracy actually load-bearing here when the wall-clock target is ~10s and the script itself takes longer than that?
4. why 10s specifically? the prior comment said "15 minutes balances freshness against unnecessary DNS traffic" — that's gone now. what data drove 10s vs 30s vs 1min? if there was a specific stale-IP incident, link the ICM so the next person reading this .timer knows what 10s is calibrated against and when it can be relaxed again.
5. e2e / perf signal. plan to canary on a small region and watch (a) upstream DNS qps, (b) node CPU/journald write rate from the per-run script logs, (c) any drift in aks-localdns-hosts-setup.service failures before rolling everywhere?

1. dns server load calculation is off. dns server throttling limit is against per vm, azure dns is 1000 qps per vm. so number of nodes does not matter in this calculation.
2. do you dig in parallel? or in batches? OnUnitInactiveSec=10s is fine.
3. 1s is fine, since you already randomized delay 5s.
4. by design
5. n/a

[yewmsft]

stop re-review

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.