Skip to content

Commit 50f4a43

Browse files
committed
fix(network/dns): default-config peer DNS routing via /etc/resolver (CHAOS-1478)
Completes the CHAOS-1478 fix: ``container run --rm busybox nslookup <peer>`` now resolves a sibling container in the default configuration, with no ``sudo``, no ``container system dns create``, and no manual ``dns.domain`` configuration. The earlier source change (23ef3d0) made the embedded DNS handler register peer hostnames in their bare form. That fix was necessary but not sufficient: queries from inside containers still had no path to reach ``127.0.0.1:2053`` because ``vmnet``'s built-in DNS proxy on ``192.168.x.1:53`` forwards unmatched queries upstream rather than to the embedded handler. Three architectural options were evaluated. (A) Bind the handler on ``192.168.x.1:53`` and (B) run a UDP forwarder there from inside the ``container-network-vmnet`` plugin both turned out to be infeasible: ``com.apple.security.virtualization`` does not grant privileged-port binding, and every helper - apiserver and ``container-network-vmnet`` included - runs as the invoking uid (verified empirically via ``bind('192.168.x.1', 53)`` -> EACCES from uid 501). macOS 26 does expose ``vmnet_network_configuration_disable_dns_proxy()``, but disabling the proxy still leaves the handler unable to claim ``:53``. We therefore fall through to (C), the documented ``/etc/resolver`` mechanism, but automate it at install time so the user never sees a runtime sudo prompt. The chain at runtime: VM glibc/musl appends ``.test`` via ``search`` directive vmnet :53 proxy forwards ``probe-pg.test`` to macOS system resolver macOS resolver matches /etc/resolver/containerization.test, routes to 127.0.0.1 port 2053 apiserver :2053 strips ``.test`` (DNSRegistrationKey), looks up bare ``probe-pg``, returns 192.168.x.y Changes: - ``ContainerPersistence/ContainerSystemConfig.swift``: ``DNSConfig`` now defaults ``domain`` to ``\"test\"`` (centralised in ``defaultDomain``). Both the no-arg initialiser and the TOML decode path fall back to the default; an explicit nil/empty string from config is still honoured. The default propagates to the embedded handler's ``dnsDomain`` argument and to per-container DNS plumbing. - ``ContainerAPIService/Client/Utility.swift``: when constructing a container's ``DNSConfiguration`` and the user has not specified any search domains, inject ``[domain]`` so bare-name queries actually pick up the suffix and traverse the resolver chain above. Honours user overrides; explicit empty strings remain empty. - ``scripts/pkg-scripts/postinstall``: new ``.pkg`` postinstall script that writes ``/etc/resolver/containerization.test`` (filename and contents matching ``HostDNSResolver`` so existing add/remove tooling composes cleanly) and refreshes ``mDNSResponder``. The .pkg installer already runs as root, so this introduces no new sudo prompt. - ``Makefile``: pass ``--scripts scripts/pkg-scripts`` to ``pkgbuild`` so the postinstall is bundled into the installer. - ``scripts/uninstall-container.sh``: best-effort cleanup of the resolver entry on uninstall, keeping the host's ``/etc/resolver`` state symmetric with install. - ``Tests/ContainerPersistenceTests/ContainerSystemConfigDNSDefault\ Tests.swift``: 8 new unit tests covering the default value, explicit overrides, the empty-string opt-out, and the TOML decode paths (missing section, empty section, explicit override). The 32 tests from the earlier CHAOS-1478 commits continue to pass (40 / 40). Live repro on a clean container restart (debug build, macOS 26): $ container system stop && sudo make all install && container system start $ container run --rm -d --name probe-pg -e POSTGRES_PASSWORD=test postgres:alpine $ container run --rm busybox nslookup probe-pg Server: 192.168.65.1 Address: 192.168.65.1:53 Name: probe-pg.test Address: 192.168.65.8 $ dig @127.0.0.1 -p 2053 probe-pg. +short # key-form fix intact 192.168.65.8 $ dig @127.0.0.1 -p 2053 probe-pg.test. +short # suffix-strip works 192.168.65.8 $ dscacheutil -q host -a name probe-pg.test # /etc/resolver host path ip_address: 192.168.65.8 Constraints honoured: * No new public Codable schema changes; ``DNSConfig.domain`` remains ``String?`` and existing TOMLs decode unchanged. * No runtime sudo prompts. Sudo is required only at ``.pkg`` install time, which the user already accepts via ``installer -pkg``. * The 32 pre-existing CHAOS-1478 unit tests continue to pass. * CHAOS-1476 (multi-alias) remains out of scope.
1 parent 11d7948 commit 50f4a43

7 files changed

Lines changed: 215 additions & 4 deletions

File tree

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ installer-pkg: $(STAGING_DIR)
125125
@codesign $(CODESIGN_OPTS) --prefix=com.apple.container. --entitlements=signing/container-network-vmnet.entitlements "$(join $(STAGING_DIR), libexec/container/plugins/container-network-vmnet/bin/container-network-vmnet)"
126126

127127
@echo Creating application installer
128-
@pkgbuild --root "$(STAGING_DIR)" --identifier com.apple.container-installer --install-location /usr/local --version ${RELEASE_VERSION} $(PKG_PATH)
128+
@pkgbuild --root "$(STAGING_DIR)" --scripts scripts/pkg-scripts --identifier com.apple.container-installer --install-location /usr/local --version ${RELEASE_VERSION} $(PKG_PATH)
129129
@rm -rf "$(STAGING_DIR)"
130130

131131
.PHONY: dsym

Package.swift

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -434,6 +434,7 @@ let package = Package(
434434
dependencies: [
435435
.product(name: "Logging", package: "swift-log"),
436436
.product(name: "SystemPackage", package: "swift-system"),
437+
.product(name: "TOML", package: "swift-toml"),
437438
"ContainerPersistence",
438439
"ContainerTestSupport",
439440
]

Sources/ContainerPersistence/ContainerSystemConfig.swift

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -117,15 +117,22 @@ final public class ContainerConfig: Codable, Sendable {
117117
}
118118

119119
final public class DNSConfig: Codable, Sendable {
120+
/// Default DNS suffix used to route container peer-name queries
121+
/// through the embedded handler. The matching `/etc/resolver/`
122+
/// entry is installed by the macOS package postinstall script
123+
/// (see `scripts/pkg-scripts/postinstall`) so peer DNS works in the
124+
/// default configuration without any runtime sudo prompt.
125+
public static let defaultDomain = "test"
126+
120127
public let domain: String?
121128

122-
public init(domain: String? = nil) {
129+
public init(domain: String? = defaultDomain) {
123130
self.domain = domain
124131
}
125132

126133
public init(from decoder: any Decoder) throws {
127134
let container = try decoder.container(keyedBy: CodingKeys.self)
128-
self.domain = try container.decodeIfPresent(String.self, forKey: .domain)
135+
self.domain = try container.decodeIfPresent(String.self, forKey: .domain) ?? Self.defaultDomain
129136
}
130137
}
131138

Sources/Services/ContainerAPIService/Client/Utility.swift

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -226,10 +226,21 @@ public struct Utility {
226226
config.dns = nil
227227
} else {
228228
let domain = management.dns.domain ?? containerSystemConfig.dns.domain
229+
// Auto-inject the configured DNS domain as a search-domain when
230+
// the user has not specified any. This is what makes bare-name
231+
// peer queries (e.g. `nslookup probe-pg`) hit the embedded DNS
232+
// handler in the default configuration: glibc/musl appends the
233+
// search suffix, the FQDN flows through vmnet's DNS proxy to the
234+
// host system resolver, and the matching `/etc/resolver/`
235+
// entry routes it to `127.0.0.1:2053`. See CHAOS-1478.
236+
var searchDomains = management.dns.searchDomains
237+
if searchDomains.isEmpty, let domain, !domain.isEmpty {
238+
searchDomains = [domain]
239+
}
229240
config.dns = .init(
230241
nameservers: management.dns.nameservers,
231242
domain: domain,
232-
searchDomains: management.dns.searchDomains,
243+
searchDomains: searchDomains,
233244
options: management.dns.options
234245
)
235246
}
Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
//===----------------------------------------------------------------------===//
2+
// Copyright © 2026 Apple Inc. and the container project authors.
3+
//
4+
// Licensed under the Apache License, Version 2.0 (the "License");
5+
// you may not use this file except in compliance with the License.
6+
// You may obtain a copy of the License at
7+
//
8+
// https://www.apache.org/licenses/LICENSE-2.0
9+
//
10+
// Unless required by applicable law or agreed to in writing, software
11+
// distributed under the License is distributed on an "AS IS" BASIS,
12+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
// See the License for the specific language governing permissions and
14+
// limitations under the License.
15+
//===----------------------------------------------------------------------===//
16+
17+
import Foundation
18+
import Testing
19+
import TOML
20+
21+
@testable import ContainerPersistence
22+
23+
/// Tests for the default `dns.domain` baked into `ContainerSystemConfig.DNSConfig`
24+
/// as part of the CHAOS-1478 routing fix.
25+
///
26+
/// The default value (`"test"`) MUST stay in sync with the resolver file written
27+
/// by `scripts/pkg-scripts/postinstall` (`/etc/resolver/containerization.test`).
28+
/// If you change the default here, also update the postinstall script.
29+
struct ContainerSystemConfigDNSDefaultTests {
30+
// MARK: - Default value
31+
32+
/// `DNSConfig()` (no arguments) yields the default domain. This is the
33+
/// path used when `ContainerSystemConfig` is constructed without a TOML
34+
/// file (e.g. fresh install, no user config).
35+
@Test func testDefaultDomainIsTest() {
36+
let config = DNSConfig()
37+
#expect(config.domain == "test")
38+
#expect(config.domain == DNSConfig.defaultDomain)
39+
}
40+
41+
/// Explicit `nil` is preserved — caller can intentionally clear the domain.
42+
/// This is needed for tests and for advanced configurations where the
43+
/// caller wants to skip search-domain injection entirely.
44+
@Test func testExplicitNilIsPreserved() {
45+
let config = DNSConfig(domain: nil)
46+
#expect(config.domain == nil)
47+
}
48+
49+
/// Explicit non-default domain is preserved.
50+
@Test func testExplicitDomainIsPreserved() {
51+
let config = DNSConfig(domain: "example.com")
52+
#expect(config.domain == "example.com")
53+
}
54+
55+
// MARK: - TOML decode
56+
57+
/// TOML with no `[dns]` section at all → top-level config has the default.
58+
/// This is the most common case: a user with a minimal runtime-config.toml.
59+
@Test func testTOMLDecodeMissingDnsSectionUsesDefault() throws {
60+
let toml = ""
61+
let decoded = try TOMLDecoder().decode(ContainerSystemConfig.self, from: Data(toml.utf8))
62+
#expect(decoded.dns.domain == "test")
63+
}
64+
65+
/// TOML with `[dns]` but no `domain` key → defaults to `"test"`.
66+
@Test func testTOMLDecodeEmptyDnsSectionUsesDefault() throws {
67+
let toml = """
68+
[dns]
69+
"""
70+
let decoded = try TOMLDecoder().decode(ContainerSystemConfig.self, from: Data(toml.utf8))
71+
#expect(decoded.dns.domain == "test")
72+
}
73+
74+
/// TOML with explicit `dns.domain = "foo"` → user override takes effect.
75+
/// This guards the "user can opt out / replace" pathway.
76+
@Test func testTOMLDecodeExplicitDomainOverridesDefault() throws {
77+
let toml = """
78+
[dns]
79+
domain = "internal.example"
80+
"""
81+
let decoded = try TOMLDecoder().decode(ContainerSystemConfig.self, from: Data(toml.utf8))
82+
#expect(decoded.dns.domain == "internal.example")
83+
}
84+
85+
/// TOML with explicit empty-string domain → preserved as empty string.
86+
/// Empty string is treated as "no domain" by the search-domain injection
87+
/// logic in `Utility.containerConfigFromFlags`. Decoding must NOT silently
88+
/// substitute the default in this case — the user explicitly opted out.
89+
@Test func testTOMLDecodeExplicitEmptyStringDomainIsPreserved() throws {
90+
let toml = """
91+
[dns]
92+
domain = ""
93+
"""
94+
let decoded = try TOMLDecoder().decode(ContainerSystemConfig.self, from: Data(toml.utf8))
95+
#expect(decoded.dns.domain == "")
96+
}
97+
98+
// MARK: - Top-level ContainerSystemConfig wiring
99+
100+
/// `ContainerSystemConfig()` with all defaults exposes the DNS default at
101+
/// the top level — verifies the construction chain doesn't drop it.
102+
@Test func testTopLevelDefaultExposesDNSDefault() {
103+
let config = ContainerSystemConfig()
104+
#expect(config.dns.domain == "test")
105+
}
106+
}

scripts/pkg-scripts/postinstall

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
#!/bin/bash
2+
# Copyright © 2025-2026 Apple Inc. and the container project authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# https://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# Postinstall hook for the container .pkg installer (CHAOS-1478 routing fix).
17+
#
18+
# The embedded DNS handler in `container-apiserver` listens on `127.0.0.1:2053`
19+
# but containers query DNS at `192.168.64.1:53` (the vmnet bridge gateway).
20+
# Apple's vmnet framework intercepts `:53` on the bridge gateway and forwards
21+
# unmatched queries to the macOS system resolver, which by default does not
22+
# know about peer container hostnames.
23+
#
24+
# Binding the embedded handler directly on the bridge gateway is not possible:
25+
# port 53 is privileged and the daemon runs as the invoking user (uid != 0).
26+
# Routing the queries via `/etc/resolver/<domain>` is the supported macOS path.
27+
# The resolver file must be installed by a privileged context, so we drop it
28+
# here at install time (the .pkg installer already runs with administrator
29+
# privileges that the user has accepted).
30+
#
31+
# This is the matching server-side configuration for the default `dns.domain`
32+
# value baked into `ContainerSystemConfig.DNSConfig.defaultDomain`. Containers
33+
# get `search test` injected automatically (see Utility.swift), so a bare-name
34+
# query like `nslookup probe-pg` resolves through the chain:
35+
#
36+
# VM glibc/musl : appends ".test" via search domain
37+
# vmnet :53 proxy : forwards "probe-pg.test" to macOS system resolver
38+
# macOS resolver : matches /etc/resolver/containerization.test ->
39+
# 127.0.0.1 port 2053
40+
# apiserver :2053 : strips ".test" suffix, looks up "probe-pg",
41+
# returns 192.168.64.x
42+
#
43+
# The matching `containerizationPrefix` and resolver-file format come from
44+
# `Sources/Services/ContainerAPIService/Client/HostDNSResolver.swift`.
45+
46+
set -euo pipefail
47+
48+
# Default DNS suffix for container peer-name resolution.
49+
# MUST stay in sync with `ContainerSystemConfig.DNSConfig.defaultDomain`.
50+
readonly DOMAIN="test"
51+
52+
# `/etc/resolver/` filename convention from `HostDNSResolver.containerizationPrefix`.
53+
readonly RESOLVER_DIR="/etc/resolver"
54+
readonly RESOLVER_FILE="${RESOLVER_DIR}/containerization.${DOMAIN}"
55+
56+
# Embedded handler bind from `APIServer+Start.swift` (`Self.dnsPort`).
57+
readonly DNS_PORT=2053
58+
59+
mkdir -p "${RESOLVER_DIR}"
60+
61+
# Idempotent: overwrite to ensure the file matches the current expected
62+
# contents even after a partial / older install.
63+
cat >"${RESOLVER_FILE}" <<EOF
64+
domain ${DOMAIN}
65+
search ${DOMAIN}
66+
nameserver 127.0.0.1
67+
port ${DNS_PORT}
68+
EOF
69+
chmod 0644 "${RESOLVER_FILE}"
70+
71+
# Refresh mDNSResponder so it picks up the new resolver entry without a reboot.
72+
# Best-effort: failure here is not fatal — the resolver file alone is enough
73+
# once the system reloads naturally.
74+
if pgrep -q mDNSResponder; then
75+
killall -HUP mDNSResponder 2>/dev/null || true
76+
fi
77+
78+
exit 0

scripts/uninstall-container.sh

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,14 @@ for ((i=${#DIRS[@]}-1; i>=0; i--)); do
8080
done
8181

8282
sudo pkgutil --forget com.apple.container-installer > /dev/null
83+
84+
# Remove the /etc/resolver/ entry installed by scripts/pkg-scripts/postinstall (CHAOS-1478).
85+
# Best-effort: missing file is fine; mDNSResponder reload is best-effort too.
86+
if [ -f /etc/resolver/containerization.test ]; then
87+
sudo rm -f /etc/resolver/containerization.test
88+
sudo killall -HUP mDNSResponder 2>/dev/null || true
89+
fi
90+
8391
echo 'Removed `container` tool and helpers'
8492

8593
if [ "$DELETE_DATA" = true ]; then

0 commit comments

Comments
 (0)