-
Notifications
You must be signed in to change notification settings - Fork 10
test(e2e): Machine networking — first exercise of the network plane #467
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
AprilNEA
wants to merge
7
commits into
master
Choose a base branch
from
test/machine-network
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+656
−1
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
d525765
test(e2e): Machine networking — first exercise of the network plane
AprilNEA ed779e7
fix(e2e): assert the nslookup answer block, not the resolver banner
AprilNEA bea15be
Merge remote-tracking branch 'origin/master' into test/machine-network
AprilNEA 01d1d78
fix(e2e): make the nslookup preamble fixture faithful to busybox
AprilNEA 7e22638
test(e2e): make M3 a content check, not a length check
AprilNEA a96c8b1
test(e2e): let the pattern origin linger before closing, like the blo…
AprilNEA 9469f28
docs(e2e): correct the nslookup exit-code and error-stream claims
AprilNEA File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Machine networking E2E | ||
|
|
||
| `tests/e2e/tests/machine_network.rs` — the first exercise of a **Machine's | ||
| actual network plane**. The lifecycle test (`machine.rs`) only asserts an | ||
| agent-reported IP over vsock; it never drives a packet. This runs real | ||
| traffic from inside a Machine over the `machines.exec` vsock channel (the | ||
| Machine's `docker exec`). | ||
|
|
||
| ``` | ||
| cargo test -p arcbox-e2e --test machine_network -- --ignored --nocapture | ||
| ``` | ||
|
|
||
| Needs internet (create pulls alpine from the live `image.arcboxcdn.com` | ||
| mirror) and a guest `arcbox-agent` (musl cross-build, or the installed | ||
| app's agent staged by newest-mtime). | ||
|
|
||
| ## Datapath (source-verified 2026-07-21) | ||
|
|
||
| A Machine's **primary NIC is the same socketpair userspace netstack + | ||
| TcpBridge as the System VM's** (`virt/arcbox-vmm/src/vmm/darwin.rs` — | ||
| `gateway=10.0.2.1, guest=10.0.2.2`, DHCP + `DnsForwarder` at the gateway). | ||
| Egress is pure in-process host-socket proxying | ||
| (`common/arcbox-proxy/src/egress/mod.rs`) — **no privileged helper, no host | ||
| route, no `/etc/resolver`**. The System VM's helper-installed `172.16/12` | ||
| route (`route_reconciler`) is System-VM-only and was never wired to | ||
| Machines (`app/arcbox-core/src/machine.rs` never calls it). So Machine | ||
| networking runs fully in the isolated e2e daemon. | ||
|
|
||
| The agent channel is vsock, independent of the network plane | ||
| (`MachineManager::connect_agent`), which is why `exec` drives in-Machine | ||
| commands even while testing the network. | ||
|
|
||
| ## Scenarios (one Machine, one boot) | ||
|
|
||
| | # | What | Assertion | | ||
| |---|---|---| | ||
| | M1 | egress TCP | `wget` a host-local origin at `10.0.2.1:<port>`; `wc -c` byte-exact — first proof a Machine reaches the network *(implemented)* | | ||
| | M2 | DNS | `nslookup host.docker.internal` / `gateway.docker.internal` resolve to `10.0.2.1` via the in-VMM `DnsForwarder`. Asserted on the parsed **answer block only** — busybox echoes its resolver (`Server: 10.0.2.1`) and exits 0 on NXDOMAIN, so matching whole output is a tautology when the resolver is the expected answer *(implemented)* | | ||
| | M3 | egress volume | 16 MiB download from a seeded-pattern origin, hashed in-guest (`sha256sum`) against the origin's digest and bounded — `wc -c` against `spawn_blob_server`'s repeated all-zero chunk would pass on any stream of the right length, however reordered or corrupted *(implemented)* | | ||
| | M4 | metadata | `inspect` reports gateway `10.0.2.1` and it as a DNS server (gated); `ip_address` is characterized only — see the finding below *(implemented)* | | ||
| | M5 | SSH contract | `ssh_info` is still `Unimplemented` — pins the gap so a future SSH feature trips this test *(implemented)* | | ||
|
|
||
| ## Not covered — by architecture, not omission | ||
|
|
||
| Documented here because the architecture, not the harness, is the reason; | ||
| no active test (would be flaky/meaningless today): | ||
|
|
||
| - **Machine ↔ Machine, Machine → container, Machine → System VM**: each VM | ||
| gets its own private per-process socketpair netstack; the second (vmnet | ||
| bridge) NIC is never brought up guest-side for Machines | ||
| (`guest/arcbox-agent/src/init.rs` `machine_init()` does DHCP on the | ||
| primary NIC only). Two Machines can even both be `10.0.2.2` — there is no | ||
| shared segment and no cross-VM route. When cross-machine networking is | ||
| added, M5's pattern (assert-the-gap-then-grow) is the template. | ||
| - **host → Machine inbound / SSH**: `ssh_info` is unimplemented and | ||
| `InboundListenerManager` is only ever wired to the System VM | ||
| (`app/arcbox-docker/src/handlers/container/mod.rs`), never to a Machine. | ||
| M5 pins this. | ||
|
|
||
| ## Finding (2026-07-21): `inspect` reports a DNS answer, not the Machine's IP | ||
|
|
||
| The datapath logs `gateway=10.0.2.1, guest=10.0.2.2`, and egress/DNS work | ||
| through it (M1–M3 pass), but `inspect().network.ip_address` came back as | ||
| `198.18.11.51` — the Surge/Clash fake-IP range this host runs, not the | ||
| datapath's `10.0.2.2`. | ||
|
|
||
| Root cause (traced 2026-08-01, was filed as "host-environment-tangled"): | ||
| the field is not an interface address at all. `select_routable_ip` | ||
| (`app/arcbox-core/src/machine.rs`) picks the first usable entry from | ||
| `SystemInfo.ip_addresses`, which the guest agent fills by running | ||
| `hostname -I`, falling back to `hostname -i` | ||
| (`guest/arcbox-agent/src/agent/linux/system_info.rs`). Alpine ships busybox | ||
| `hostname`, which has no `-I`, and its `-i` **resolves the guest's own | ||
| hostname through DNS** instead of enumerating interfaces. So the reported | ||
| address is whatever the resolver answers for the Machine's hostname — | ||
| a fake-IP here, plausibly NXDOMAIN (hence an empty field) on a clean host. | ||
| The desktop UI shows this as "the machine's IP", so users see a bogus value. | ||
|
|
||
| Consequence for M4: gating on the datapath subnet — or on the guest's real | ||
| interfaces — would fail for a producer reason, not a datapath reason. M4 | ||
| therefore gates the robust facts (gateway, `dns_servers`) and only | ||
| characterizes `ip_address` (valid non-special IPv4 + WARN on a non-datapath | ||
| value). The fix belongs in the guest agent (enumerate interfaces — | ||
| `/proc/net` or `getifaddrs` — rather than shelling out to `hostname`); | ||
| tighten M4 to the datapath subnet once that lands. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Third site from the exit-code thread, still stale.
9469f28fixed the twomachine_network.rssites the thread named (:288and:443-444), but this M2 row still says busybox "exits 0 on NXDOMAIN" — which is wrong for both builds: BIG setsG.exitcode = EXIT_FAILURE, legacy returns(rc != 0). Either way, exit 1.Worth catching now specifically because the thread that tracked it is resolved, so nothing else is watching this line. Same reasoning as last round applies here too — this doc is the plan of record for the network-plane work, so a wrong busybox claim in it will be read as authoritative.
The rest of the row is accurate and the conclusion it draws is unaffected: matching whole output is a tautology when the resolver is the expected answer, and
m2_dnsgating on parsed answers only remains correct regardless of the exit code.