Skip to content

vmnet: Support networks managed by vmnet-broker#22561

Draft
nirs wants to merge 5 commits intokubernetes:masterfrom
nirs:vmnet-network-support
Draft

vmnet: Support networks managed by vmnet-broker#22561
nirs wants to merge 5 commits intokubernetes:masterfrom
nirs:vmnet-network-support

Conversation

@nirs
Copy link
Copy Markdown
Contributor

@nirs nirs commented Jan 26, 2026

Summary

This PR adds support for using vmnet-broker managed networks with vmnet-helper. When vmnet-broker is installed and vmnet-helper supports the --network option, minikube will use broker-managed networks instead of legacy network (192.168.105.1/24).

Benefits

  • VM-to-VM communication: VMs on the same network can communicate with each other
  • Shared network management: vmnet-broker manages the network lifecycle across multiple VMs
  • Consistent networking: All VMs share the same subnet managed by the broker

Requirements

To use this feature:

  1. macOS 26 or later

  2. vmnet-helper v0.10.0+ with --network option support.

    curl -fsSL https://github.com/nirs/vmnet-helper/releases/download/v0.10.0-pre1/install.sh \
       | VMNET_VERSION=v0.10.0-pre1 bash
    
  3. vmnet-broker installed as a launchd daemon

    curl -fsSL https://github.com/nirs/vmnet-broker/releases/latest/download/install.sh | sudo bash
    

If either requirement is not met, minikube falls back to the current interface-id mode.

Changes

  1. drivers: consolidate GenerateMACAddress to common package - Move duplicate MAC generation to shared location
  2. vmnet: log interface address range from vmnet-helper - Add logging for vmnet interface details
  3. vmnet: add broker and network option detection - Add detection functions for broker and --network support
  4. vmnet: add NetworkName field and update drivers - Add NetworkName to Helper struct, update vfkit/krunkit drivers
  5. vmnet: add network mode support using vmnet-broker - Use --network option when available, generate MAC address

Testing

Testing vfkit

% minikube start --no-kubernetes -p vfkit --driver vfkit --network vmnet-shared
😄  [vfkit] minikube v1.37.0 on Darwin 26.2 (arm64)
✨  Using the vfkit driver based on user configuration
❗  Starting v1.39.0, minikube will default to "containerd" container runtime. See #21973 for more info.
👍  Starting minikube without Kubernetes in cluster vfkit
🔥  Creating vfkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
🐳  Preparing Docker 28.5.2 ...
🏄  Done! minikube is ready without Kubernetes!

% jq .Driver.VmnetHelper ~/.minikube/machines/vfkit/config.json
{
  "MachineDir": "/Users/nir/.minikube/machines/vfkit",
  "InterfaceID": "c951c4d5-2341-4f3e-a2a8-7c28baa1f287",
  "NetworkName": "vmnet:shared",
  "Offloading": false,
  "NeedsSudo": false
}

% minikube logs | grep vmnet-helper
I0126 17:21:04.776481   48702 main.go:144] libmachine: Validated vmnet-helper (path="/opt/vmnet-helper/bin/vmnet-helper", version="v0.10.0-pre1", commit="6e0c4a8194e036b998f9c7745953880ab77baa54", needsSudo=false, supportsNetwork=true, brokerInstalled=true)
I0126 17:21:05.186074   48702 main.go:144] libmachine: Started /opt/vmnet-helper/bin/vmnet-helper --socket /Users/nir/.minikube/machines/vfkit/vmnet-helper.sock --network shared (pid=48707)
I0126 17:21:05.737149   48702 main.go:144] libmachine: executing: vfkit --memory 6144 --cpus 2 --restful-uri unix:///Users/nir/.minikube/machines/vfkit/vfkit.sock --log-level debug --bootloader linux,kernel=/Users/nir/.minikube/machines/vfkit/bzimage,initrd=/Users/nir/.minikube/machines/vfkit/initrd,cmdline="console=hvc0" --device virtio-net,unixSocketPath=/Users/nir/.minikube/machines/vfkit/vmnet-helper.sock,mac=82:54:c6:45:8c:3a --device virtio-rng --device virtio-blk,path=/Users/nir/.minikube/machines/vfkit/boot2docker.iso --device virtio-blk,path=/Users/nir/.minikube/machines/vfkit/disk.img --device virtio-serial,logFilePath=/Users/nir/.minikube/machines/vfkit/serial.log

% tail /Library/Logs/vmnet-broker/vmnet-broker.log
INFO  [main] starting version=v0.3.0 commit=a459afb853e8cb06e0fdb28df3350d4859d6a460 pid=48708
DEBUG [main] setting up signal handlers
DEBUG [main] setting up listener
INFO  [peer 48707] connected (connected peers 1)
DEBUG [peer 48707] starting transaction to prevent termination while peers are connected
INFO  [peer 48707] created network 'shared' subnet '192.168.64.1' mask '255.255.255.0' ipv6_prefix 'fdef:a4a1:e866:3c8e::' prefix_len 64
INFO  [peer 48707] acquired network 'shared' (peers 1)
DEBUG [peer 48707] send network 'shared' to peer

Testing krunkit

% minikube start --no-kubernetes -p krunkit --driver krunkit              
😄  [krunkit] minikube v1.37.0 on Darwin 26.2 (arm64)
✨  Using the krunkit (experimental) driver based on user configuration
❗  Starting v1.39.0, minikube will default to "containerd" container runtime. See #21973 for more info.
👍  Starting minikube without Kubernetes in cluster krunkit
🔥  Creating krunkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
🐳  Preparing Docker 28.5.2 ...
🏄  Done! minikube is ready without Kubernetes!

% jq .Driver.VmnetHelper ~/.minikube/machines/krunkit/config.json
{
  "MachineDir": "/Users/nir/.minikube/machines/krunkit",
  "InterfaceID": "123b7f66-ee7e-4875-a5e2-d67a69a092da",
  "NetworkName": "vmnet:shared",
  "Offloading": false,
  "NeedsSudo": false
}

% minikube logs | grep vmnet-helper
I0126 17:23:43.320499   48849 main.go:144] libmachine: Validated vmnet-helper (path="/opt/vmnet-helper/bin/vmnet-helper", version="v0.10.0-pre1", commit="6e0c4a8194e036b998f9c7745953880ab77baa54", needsSudo=false, supportsNetwork=true, brokerInstalled=true)
I0126 17:23:43.611203   48849 main.go:144] libmachine: Started /opt/vmnet-helper/bin/vmnet-helper --socket /Users/nir/.minikube/machines/krunkit/vmnet-helper.sock --network shared (pid=48854)
I0126 17:23:43.628791   48849 main.go:144] libmachine: executing: krunkit --memory 6144 --cpus 2 --restful-uri unix:///Users/nir/.minikube/machines/krunkit/krunkit.sock --device virtio-net,type=unixgram,path=/Users/nir/.minikube/machines/krunkit/vmnet-helper.sock,mac=c6:be:2a:14:39:ff,offloading=false --device virtio-serial,logFilePath=/Users/nir/.minikube/machines/krunkit/serial.log --krun-log-level 3 --device virtio-blk,path=/Users/nir/.minikube/machines/krunkit/boot2docker.iso --device virtio-blk,path=/Users/nir/.minikube/machines/krunkit/disk.img

% tail -20 /Library/Logs/vmnet-broker/vmnet-broker.log
INFO  [main] starting version=v0.3.0 commit=a459afb853e8cb06e0fdb28df3350d4859d6a460 pid=48708
DEBUG [main] setting up signal handlers
DEBUG [main] setting up listener
INFO  [peer 48707] connected (connected peers 1)
DEBUG [peer 48707] starting transaction to prevent termination while peers are connected
INFO  [peer 48707] created network 'shared' subnet '192.168.64.1' mask '255.255.255.0' ipv6_prefix 'fdef:a4a1:e866:3c8e::' prefix_len 64
INFO  [peer 48707] acquired network 'shared' (peers 1)
DEBUG [peer 48707] send network 'shared' to peer
INFO  [peer 48854] connected (connected peers 2)
INFO  [peer 48854] acquired network 'shared' (peers 2)
DEBUG [peer 48854] send network 'shared' to peer

VM-to-VM communication

% minikube profile list
E0126 17:25:36.855232   48928 status.go:457] kubeconfig endpoint: get endpoint: "krunkit" does not appear in /Users/nir/.kube/config
E0126 17:25:36.898160   48928 status.go:457] kubeconfig endpoint: get endpoint: "vfkit" does not appear in /Users/nir/.kube/config
┌─────────┬─────────┬─────────┬───────────────┬─────────┬─────────┬───────┬────────────────┬────────────────────┐
│ PROFILE │ DRIVER  │ RUNTIME │      IP       │ VERSION │ STATUS  │ NODES │ ACTIVE PROFILE │ ACTIVE KUBECONTEXT │
├─────────┼─────────┼─────────┼───────────────┼─────────┼─────────┼───────┼────────────────┼────────────────────┤
│ krunkit │ krunkit │ docker  │ 192.168.64.24 │ N/A     │ Stopped │ 1     │                │                    │
│ vfkit   │ vfkit   │ docker  │ 192.168.64.23 │ N/A     │ Stopped │ 1     │                │                    │
└─────────┴─────────┴─────────┴───────────────┴─────────┴─────────┴───────┴────────────────┴────────────────────┘

% minikube ssh -p vfkit -- ping -c1 192.168.64.24
PING 192.168.64.24 (192.168.64.24): 56 data bytes
64 bytes from 192.168.64.24: seq=0 ttl=64 time=0.408 ms

--- 192.168.64.24 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.408/0.408/0.408 ms

% minikube ssh -p krunkit -- ping -c1 192.168.64.23
PING 192.168.64.23 (192.168.64.23): 56 data bytes
64 bytes from 192.168.64.23: seq=0 ttl=64 time=0.322 ms

--- 192.168.64.23 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.322/0.322/0.322 ms

Restart cluster flow

% minikube start --no-kubernetes --driver vfkit --network vmnet-shared
bck-i-search: minikube start_
nir@space-gray minikube (vmnet-network-support)% minikube start --no-kubernetes --driver vfkit --network vmnet-shared
😄  minikube v1.37.0 on Darwin 26.2 (arm64)
✨  Using the vfkit driver based on user configuration
❗  Starting v1.39.0, minikube will default to "containerd" container runtime. See #21973 for more info.
👍  Starting minikube without Kubernetes in cluster minikube
🔥  Creating vfkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
🐳  Preparing Docker 28.5.2 ...
🏄  Done! minikube is ready without Kubernetes!

% minikube ip
192.168.64.29

% minikube stop
✋  Stopping node "minikube"  ...
🛑  1 node stopped.

% minikube start
😄  minikube v1.37.0 on Darwin 26.2 (arm64)
✨  Using the vfkit driver based on existing profile
👍  Starting minikube without Kubernetes in cluster minikube
🔄  Restarting existing vfkit VM for "minikube" ...
🐳  Preparing Docker 28.5.2 ...
🏄  Done! minikube is ready without Kubernetes!

% minikube ip
192.168.64.29

TODO

  • Support for --network "vmnet:" in vfkit start flags
  • Support for --network "vmnet:" in krunkit start flags (currently ignores --network flag)
  • Normalize vmnet-shared to vmnet:shared in start flags instead of registry
  • Add unit tests for detection functions
  • Test upgrade flows (old cluster with new minikube)
  • Test edge cases (broker not running, helper downgrade)

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 26, 2026
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 26, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nirs
Once this PR has been reviewed and has the lgtm label, please assign prezha for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from medyagh January 26, 2026 15:48
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 26, 2026
@nirs nirs force-pushed the vmnet-network-support branch from 4727b56 to 9196423 Compare January 26, 2026 16:54
nirs added 5 commits January 26, 2026 20:49
Move the duplicated generateMACAddress() function from vfkit and qemu2
packages to pkg/drivers/common as an exported function. This reduces
code duplication and ensures consistent MAC address generation across
all drivers.
Extend interfaceInfo struct to parse additional fields from vmnet-helper
JSON response and log them for easier debugging. The address range and
subnet mask help diagnose network configuration issues.
Add functions to detect vmnet-broker installation and vmnet-helper
--network option support:

- brokerInstalled(): checks if vmnet-broker launchd plist exists
- helperSupportsNetwork(): parses --help output for --network option
- SupportsNetwork field in helperInfo for caching detection result

This prepares for using vmnet-broker managed networks when available.
Add NetworkName field to Helper struct to specify the vmnet network.
The format is "vmnet:name" where name is passed to vmnet-helper
--network option when using broker mode.

Update drivers:
- vfkit: support both "vmnet-shared" (legacy) and "vmnet:<name>" format,
  normalize legacy format to "vmnet:shared"
- krunkit: always use vmnet with default "vmnet:shared"

Empty NetworkName is treated as "vmnet:shared" for backward compatibility
with machines created by older minikube versions.
Add support for vmnet-broker managed networks. When vmnet-broker is
installed and vmnet-helper supports the --network option, minikube uses
network mode instead of interface-id mode.

Interface-id mode (existing):
- Driver generates UUID, passes to vmnet-helper --interface-id
- vmnet generates deterministic MAC from UUID
- MAC returned in JSON response, stored in driver

Network mode (new):
- Driver generates random MAC address
- vmnet-helper uses --network option with broker
- Broker manages shared network across VMs
- VMs can communicate with each other

The mode is selected automatically based on:
- vmnet.GeneratesMACAddress(): returns true when vmnet generates MAC
  (interface-id mode), false when driver should generate (network mode)
- Drivers call this in configure() to decide UUID vs MAC generation
- Fallback to interface-id mode if requirements not met
@nirs nirs force-pushed the vmnet-network-support branch from 9196423 to bf3d2a9 Compare January 26, 2026 18:49
@k8s-triage-robot
Copy link
Copy Markdown

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 26, 2026
@nirs
Copy link
Copy Markdown
Contributor Author

nirs commented Apr 26, 2026

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 26, 2026
@nirs
Copy link
Copy Markdown
Contributor Author

nirs commented Apr 26, 2026

/lifcycle-frozen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants