multiple issues with running hyperkit driver as root to use vmnet framework to obtain IP address

[cequencer]

Thank you team for the hard work on this project.

I am running into multiple issues having hyperkit working correctly. I am putting everything together because I believe the root issue is that hyperkit driver is not working correctly with vmnet framework. I am attempting to diagnose the problem without much prior knowledge in this project, hence capturing what I can gather so I can get second opinion from the pros.

At high level, I am trying to create a docker-machine client using hyperkit driver on a clean macOS Mojave Version 10.14.6 using brew to install hyperkit, docker, and docker-machine.

$ docker-machine create --driver hyperkit client-hyperkit-vm
Running pre-create checks...
Creating machine...
(client-hyperkit-vm) Copying /Users/cequencer/.docker/machine/cache/boot2docker.iso to /Users/cequencer/.docker/machine/machines/client-hyperkit-vm/boot2docker.iso...
(client-hyperkit-vm) /dev/disk2          	                               	/Users/cequencer/.docker/machine/machines/client-hyperkit-vm/b2d-image
(client-hyperkit-vm) "disk2" ejected.
(client-hyperkit-vm) Using UUID
(client-hyperkit-vm) Generated MAC 6:a4:b2:15:e8:ad
(client-hyperkit-vm) Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 waitusb=5:LABEL=boot2docker-data base norestore noembed
Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary Error: Could not find an IP address for 6:a4:b2:15:e8:ad
Temporary Error: Could not find an IP address for 6:a4:b2:15:e8:ad

$ docker-machine ls
NAME                 ACTIVE   DRIVER     STATE     URL   SWARM   DOCKER    ERRORS
client-hyperkit-vm   -        hyperkit   Stopped                 Unknown   IP address is not set

The interesting part is that the interface with the above MAC address does not exist:

$ ifconfig -a | grep -i '6:a4:b2:15:e8:ad' | wc -l
       0

My current suspicion references to the requirement identified in the README.md file for this project. It outlines the following requirement, but no exact instruction is provided on how to run the hyperkit driver as root to use the vmnet framework to setup networking correctly. Which is why I believe the hyperkit driver could not obtain an IP address for the MAC that it generated in my original problem capture.

In the post install notification presented below brew install docker-machine-driver-hyperkit:

$ brew install docker-machine-driver-hyperkit
==> Downloading https://homebrew.bintray.com/bottles/docker-machine-driver-hyperkit-1.0.0.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/1b/1b3ba8ce6ae05b27463ef2b8ebfbdeec911a0b6f1ba20188279b79dac81b4754?__gda__=exp=1569428954~hmac=16f334dfac70d76b
######################################################################## 100.0%
==> Pouring docker-machine-driver-hyperkit-1.0.0.mojave.bottle.tar.gz
==> Caveats
This driver requires superuser privileges to access the hypervisor. To
enable, execute:
  sudo chown root:wheel /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
  sudo chmod u+s /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
==> Summary
🍺  /usr/local/Cellar/docker-machine-driver-hyperkit/1.0.0: 5 files, 13.5MB

I have performed the required action to grant superuser privileges to hyperkit.

$ ls -al /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
-r-sr-xr-x  1 root  wheel  14116260 Mar  1  2018 /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
$ file /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
/usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit: setuid Mach-O 64-bit executable x86_64
$ md5 /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
MD5 (/usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit) = d6fb6618667ad60eabbf3bf8190ca5fc

Extra details of the package that was installed for lineage tracking.

$ brew info docker-machine-driver-hyperkit
docker-machine-driver-hyperkit: stable 1.0.0 (bottled)
Docker Machine driver for hyperkit
https://github.com/machine-drivers/docker-machine-driver-hyperkit
/usr/local/Cellar/docker-machine-driver-hyperkit/1.0.0 (5 files, 13.5MB) *
  Poured from bottle on 2019-09-25 at 09:17:15
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/docker-machine-driver-hyperkit.rb
==> Dependencies
Build: dep ✘, go ✘
Required: docker-machine ✔
==> Requirements
Required: macOS >= 10.10 ✔
==> Caveats
This driver requires superuser privileges to access the hypervisor. To
enable, execute:
  sudo chown root:wheel /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
  sudo chmod u+s /usr/local/opt/docker-machine-driver-hyperkit/bin/docker-machine-driver-hyperkit
==> Analytics
install: 1,037 (30 days), 3,249 (90 days), 14,326 (365 days)
install_on_request: 1,037 (30 days), 3,247 (90 days), 14,319 (365 days)
build_error: 0 (30 days)

Additional info for the installed binary driver in question:

$ which docker-machine-driver-hyperkit
/usr/local/bin/docker-machine-driver-hyperkit

$ file /usr/local/bin/docker-machine-driver-hyperkit
/usr/local/bin/docker-machine-driver-hyperkit: setuid Mach-O 64-bit executable x86_64

$ docker-machine-driver-hyperkit version
This is a Docker Machine plugin binary.
Plugin binaries are not intended to be invoked directly.
Please use this plugin through the main 'docker-machine' binary.
(API version: 1)

Another issue is that I can not kill this client machine and it reports an exit code of 1:

$ docker-machine kill client-hyperkit-vm
Killing "client-hyperkit-vm"...
Machine "client-hyperkit-vm" is already stopped.
$ echo $?
1
$ docker-machine status client-hyperkit-vm
Stopped
$ echo $?
0
$ docker-machine kill client-hyperkit-vm
Killing "client-hyperkit-vm"...
Machine "client-hyperkit-vm" is already stopped.
$ echo $?
1
$ docker-machine config client-hyperkit-vm
Error running connection boilerplate: IP address is not set
$ echo $?
1
$ docker-machine inspect client-hyperkit-vm
{
    "ConfigVersion": 3,
    "Driver": {
        "IPAddress": "",
        "MachineName": "client-hyperkit-vm",
        "SSHUser": "",
        "SSHPort": 0,
        "SSHKeyPath": "",
        "StorePath": "/Users/cequencer/.docker/machine",
        "SwarmMaster": false,
        "SwarmHost": "",
        "SwarmDiscovery": "",
        "Boot2DockerURL": "",
        "DiskSize": 0,
        "CPU": 0,
        "Memory": 0,
        "Cmdline": "",
        "NFSShares": null,
        "NFSSharesRoot": "",
        "UUID": "",
        "BootKernel": "",
        "BootInitrd": "",
        "Initrd": "",
        "Vmlinuz": ""
    },
    "DriverName": "hyperkit",
    "HostOptions": {
        "Driver": "",
        "Memory": 0,
        "Disk": 0,
        "EngineOptions": {
            "ArbitraryFlags": [],
            "Dns": null,
            "GraphDir": "",
            "Env": [],
            "Ipv6": false,
            "InsecureRegistry": [],
            "Labels": [],
            "LogLevel": "",
            "StorageDriver": "",
            "SelinuxEnabled": false,
            "TlsVerify": true,
            "RegistryMirror": [],
            "InstallURL": "https://get.docker.com"
        },
        "SwarmOptions": {
            "IsSwarm": false,
            "Address": "",
            "Discovery": "",
            "Agent": false,
            "Master": false,
            "Host": "tcp://0.0.0.0:3376",
            "Image": "swarm:latest",
            "Strategy": "spread",
            "Heartbeat": 0,
            "Overcommit": 0,
            "ArbitraryFlags": [],
            "ArbitraryJoinFlags": [],
            "Env": null,
            "IsExperimental": false
        },
        "AuthOptions": {
            "CertDir": "/Users/cequencer/.docker/machine/certs",
            "CaCertPath": "/Users/cequencer/.docker/machine/certs/ca.pem",
            "CaPrivateKeyPath": "/Users/cequencer/.docker/machine/certs/ca-key.pem",
            "CaCertRemotePath": "",
            "ServerCertPath": "/Users/cequencer/.docker/machine/machines/client-hyperkit-vm/server.pem",
            "ServerKeyPath": "/Users/cequencer/.docker/machine/machines/client-hyperkit-vm/server-key.pem",
            "ClientKeyPath": "/Users/cequencer/.docker/machine/certs/key.pem",
            "ServerCertRemotePath": "",
            "ServerKeyRemotePath": "",
            "ClientCertPath": "/Users/cequencer/.docker/machine/certs/cert.pem",
            "ServerCertSANs": [],
            "StorePath": "/Users/cequencer/.docker/machine/machines/client-hyperkit-vm"
        }
    },
    "Name": "client-hyperkit-vm"
}

Extra additional runtime environment information of the host:

$ uname -a
Darwin macbookpro 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

Using Hyperkit to query the MAC address show no results and exited with error condition:

$ which hyperkit
/usr/local/bin/hyperkit
$ ls -al /usr/local/bin/hyperkit
lrwxr-xr-x  1 cequencer  admin  42 Sep 23 06:06 /usr/local/bin/hyperkit -> ../Cellar/hyperkit/0.20190802/bin/hyperkit
$ hyperkit -v
hyperkit: 0.20190802

Homepage: https://github.com/docker/hyperkit
License: BSD
$ hyperkit -M
Usage: hyperkit [-behuwxMACHPWY] [-c vcpus] [-F <pidfile>] [-g <gdb port>] [-l <lpc>]
                [-m mem] [-p vcpu:hostcpu] [-s <pci>] [-U uuid] -f <fw>
       -A: create ACPI tables
       -c: # cpus (default 1)
       -C: include guest memory in core file
       -e: exit on unhandled I/O access
       -f: firmware
       -F: pidfile
       -g: gdb port
       -h: help
       -H: vmexit from the guest on hlt
       -l: LPC device configuration. Ex: -l com1,stdio -l com2,autopty -l com2,/dev/myownpty
       -m: memory size in MB, may be suffixed with one of K, M, G or T
       -M: print MAC address and exit if using vmnet
       -P: vmexit from the guest on pause
       -s: <slot,driver,configinfo> PCI slot config
       -u: RTC keeps UTC time
       -U: uuid
       -v: show build version
       -w: ignore unimplemented MSRs
       -W: force virtio to use single-vector MSI
       -x: local apic is in x2APIC mode
       -Y: disable MPtable generation
$ echo $?
1

[cequencer]

I learned that there are three different versions of the hyperkit driver an user may have from this issue. The recommended approach is not to install it via brew, instead follow the instructions here.

From that issue I learned about the big pull request #13 that contains network code changes that could potentially fix this issue. I guess for now the only way to find out is to wait for this pull request #13 to complete and the latest stable release made available in brew upgrade to verify.