Build instructions

[travisghansen]

Question

I don’t do much with mac so forgive the ignorance. I have signed up for an apple developer account etc and have created what I believe to the correct assets for properly signing the app. Codesign seems to execute properly and verify comes back clean but I still get an error about missing the virtualization entitlement.

Any tips?

This feels a bit out of scope but any detailed instructions would be fantastic (or simply publishing the app so I we don’t have to do all this craziness). I am sure it’s mostly ignorance on my part but the barrier seems quite high to get ot functional.

[Raikerian]

You need to use this entitlements https://github.com/agoda-com/macOS-vz-kubelet/blob/main/resources/vz.entitlements (providing you don't need direct network card access, as that one is more complicated).
Our goreleaser config provides code signing with entitlements for you https://github.com/agoda-com/macOS-vz-kubelet/blob/main/.goreleaser.yml#L26
Just replace release.entitlements with vz.entitlements for your case.

[Raikerian]

I think in the case where you don't need com.apple.vm.networking entitlement, you also don't need to setup app id or provisioning profile. This also removes a need for an .app folder structure. Just code signing your binary with entitlements should be enough.

[travisghansen]

Yeah, I’ve tried a bunch of different things and none of them really are work. I’ll keep digging.

Any reason you don’t publish distributable binaries?

[travisghansen]

Indeed signing with only the vz.entitlements seemed to help. I have made further progress now and presumable VMs are running or it logs that they are.

[Raikerian]

Any reason you don’t publish distributable binaries?

Mainly due to apple code signing process.

[knfoo]

@travisghansen can you share what you did to get that far ?

[Raikerian]

Hi @knfoo ,

Can you please describe what kind of issues you are experiencing, I will try to help. Generally, just build it like normal go binary, the only difference is the extra step after that to sign it:

codesign --entitlements ./resources/vz.entitlements -s - <your_binary_path>

[knfoo]

Disclaimer, I am not a Mac user and have never made an application run on a Mac, so I am in very new to this.

I did build the go binary like this:
go build -o macOS-vz-kubelet ./cmd/virtual-kubelet/

But when I run the codesign command I get this error:

codesign --entitlements ./resources/vz.entitlements -s - macOS-vz-kubelet

macOS-vz-kubelet: bundle format unrecognized, invalid, or unsuitable
In subcomponent: /Users/macdkc01/kubelet/macOS-vz-kubelet/.go

I am able to run the binary though:

/macOS-vz-kubelet 

WARN[0000] failed to create docker client: failed to ping docker client: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?; some features (like non-macOS containers) will be unavailable 
WARN[0000] Failed to create container client             err="must provide a containers client" method=VZClient.NewVzClientAPIs
INFO[0000] Pod cache in-sync                            
INFO[0000] starting workers                             
INFO[0000] started workers                              

I am not sure if it is needed to get the codesign to work as I have not tried building a VM nor tried to start one.

[travisghansen]

@knfoo you need to create (or import) a certificate authority that your machine(s) trust. You can build one like this: https://www.simplified.guide/macos/keychain-cert-code-signing-create

And then sign the binary with that cert..

codesign --deep -s '<name of the cert created using the steps in the link above>' --entitlements ${PWD}/resources/vz.entitlements --force -vvvv ${PWD}/virtual-kubelet

[knfoo]

@travisghansen that seems to do the trick now I got a little further.

My macos pod starts and I can kubectl exec into it.

Is there a way to attach to a vnc or other gui on the macs VM's ?

[travisghansen]

@knfoo if you run the vnc client from the host machine sure. Just get the PodIP (which is the VM IP) and connect to it.

You need to make sure VNC (or whatever remote protocol you use) is enabled obviously in the image and configure to start at boot etc.

If you want to be able to attach from a machine other than your host machine then you will likely need to use the bridged networking config VZ_BRIDGE_INTERFACE but doing that is much more involved as you need the additional entitlements which requires an appid and apple account etc.

[knfoo]

@travisghansen Thanks for your quick and detailed response 🤗