interfaces/builtin: add vsock interface for VM guest services - #16963
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #16963 +/- ##
=======================================
Coverage 78.86% 78.86%
=======================================
Files 1395 1397 +2
Lines 195858 195802 -56
Branches 2466 2466
=======================================
- Hits 154469 154426 -43
+ Misses 32121 32103 -18
- Partials 9268 9273 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Fri Jul 10 10:39:59 UTC 2026 Failures:Preparing:
Executing:
Restoring:
Skipped tests from snapd-testing-skipIf you wish to have any of the below tests run in your PR, in your PR description, add 'unskip:' followed by a copy-and-pasted list of the below tests you wish to run (unskip plus test list must be valid yaml)
|
|
|
||
| # Allow binding on vsock sockets (AF_VSOCK) for VM/container guest services. | ||
| # AF_VSOCK socket creation is permitted by the default seccomp template. | ||
| network vsock, |
There was a problem hiding this comment.
This is a definite nack as it would allow nearly any snap talk to any snap and to the hypervisor without any oversight.
This needs to be a different interface and we may need to have filtering on which CID you can actually talk to.
There was a problem hiding this comment.
i created a separate vsock interface.
c3a76de to
a684831
Compare
bugraaydogar
changed the title
| # VM guests and the host or hypervisor (AF_VSOCK). | ||
| network vsock, | ||
| /dev/vsock rw, | ||
| /dev/vmci rw, |
There was a problem hiding this comment.
hm this one is vmware specific, why include it here?
There was a problem hiding this comment.
because the customer is trying to snap open-vm-tools and have observed the access denials. There is no other interface that allows access to vsock and vmci so i thought that would be a meaningful interface to add vmci as well. Otherwise, they would need to use custom-device interface for /dev/vmci
There was a problem hiding this comment.
I wonder if we should perhaps
- add devices used by other virtualization stacks for similar purposes here, such as
/dev/vhost-vsockor - change the interface name to make it vmware specific.
@pedronis any suggestion?
There was a problem hiding this comment.
Through vsock you can only obtain the CID, whereas vhost-vsock exposes a bit more. I think it fits better within the scope of *-support interfaces, e.g. we expose it in microstack-support already.
There was a problem hiding this comment.
Right, vhost-vsock is more the counterpart of /dev/vmci, isn't it?. In that sense I think we should either
- add both ( /dev/vmci and /dev/vhost-vsock )
- add none of them (and grant access to /dev/vmc via custom-device)
- rename the interface to make it clear it is vmware specific
There was a problem hiding this comment.
For what I've seen, /dev/vmci is used by both, guest and host. The main difference is that this device node is backed by different kernel modules on these two scenarios. On "guest mode", it seems that vmci is used as the transport layer for vsock traffic (among other things).
Whilst vmci is needed for the transport layer, I don't know if vsock communication could possibly work without explicit access to the device node, /dev/vmci. On the other hand, /dev/vmci is possibly still needed by open-vm-tools to provide some functionality not carried over vsock.
If we can check that access to /dev/vmci is not required for vsock communication on vmware systems, I would vote for keeping this access out of the interface and use custom-device for it. If this ends up being a hard requirement, I think it will make sense to include it here.
There was a problem hiding this comment.
to sum up:
- /dev/vmci - used by both host and guest, pretty wide functional surface, cannot be mediated easily
- /dev/vsock - used by guest (and or host too), a single ioctl https://elixir.bootlin.com/linux/v7.1.2/source/net/vmw_vsock/af_vsock.c#L2739
- /dev/vhost-vsock - used by the host to set up guests, bunch of ioctls: https://elixir.bootlin.com/linux/v7.1.2/source/drivers/vhost/vsock.c#L873, not accessible from the guest, unless the guest itself has nested VMs and effectively acts as another level host
- AF_VSOCK once you open a socket, you can only talk to the host, there's no default guest-to-guest bridging. The guest can reach out to host to any port, assuming the host has a process listening on AF_VSOCK
TBH, what if we consider this:
- splitting up vsock access to e.g. vsock-guest interface - allow /dev/vsock, allow AF_VSOCK, IMO does not need to be superprivielged
- the vhost-vsock is already part of microstack, but we can add another targeted interface if needed, not part of this PR anyway
- move vmci into e.g. vmware-vmci, make it superprivileged, allow AF_VSOCK, allow /dev/vmci rw access
There was a problem hiding this comment.
I think I mostly agree
splitting up vsock access to e.g. vsock-guest interface - allow /dev/vsock, allow AF_VSOCK, IMO does not need to be superprivielged
I would still make it privileged (maybe not super-privileged) as I don't think there are many snap that need it for legitimate reasons nowadays.
the vhost-vsock is already part of microstack, but we can add another targeted interface if needed, not part of this PR anyway
+1
move vmci into e.g. vmware-vmci, make it superprivileged, allow AF_VSOCK, allow /dev/vmci rw access
I wander whether access to /dev/vmci will endup being a one-off just for open-vm-tools. I wonder whether vsock interface + access to /dev/vmci via custom-device could be preferable over a dedicated interface for now. That's mainly a design decision, I have no strong opinion on this.
There was a problem hiding this comment.
@bugraaydogar would this work for you? making the interface privileged, renaming to e.g. vsock-guest and limiting it to allow acecss to /dev/vsock + AF_VSOCK sockets & acecpt/bind/listen etc. and then custom-device for the snap in question to allow /dev/vmci
There was a problem hiding this comment.
@bboozzoo thanks for the input. I updated as requested. Please let me know if you have any further finding.
| const vsockBaseDeclarationPlugs = ` | ||
| vsock: | ||
| allow-installation: false | ||
| deny-auto-connection: true |
There was a problem hiding this comment.
I don't quite agree with @zyga's initial review. If anything, it's not much different from network access, which we auto connect by default. In fact, VM ports could already be exposed and just by adding the network plug snap would already be able to connect.
There was a problem hiding this comment.
@bboozzoo could you please add a security reviewer so that we could get a final input from the security professional?
There was a problem hiding this comment.
Hi @jslarraz
Sorry for the ping. Could you please review the PR? I'm trying to push these changes for the upcoming snapd 2.76 release as requested by the customer.
There was a problem hiding this comment.
Making this interface super-privileged makes sense to me because:
- In contrast to
network, this interface is really niche, I don't expect many snaps requiring legitimate access to it. - It increases the attack surface granting access to a kernel module, which possibly is not as widely tested as others
bboozzoo
added
the
Needs security review
| const vsockBaseDeclarationPlugs = ` | ||
| vsock: | ||
| allow-installation: false | ||
| deny-auto-connection: true |
There was a problem hiding this comment.
Making this interface super-privileged makes sense to me because:
- In contrast to
network, this interface is really niche, I don't expect many snaps requiring legitimate access to it. - It increases the attack surface granting access to a kernel module, which possibly is not as widely tested as others
| # VM guests and the host or hypervisor (AF_VSOCK). | ||
| network vsock, | ||
| /dev/vsock rw, | ||
| /dev/vmci rw, |
There was a problem hiding this comment.
I wonder if we should perhaps
- add devices used by other virtualization stacks for similar purposes here, such as
/dev/vhost-vsockor - change the interface name to make it vmware specific.
@pedronis any suggestion?
| # Description: Allow access to vsock sockets for VM/container host communication. | ||
| # socket AF_VSOCK is already permitted by the default seccomp template. | ||
| bind | ||
| listen |
There was a problem hiding this comment.
I expected only bind to be needed on the guest side, and listen and accept only be required by the host piece. Did you run into issues without these other syscalls?
There was a problem hiding this comment.
It's really like network and works in both directions, both guest and host can expose services for each other.
There was a problem hiding this comment.
In that sense, the relevant side could plug network-bind independently if needed
There was a problem hiding this comment.
Thanks for info @bboozzoo. Then, I think it is also fine if we keep them here
a684831 to
390fea4
Compare
|
@bugraaydogar @jslarraz @bboozzoo I would like to perhaps limit the CIDs we can talk to. Specifically talking to the hypervisor is probably a special case that we could deny. As for keeping it for remaining use cases, I think it's okay if it is super-privileged. This effectively lets you talk to any VM on the host and between any snap on the loopback interface. It is true that network allows that but not because we wanted to, but because we didn't have anything better to filter with. I wonder if we need the device access. IIRC there's some ioctl that's kind of obsolete now (to know one's address). |
|
@zyga Let me clarify the use case since I think we might be looking at this from different angles. The primary customer use case is Ubuntu Core running as a guest inside a hypervisor. Snaps inside the guest use vsock to communicate up to the hypervisor for guest utilities. That being said, i don't think we should be blocking any other use case. It is going to be a super-privileged interface and developers must already know and understand the overall threats within their solutions. I'm not favor of increasing the complexity of the interface however, the decision is on the security team and they are already fine with the existing implementation unless they changed their ideas. |
|
I don't have any reservations for a privileged interface. |
390fea4 to
12b665e
Compare
|
12b665e to
91d5f23
Compare
91d5f23 to
1278bc0
Compare
jslarraz
removed
the
Needs security review
bboozzoo
added
the
Auto rerun spread
| const vsockGuestConnectedPlugSecComp = ` | ||
| # Description: Allow access to vsock sockets for VM guest to host communication. | ||
| # socket AF_VSOCK is already permitted by the default seccomp template. | ||
| bind |
There was a problem hiding this comment.
do we need ioctl IOCTL_VM_SOCKETS_GET_LOCAL_CID? I would think so.
There was a problem hiding this comment.
I guess, we only deny TIOCSTI & TIOCLINUX ioctl calls. See => https://github.com/canonical/snapd/blob/master/interfaces/seccomp/template.go#L228
There was a problem hiding this comment.
yes, it should be working already, we allow all ioctls but the ones that allow nasty things
6 participants
A customer would like to use open-vm-tools heavily relies on the vsock interface for communication between hypervisor and guest. They were using
custom-deviceinterface to accessdevfiles like/dev/vsockand/dev/vmci.However, they also realized that they are getting denial due to missing
vsocksupport. Thecustom-deviceinterface does not allow addingnetwork vsock. This PR aims to solve this problem.Denied Messages:
Internal reference: LP#2147028