Add support for virtio device reset by ilstam · Pull Request #5891 · firecracker-microvm/firecracker

ilstam · 2026-05-15T16:36:36Z

Add support for virtio device reset

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

This functionality cannot be added in rust-vmm.

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

codecov · 2026-05-15T16:45:59Z

Codecov Report

❌ Patch coverage is 30.43478% with 96 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.80%. Comparing base (01f90a8) to head (6f0ab8e).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/vmm/src/devices/virtio/vsock/unix/muxer.rs	0.00%	19 Missing ⚠️
src/vmm/src/devices/virtio/block/virtio/device.rs	0.00%	11 Missing ⚠️
src/vmm/src/devices/virtio/block/device.rs	0.00%	10 Missing ⚠️
src/vmm/src/devices/virtio/balloon/device.rs	0.00%	9 Missing ⚠️
src/vmm/src/devices/virtio/vsock/device.rs	0.00%	9 Missing ⚠️
src/vmm/src/devices/virtio/vsock/packet.rs	0.00%	8 Missing ⚠️
.../vmm/src/devices/virtio/block/vhost_user/device.rs	0.00%	6 Missing ⚠️
...vmm/src/devices/virtio/block/virtio/io/async_io.rs	0.00%	6 Missing ⚠️
src/vmm/src/devices/virtio/mem/device.rs	0.00%	6 Missing ⚠️
src/vmm/src/devices/virtio/pmem/device.rs	0.00%	6 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5891      +/-   ##
==========================================
- Coverage   83.01%   82.80%   -0.21%     
==========================================
  Files         277      277              
  Lines       30129    30221      +92     
==========================================
+ Hits        25012    25025      +13     
- Misses       5117     5196      +79

Flag	Coverage Δ
5.10-m5n.metal	`83.08% <30.43%> (-0.24%)`	⬇️
5.10-m6a.metal	`82.42% <30.43%> (-0.24%)`	⬇️
5.10-m6g.metal	`79.77% <30.43%> (-0.19%)`	⬇️
5.10-m6i.metal	`83.08% <30.43%> (-0.24%)`	⬇️
5.10-m7a.metal-48xl	`82.42% <30.43%> (-0.24%)`	⬇️
5.10-m7g.metal	`79.78% <30.43%> (-0.18%)`	⬇️
5.10-m7i.metal-24xl	`83.06% <30.43%> (-0.23%)`	⬇️
5.10-m7i.metal-48xl	`83.06% <30.43%> (-0.23%)`	⬇️
5.10-m8g.metal-24xl	`79.77% <30.43%> (-0.18%)`	⬇️
5.10-m8g.metal-48xl	`79.77% <30.43%> (-0.18%)`	⬇️
5.10-m8i.metal-48xl	`83.06% <30.43%> (-0.24%)`	⬇️
5.10-m8i.metal-96xl	`83.06% <30.43%> (-0.24%)`	⬇️
6.1-m5n.metal	`83.11% <30.43%> (-0.24%)`	⬇️
6.1-m6a.metal	`82.45% <30.43%> (-0.24%)`	⬇️
6.1-m6g.metal	`79.78% <30.43%> (-0.18%)`	⬇️
6.1-m6i.metal	`83.11% <30.43%> (-0.23%)`	⬇️
6.1-m7a.metal-48xl	`82.44% <30.43%> (-0.24%)`	⬇️
6.1-m7g.metal	`79.77% <30.43%> (-0.19%)`	⬇️
6.1-m7i.metal-24xl	`83.12% <30.43%> (-0.24%)`	⬇️
6.1-m7i.metal-48xl	`83.12% <30.43%> (-0.24%)`	⬇️
6.1-m8g.metal-24xl	`79.77% <30.43%> (-0.18%)`	⬇️
6.1-m8g.metal-48xl	`79.77% <30.43%> (-0.18%)`	⬇️
6.1-m8i.metal-48xl	`83.12% <30.43%> (-0.24%)`	⬇️
6.1-m8i.metal-96xl	`83.12% <30.43%> (-0.23%)`	⬇️
6.18-m5n.metal	`83.11% <30.43%> (-0.24%)`	⬇️
6.18-m6a.metal	`82.45% <30.43%> (-0.24%)`	⬇️
6.18-m6g.metal	`79.78% <30.43%> (-0.18%)`	⬇️
6.18-m6i.metal	`83.11% <30.43%> (-0.24%)`	⬇️
6.18-m7a.metal-48xl	`82.44% <30.43%> (-0.24%)`	⬇️
6.18-m7g.metal	`79.77% <30.43%> (-0.19%)`	⬇️
6.18-m7i.metal-24xl	`83.12% <30.43%> (-0.24%)`	⬇️
6.18-m7i.metal-48xl	`83.12% <30.43%> (-0.25%)`	⬇️
6.18-m8g.metal-24xl	`79.77% <30.43%> (-0.18%)`	⬇️
6.18-m8g.metal-48xl	`79.77% <30.43%> (-0.18%)`	⬇️
6.18-m8i.metal-48xl	`83.12% <30.43%> (-0.23%)`	⬇️
6.18-m8i.metal-96xl	`83.12% <30.43%> (-0.24%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Manciukic · 2026-05-18T12:59:50Z

+        for queue in self.queues_mut() {
+            *queue = Queue::new(queue.max_size);
+        }
+        self._reset()


should this be called before the rest? This is a semantic change from the old code where a failed reset left the device activated.

It left it activated in Firecracker but set the state as FAILED so in theory the guest shouldn't continue using it. But does it matter much since we'll support reset for all devices except for vhost-user-block (for now)?

deactivate(), set_acked_features(0), and the queue wipe all run before _reset() is consulted. For a backend that returns false (vhost-user-block), the VMM-side state is already mutated while the external backend was never told to stop

Ok, I re-ordered the links and made reset() call _reset() first thing. However, obviously the device will end up in a FAILED state regardless.

Manciukic · 2026-05-18T13:01:56Z

        self.device_state.is_activated()
    }
+
+    fn deactivate(&mut self) {


what about the rate limiters? are we assuming we want to preserve the state across reset?

Should we not?

I think we must since otherwise guest can bypass rate limiter by resetting

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

bk0v · 2026-05-27T14:24:02Z

+        for queue in self.queues_mut() {
+            *queue = Queue::new(queue.max_size);
+        }
+        self._reset()


deactivate(), set_acked_features(0), and the queue wipe all run before _reset() is consulted. For a backend that returns false (vhost-user-block), the VMM-side state is already mutated while the external backend was never told to stop

bk0v · 2026-06-10T14:14:03Z

+    }
+
+    fn _reset(&mut self) -> bool {
+        false


Returning false here combined with the new MMIO path means any status=0 write after ACKNOWLEDGE permanently sets FAILED for vhost-user-block.

As noted in the other comment, even before this PR we'd set the device to FAILED state if we got a reset after DRIVER_OK. Does it matter now that this can happen after ACKNOWLEDGE?

bk0v · 2026-06-10T14:15:35Z

        } else if status == INIT {
-            {
-                let mut locked_device = self.device.lock().expect("Poisoned lock");
-                if locked_device.is_activated() {


Was dropping the is_activated() guard intentional? Previously the device-level reset only ran when is_activated(). Now it runs whenever device_status != INIT. Combined with _reset() -> false backends this set FAILED for resets issued during early init

It was intentional yes, as it was explicitly called out in the commit message. As far as I can tell the "activated" state is a Firecracker term rather than a virtio spec term. I don't think the spec says we must ignore resets before reaching a certain state such as DRIVER_OK. It's true that for vhost-user-block an early reset will lead to a FAILED state, but is this a problem? Already before this PR a reset after DRIVER_OK would lead to a FAILED state, does it matter that it now leads to a FAILED state even before DRIVER_OK?

bk0v · 2026-06-10T14:16:42Z

+    fn deactivate(&mut self);
+
+    /// Reset the device. Returns true on success, false otherwise.
+    /// It must not be overridden.


Should "must not be overridden" be enforced somehow?

I felt the same, but unfortunately Rust doesn't seem to have a final keyword like C++. I'm open to suggestions though if you can think of a different way to do it.

The reset() trait method returns Option<(Arc<dyn VirtioInterrupt>, Vec<EventFd>)> for no apparent reason and the values are never used by any caller. Change the return value to a bool to signify success or failure. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Simplify the MMIO transport reset logic by always performing the MMIO transport reset unconditionally and calling device.reset() regardless of activation state. This requires updating the affected unit tests and implementing reset() for the DummyDevice. The unit test assumed that reset() is never supported (i.e. always returns false), but that is going to change soon so we want to make sure that resetting moves the device to the deactivated state. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

The reset success path locked self.device, then tried to lock it again via self.virtio_device().lock() to reset queues. This was a deadlock that was never triggered because no device previously implemented reset() (all returned false). Use the already-held guard instead. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Don't clear virtio_interrupt on reset. The interrupt object holds Arc references to the PCI device's MSI-X configuration which remains valid across reset. Clearing it will cause a panic at the unwrap() in the activation path when the guest re-probes the device after reset. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Reset device_feature_select and driver_feature_select to 0 on device reset, matching what the MMIO transport does with features_select and acked_features_select. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add test_bus_device_reset_failure to verify that when device.reset() returns false, the FAILED bit is set in device_status. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a deactivate() method that sets the device state to Inactive. This will be used by the generic reset implementation in a subsequent patch. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Move the queue reset logic from the MMIO and PCI transport code into the default reset() implementation in the VirtioDevice trait. This is generic virtio state that should be reset for all devices, regardless of transport. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Complete the reset() implementation by deactivating the device and setting acked_features to 0. This must happen for all virtio backends. Introduce a _reset() method that is called at the beginning of reset() and can be overridden by virtio backends with backend specific reset code. Make all backends return false for now since none of them supports reset yet. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a clear() method to RxBuffers that resets all fields to their initial state. This will be used by the virtio-net reset implementation. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Implement the _reset() method for the virtio-net device, resetting the net specific state in-place. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-net device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Suggested-by: Adam Jensen <adam@acj.sh> Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Implement the _reset() method for the virtio-block device, resetting the device-specific state in-place. For the async io_uring engine, replace the ring with a fresh one so that any in-flight operations are cancelled by the kernel when the old ring fd is closed. This only adds support for the Virtio backend for now and not VhostUser. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-block device reset works end-to-end by unbinding and rebinding the guest driver for a scratch block device and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Pmem does not have any backend specific state that needs resetting so implement the _reset() method by simply returning true. The rest of the state is handled by the generic reset(). Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-pmem device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Implement the _reset() method for the virtio-balloon device, resetting the balloon specific state in-place. Do not deflate the balloon on reset. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-balloon device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

The entropy device does not have any backend specific state that needs resetting so implement the _reset() method by simply returning true. The test_failed_reset_blocks_reinitialization() test used an entropy device assuming it doesn't support reset. Since it now does, adjust the test to check that the device is first deactivated after a reset and then the driver can perform the initialization sequence again. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-rng device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Implement the _reset() method for the virtio-vsock device, resetting the vsock specific state in-place. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-vsock device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

The virtio-mem device does not have any backend specific state that needs resetting so implement the _reset() method by simply returning true. Plugged memory regions remain intact. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

Add a test that verifies virtio-mem device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam added a commit to ilstam/firecracker that referenced this pull request May 15, 2026

CHANGELOG: Mention virtio device reset support

e3917dd

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam requested review from Manciukic and micz010 as code owners May 15, 2026 16:40

Manciukic reviewed May 18, 2026

View reviewed changes

ShadowCurse reviewed May 19, 2026

View reviewed changes

Comment thread src/vmm/src/devices/virtio/device.rs Outdated

Comment thread src/vmm/src/devices/virtio/block/device.rs Outdated

ilstam added a commit to ilstam/firecracker that referenced this pull request May 21, 2026

CHANGELOG: Mention virtio device reset support

e36355d

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam added a commit to ilstam/firecracker that referenced this pull request May 21, 2026

CHANGELOG: Mention virtio device reset support

a4008c0

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from e3917dd to a4008c0 Compare May 21, 2026 16:36

ilstam added a commit to ilstam/firecracker that referenced this pull request May 21, 2026

CHANGELOG: Mention virtio device reset support

e65fd03

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from a4008c0 to e65fd03 Compare May 21, 2026 16:42

ilstam added a commit to ilstam/firecracker that referenced this pull request May 21, 2026

CHANGELOG: Mention virtio device reset support

ec24052

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from e65fd03 to ec24052 Compare May 21, 2026 18:09

ilstam added a commit to ilstam/firecracker that referenced this pull request May 21, 2026

CHANGELOG: Mention virtio device reset support

8ebdd80

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from ec24052 to 8ebdd80 Compare May 21, 2026 18:12

ilstam added a commit to ilstam/firecracker that referenced this pull request May 21, 2026

CHANGELOG: Mention virtio device reset support

df8a0e5

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from 8ebdd80 to df8a0e5 Compare May 21, 2026 18:26

ilstam added a commit to ilstam/firecracker that referenced this pull request May 26, 2026

CHANGELOG: Mention virtio device reset support

f3d3ec5

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from 82906df to f3d3ec5 Compare May 26, 2026 14:04

ilstam added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label May 26, 2026

bk0v reviewed May 27, 2026

View reviewed changes

Comment thread src/vmm/src/devices/virtio/block/virtio/device.rs

ilstam added a commit to ilstam/firecracker that referenced this pull request Jun 8, 2026

CHANGELOG: Mention virtio device reset support

eb45f85

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from f3d3ec5 to eb45f85 Compare June 8, 2026 14:07

ilstam added a commit to ilstam/firecracker that referenced this pull request Jun 8, 2026

CHANGELOG: Mention virtio device reset support

d6786a5

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from eb45f85 to d6786a5 Compare June 8, 2026 14:53

ilstam added a commit to ilstam/firecracker that referenced this pull request Jun 9, 2026

CHANGELOG: Mention virtio device reset support

853da29

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from d6786a5 to 853da29 Compare June 9, 2026 18:00

bk0v reviewed Jun 10, 2026

View reviewed changes

ilstam added 2 commits June 10, 2026 18:33

ilstam added 23 commits June 10, 2026 18:33

virtio: pci: Reset feature select registers on device reset

ff1e508

Reset device_feature_select and driver_feature_select to 0 on device reset, matching what the MMIO transport does with features_select and acked_features_select. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: mmio: Add unit test for device reset failure

fb227c3

Add test_bus_device_reset_failure to verify that when device.reset() returns false, the FAILED bit is set in device_status. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: Add deactivate() method to VirtioDevice trait

ad333db

Add a deactivate() method that sets the device state to Inactive. This will be used by the generic reset implementation in a subsequent patch. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: net: Add a RxBuffers::clear() method

05d33f0

Add a clear() method to RxBuffers that resets all fields to their initial state. This will be used by the virtio-net reset implementation. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: net: Implement device reset

511ed8c

Implement the _reset() method for the virtio-net device, resetting the net specific state in-place. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: pmem: Implement device reset

2e6d6f9

Pmem does not have any backend specific state that needs resetting so implement the _reset() method by simply returning true. The rest of the state is handled by the generic reset(). Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

tests: Add integration test for virtio-pmem device reset

dfacf7e

Add a test that verifies virtio-pmem device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: balloon: Implement device reset

93f3bba

Implement the _reset() method for the virtio-balloon device, resetting the balloon specific state in-place. Do not deflate the balloon on reset. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

tests: Add integration test for virtio-balloon device reset

cd9e5df

Add a test that verifies virtio-balloon device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

tests: Add integration test for virtio-rng device reset

f27986e

Add a test that verifies virtio-rng device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: vsock: Implement device reset

7528064

Implement the _reset() method for the virtio-vsock device, resetting the vsock specific state in-place. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

tests: Add integration test for virtio-vsock device reset

208119d

Add a test that verifies virtio-vsock device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

virtio: mem: Implement device reset

a041c80

The virtio-mem device does not have any backend specific state that needs resetting so implement the _reset() method by simply returning true. Plugged memory regions remain intact. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

tests: Add integration test for virtio-mem device reset

93dbf36

Add a test that verifies virtio-mem device reset works end-to-end by unbinding and rebinding the guest driver and checking the device remains functional. Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

CHANGELOG: Mention virtio device reset support

6f0ab8e

firecracker-microvm#5891 Signed-off-by: Ilias Stamatis <ilstam@amazon.com>

ilstam force-pushed the device-reset branch from 853da29 to 6f0ab8e Compare June 10, 2026 17:34

Conversation

ilstam commented May 15, 2026

License Acceptance

PR Checklist

Uh oh!

codecov Bot commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bk0v May 27, 2026 • edited by ilstam Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bk0v May 27, 2026 • edited by ilstam Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented May 15, 2026 •

edited

Loading

bk0v May 27, 2026 •

edited by ilstam

Loading

bk0v May 27, 2026 •

edited by ilstam

Loading