fix: Call KVM_KVMCLOCK_CTRL not only after pause but also before snapshot resume #5494
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5494 +/- ##
=======================================
Coverage 82.84% 82.84%
=======================================
Files 269 269
Lines 27737 27737
=======================================
Hits 22978 22978
Misses 4759 4759
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
zulinx86
force-pushed
the
kvmclock_ctrl_before_resume
branch
from
301f879 to
ff48e0e
Compare
zulinx86
requested review from
Manciukic,
kalyazin,
pb8o and
xmarcalx
as code owners
zulinx86
force-pushed
the
kvmclock_ctrl_before_resume
branch
from
ff48e0e to
3fc6bf3
Compare
zulinx86
added
the
Status: Awaiting review
kalyazin
reviewed
Oct 28, 2025
zulinx86
force-pushed
the
kvmclock_ctrl_before_resume
branch
2 times, most recently
from
64f27a2 to
0dfdc66
Compare
kalyazin
previously approved these changes
Oct 28, 2025
|
To double check what's the impact on the resume path, I've kicked off a BK A/B build: https://buildkite.com/firecracker/performance-a-b-tests/builds/736 |
thanks! |
Manciukic
reviewed
Oct 28, 2025
zulinx86
force-pushed
the
kvmclock_ctrl_before_resume
branch
2 times, most recently
from
07cf216 to
ab0f55e
Compare
zulinx86
changed the title
KVM_KVMCLOCK_CTRL ioctl sets `pvclock_set_guest_stopped_request` flag of `kvm_vcpu_arch` [1]. On the next guest time update, if the flag is set, KVM ORs in `PVCLOCK_GUEST_STOPPED` and `kvm_setup_guest_pvclock()` pushes the `hv_clock` into the guest's pvclock page [2]. If the `hv_clock` has not been written to the guest's pvclock page when taking a snapshot, it is not saved in the snapshot memory (i.e. `PVCLOCK_GUEST_STOPPED` isn't set in resumed VMs). So we should call KVM_KVMCLOCK_CTRL ioctl before resuming a VM in addition to after pausing a VM. [1]: https://elixir.bootlin.com/linux/v6.16.3/source/arch/x86/kvm/x86.c#L5734 [2]: https://elixir.bootlin.com/linux/v6.16.3/source/arch/x86/kvm/x86.c#L3286-L3295 Signed-off-by: Takahiro Itazuri <[email protected]>
zulinx86
force-pushed
the
kvmclock_ctrl_before_resume
branch
from
ab0f55e to
0894dbc
Compare
bchalios
approved these changes
Oct 29, 2025
kalyazin
approved these changes
Oct 29, 2025
maggie-lou
added a commit
to buildbuddy-io/buildbuddy
that referenced
this pull request
Oct 30, 2025
Also includes this patch: firecracker-microvm/firecracker#5494 I'm interested in these bug fixes since v1.11.0 (which we were on previously): firecracker-microvm/firecracker#5122 firecracker-microvm/firecracker#5260 We should also look into enabling PCI. From their release notes: > In our micro-benchmarks, we measured up to 50% better latency for block and network, up to 70% better block throughput, and up to 25% higher network throughput (results depend on instance type and kernel). Also I had previously cherry-picked a firecracker patch that fixed some of the rcu_sched failures. It was formally merged into the firecracker repo [here](firecracker-microvm/firecracker#5494) and this final version has some implementation differences from the patch we'd applied. I can't directly apply the updated patch on v1.11.0 due to some merge conflicts, but it applies cleanly on top of v1.13.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #5322.
Changes
Reason
KVM_KVMCLOCK_CTRL ioctl sets
pvclock_set_guest_stopped_requestflag ofkvm_vcpu_arch1. On the next guest time update, if the flag is set, KVM ORs inPVCLOCK_GUEST_STOPPEDandkvm_setup_guest_pvclock()pushes thehv_clockinto the guest's pvclock page 2. If thehv_clockhas not been written to the guest's pvclock page when taking a snapshot, it is not saved in the snapshot memory (i.e.PVCLOCK_GUEST_STOPPEDisn't set in resumed VMs). So we should call KVM_KVMCLOCK_CTRL ioctl before resuming a VM rather than after pausing a VM. That covers both the pause-and-resume case and the restore-and-resume case.License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyleto verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
[ ] I have updated any relevant documentation (both in code and in the docs)in the PR.
CHANGELOG.md.[ ] When making API changes, I have followed theRunbook for Firecracker API changes.
integration tests.
[ ] I have linked an issue to every newTODO.rust-vmm.