I've recently deep-dived into QEMU performance especially in order to cut down boot/bootstrapping times for quickly running tests and I'd like to share some tricks:

- Templating: https://qemu.readthedocs.io/en/v9.1.0/system/vm-templating.h... - virtio-fs: https://virtio-fs.gitlab.io/ - Overlay storage images: https://wiki.archlinux.org/title/QEMU#Overlay_storage_images

Cloud init will handle unique IDs although might add some boot time

You need to get it to re-run, which I'm not sure there is a standard way to do during runtime. Can it even do all its job without a reboot?

Maybe I misunderstood, is this skipping boot as well?

I was thinking the VM would still follow the normal boot process (in which case you can reset cloud init before powering down)

Yes they are saving memory state to a file.

Is there a reason to use 9p over virtiofs in this case? It looks to me as though virtiofs was made exactly with the idea to deal good performance.

NixOS has switched to virtiofs 4 days ago: https://github.com/NixOS/nixpkgs/pull/362081

Not all use-cases are faster with virtiofs.

> Not all use-cases are faster with virtiofs

Aren't they though? Do you have a counter example?

Interesting and may be worth writing a qemu issue about.

The author commented on that in 2022 https://news.ycombinator.com/item?id=33200731

Is 9p compatible with saving VM states? I recently experimented with virtiofs and was disappointed to find it was incompatible so it didn't really work for my use case.

I don't even really need performance, I only really want to be able to grab driver files from a common location and ditch the samba shares.

For better convenience there is now https://github.com/KDAB/hotspot to render the flame graph from your perf data. You can also use online https://profiler.firefox.com/.

Discussed at the time:

Digging into a QEMU problem of slow data copying - https://news.ycombinator.com/item?id=33192152 - Oct 2022 (31 comments)

wow, clickbait title, then 9p and nix in the first couple paragraphs firmly set the hook.

title was "Make your QEMU 10 times faster with this one weird trick"

This was interesting, but by the end of it, I was still expecting some weird trick to work around the problem without the patch.

They delivered on the clickbait title to engagement time metric, as the title promises one weird trick and the content has no TLDR.

I've got you:

hashtables faster than linked lists in places where lookups are frequent & item count is high

Well, the article is fairly short, so a TLDR is perhaps not necessary.

Tl;dr: file descriptors for the guest os are stored as linked lists by qemu, and changing that sped qemu file copying up by a ton.

And the commit here: https://gitlab.com/qemu-project/qemu/-/commit/f5265c8f917ea8...

Included in QEMU version 7.2.0 from 2022-12-14.