Implement QEMU autopkgtest virtualization backend
To allow us to run tasks under QEMU, we need to implement the Executor and Instance API. (#219 (closed))
How we envision QEMU instances to work:
- The QEMU executor downloads system-images into the image cache with ImageCache.
- An instance uses a COW overlay image on top of the system-image, stored in a temporary directory.
- Unsafe caching can be used as the VM is ephemeral.
- KVM acceleration is used if possible.
- Instance memory and disk sizes are static, for now.
- Host networking is used, for now.
- Communication with the VM is via QMP and qemu-guest-agent.
Requirements:
-
autopkgtest_virt_argsshould just pass the path to the system-image, on the command line. - Creating the instance should:
- Download the system-image into the image cache, if necessary.
- Create a per-instance temporary directory.
- Create an overlay image for the instance:
qemu-img create -f qcow2 -F $BASE_FORMAT -b $TEMP/overlay.qcow2
- Deleting the instance (including GC) should:
- Stop the instance, if it's running
- Clean up the temporary directory.
- Starting the instance should:
- Start a headless QEMU instance with:
- The overlay image as rootfs.
- An EFI BIOS
- A port-forward from a local port to ssh in the VM (if we need it?)
- Serial ports bound to unix sockets in the instances' temporary directory.
- QMP bound to a unix socket in the instance's temporary directory.
- Start an asyncio driver to connect to the QMP socket.
- Watch the serial console and login when a prompt appears.
- Install
qemu-guest-agentin the VM.
- Start a headless QEMU instance with:
- Stopping the instance should:
- Ask QEMU to gracefully shut down, via QMP (
guest-shutdown). - Kill QEMU, if it doesn't shut down within 30 seconds, via QMP(
quit).
- Ask QEMU to gracefully shut down, via QMP (
- File push should install a file with appropriate permissions in the VM. This can be done using QMP (
guest-file-open,guest-file-write,guest-file-close). - File pull should copy a file out of the VM. This can be done using QMP (
guest-file-open,guest-file-read,guest-file-close). - Run should execute the provided command with QMP (
guest-exec).
Discussion points:
- Any concerns about QMP / guest-agent? I have no experience with QMP, but it looks like a good option.
- Can we require
qemu-guest-agentto be installed in the image? That would simplify startup. - Should we use autopkgtest's
autopkgtest_qemulibrary to implement this, instead of QMP? It does almost exactly what we need, out of the box. (Although some of the useful bits live inautopkgtest-virt-qemu). It requires python3 in the VM and communicates with the VM through files in a virtfs, which seems more sensible than trying to escape stuff for shells, but more complex than QMP. - Or, do we want to go down this road at all? Incus can drive QEMU for us, and it'll look almost exactly like a container. (#271 (closed))
- Do we want a mechanism to define RAM and disk space? Should this be part of the image, or a parameter to the task?
- Do we want to use bridges for networking? (requiring root) My inclination is to use Incus to drive Qemu instances that need more network control.
Edited by Stefano Rivera