Skip to content

Implement QEMU autopkgtest virtualization backend

To allow us to run tasks under QEMU, we need to implement the Executor and Instance API. (#219 (closed))

How we envision QEMU instances to work:

  • The QEMU executor downloads system-images into the image cache with ImageCache.
  • An instance uses a COW overlay image on top of the system-image, stored in a temporary directory.
  • Unsafe caching can be used as the VM is ephemeral.
  • KVM acceleration is used if possible.
  • Instance memory and disk sizes are static, for now.
  • Host networking is used, for now.
  • Communication with the VM is via QMP and qemu-guest-agent.

Requirements:

  • autopkgtest_virt_args should just pass the path to the system-image, on the command line.
  • Creating the instance should:
    1. Download the system-image into the image cache, if necessary.
    2. Create a per-instance temporary directory.
    3. Create an overlay image for the instance: qemu-img create -f qcow2 -F $BASE_FORMAT -b $TEMP/overlay.qcow2
  • Deleting the instance (including GC) should:
    1. Stop the instance, if it's running
    2. Clean up the temporary directory.
  • Starting the instance should:
    1. Start a headless QEMU instance with:
      • The overlay image as rootfs.
      • An EFI BIOS
      • A port-forward from a local port to ssh in the VM (if we need it?)
      • Serial ports bound to unix sockets in the instances' temporary directory.
      • QMP bound to a unix socket in the instance's temporary directory.
      • Start an asyncio driver to connect to the QMP socket.
    2. Watch the serial console and login when a prompt appears.
    3. Install qemu-guest-agent in the VM.
  • Stopping the instance should:
    1. Ask QEMU to gracefully shut down, via QMP (guest-shutdown).
    2. Kill QEMU, if it doesn't shut down within 30 seconds, via QMP(quit).
  • File push should install a file with appropriate permissions in the VM. This can be done using QMP (guest-file-open, guest-file-write, guest-file-close).
  • File pull should copy a file out of the VM. This can be done using QMP (guest-file-open, guest-file-read, guest-file-close).
  • Run should execute the provided command with QMP (guest-exec).

Discussion points:

  • Any concerns about QMP / guest-agent? I have no experience with QMP, but it looks like a good option.
  • Can we require qemu-guest-agent to be installed in the image? That would simplify startup.
  • Should we use autopkgtest's autopkgtest_qemu library to implement this, instead of QMP? It does almost exactly what we need, out of the box. (Although some of the useful bits live in autopkgtest-virt-qemu). It requires python3 in the VM and communicates with the VM through files in a virtfs, which seems more sensible than trying to escape stuff for shells, but more complex than QMP.
  • Or, do we want to go down this road at all? Incus can drive QEMU for us, and it'll look almost exactly like a container. (#271 (closed))
  • Do we want a mechanism to define RAM and disk space? Should this be part of the image, or a parameter to the task?
  • Do we want to use bridges for networking? (requiring root) My inclination is to use Incus to drive Qemu instances that need more network control.
Edited by Stefano Rivera
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information