Implement Incus Executor and Instance API

To allow us to run tasks under Incus, we need to implement the Executor and Instance API. (#219 (closed))

How we envision Incus instances to work:

• Debusine interacts with Incus using the CLI.
  • There is also a REST API, with a Python API client for LXD that should be compatible, but no Incus fork, yet.
• System Tarballs / System Images are customized for Incus (if necessary) and imported into Incus's storage.
• A single network bridge is used for all containers / VMs.
• Use systemd-networkd and systemd-resolved in containers / VMs, if possible. Fall back to alternatives in older releases if necessary.
• We'll have two separate backends for containers and VMs: incus-lxc and incus-vm. They will share a lot of common code, probably via a base-class.

Requirements:

• We'll have to document Incus install and configuration.
  1. A managed debusinebr0 network.
  2. A default storage pool.
  3. A debusine profile with the config: raw.lxc: lxc.mount.auto=sys and devices:

  eth0:
    name: host0
    network: debusinebr0
    type: nic
  root:
    path: /
    pool: default
    type: disk

• Downloading the tarball should modify it appropriately and import it with the alias: artifact/$ID. Modifications for containers:
  1. Enable systemd-networkd (this can be done by appending a symbolic link from /etc/systemd/system/multi-user.target.wants/systemd-networkd.service to /lib/systemd/system/systemd-networkd.service to the tarball image).
  2. Inject https://github.com/lxc/distrobuilder/blob/main/distrobuilder/lxc.generator as /etc/systemd/system-generators/lxc by appending it to the the tarball. It needs to be executable.
  3. Write templates:
     1. hostname.tpl: {{ container.name }}
     2. hosts.tpl:

127.0.1.1   {{ container.name }}
127.0.0.1   localhost
::1     localhost ip6-localhost ip6-loopback
ff02::1     ip6-allnodes
ff02::2     ip6-allrouters

4. Write metadata like this:

---
architecture: $arch
creation_date: $date
profiles:
  - debusine
properties:
  description: $description
  os: $distro
  release: $suite
  artifact_id: $artifact
templates:
  /etc/hostname:
    when:
    - create
    - copy
    create_only: false
    template: hostname.tpl
    properties: {}
  /etc/hosts:
    when:
    - create
    - copy
    create_only: false
    template: hosts.tpl
    properties: {}

• autopkgtest_virt_args should just emit the alias.
• Creating the instance should launch an instance based on the image and store the instance ID as a property.
• Deleting the instance (including GC) should: Shut down the instance and delete it.
• Starting the instance should: Start the instance.
• Stopping the instance should:
  1. Ask the instance to stop, gracefully.
  2. Kill the instance, if it doesn't shut down within 30 seconds.
• File push should use Incus' file push mechanism.
• File pull should use Incus' file pull mechanism.
• Run should execute the provided command via Incus' exec mechanism.

Discussion points:

• The simple customization above (and using host0 inside the instance) is enough for unstable. More customization will be needed for older instances. The tried and tested mechanism is distrobuilder's. We could crib from it. Or try to work our way through it and build a minimal set of customizations that we have to apply to images.

Notes:

• This will require autotpkgtest >= 5.32.
• And a bug fix in sbuild: https://bugs.debian.org/1061388 or Incus > 0.5.0: https://github.com/lxc/incus/issues/422
• A useful feature in incus to inject lxd.generator: https://github.com/lxc/incus/issues/465