Add generic API to drive execution in containers/VM

For many tasks, we will want to be able to start a throw-away container or a VM, make some preparatory changes inside (add files, install packages, etc.), then run a command and get some data (files, standard output of command, etc.) out of it and then destroy it with the associated data. We expect to have to drive qemu/kvm-based VM and at least one sort of container (lxd, podman, unshare, systemd-nspawn, etc.)

Use cases

The first case is the lintian task: there we want to run lintian inside a container to be able to use different versions of lintian from different releases (and from different vendors).

The second case is piuparts where we want to be able to run piuparts in a container to provide "root-level" permissions to the process without actually having those permissions on the host.

In the future, we will likely need to be able to run things like sbuild in throw-away VM for extra safety (given the arbitrary commands that can be submitted by users in a package build process).

Plan

• Review autopkgtest's API and launchpad's API for inspiration
  • Autopkgtest: VirtSubProc + sample virtualization script
  • Launchpad: Backend and LXD
• Come up with a sane Python object-based API
• Implement unshare in autotpkgtest (!409 (merged))
• Implement unshare in sbuild (#248 (closed))
• Implement the full API of unshare (#249 (closed))
• Implement unshare for piuparts (#166 (closed))
• Implement unshare for blhc (!588 (merged))
• Implement unshare for lintian
• Implement the API for Qemu (#270 (closed))
• Implement the API for LXD/Incus (#271 (closed), !545 (merged))

Proposed API

from abc import ABC
from pathlib import Path, PurePath
from subprocess import CompletedProcess

class ExecutionEnvironmentInterface(ABC):
    def __init__(debusine_api: Debusine, system_image: int, **kwargs):
        # debusine_api is the object to use the debusine client API
        # system_image is an artifact id pointing to the system tarball
        # or disk image to use (as appropriate or each technology)

    def start(self) -> None:
        """Start the environment, making it ready to run commands."""

    def stop(self) -> None:
        """Stop the environment."""

    def file_push(self, source: Path, target: PurePath) -> None:
        """Copy a file into the environment."""

    def file_pull(self, source: PurePath, target: Path) -> None:
        """Copy a file out of the environment."""

    def run(self, args: list[str], *, **kwargs) -> CompletedProcess:
        """Run a command in the environment.

        Arguments behave as if passed to `subprocess.run`.
        """

Open questions

• How do we deal with the need to prepare LXD containers or kvm disk images so that they are ready for use by autopkgtest ? Is that that part of this API too ?