Fixes to support building from non-Debian hosts

The Live Manual lists the requirements for live-build, and it explicitly mentions that a Debian host is not required. However, when using it from Arch Linux, I found a few spots that assumed a Debian host (namely, a specific PATH or the presence of dpkg tools).

changed the description

These shift calls were copied from another spot in this file, but that location made use of "$@", so the shift was in fact necessary. Here, it actually ensured the CHROOT variable could not be set (unless someone called Chroot_has_package "$package" "" "$chroot")

Ok

dpkg tools don't need to run on debian, they can run anywhere, just build it for arch?

There's a couple reasons why I think this is the better solution:

First, As the requirements section from the live build manual states, the only required packages are a POSIX-compliant shell and debootstrap. It seems clear to me that dpkg tools are not intended to be required for the use of live build. The fact that this path is only about 15 lines reinforces the fact that these uses of dpkg and dpkg-query were unintentional oversights, not an attempt to add a dpkg dependency.

Second, I can tell you from experience, it's much easier to apply this patch before building and installing live-build than it is to install the dpkg tools that are used here, especially since they are not currently packaged for Arch (debootstrap is). I've applied this patch and been using the patched version for a while.

Fine, but very few people use that setup, so if you want to be sure there won't be regressions it would be better if you added an autopkgtest test to cover that use case

Good suggestion. However, I'm not sure how to create an autopkgtest environment without the presence of dpkg, etc. The documentation seems to indicate that it always creates a basic Debian environment.

One idea I had was to create a simple chroot inside the autopkgtest environment that just contained debootstrap and then run busybox --install -s to create a very minimal environemnt, but that seemed like too much minimalism. busybox sh is more barebones than bash, so I'm guessing that there are other areas that would need to be changed to support using that.

Please go to https://salsa.debian.org/sam-lunt/live-build/-/settings/ci_cd and under "General pipelines" change "Timeout" to "3h", and then restart the autopkgtest job - unfortunately default settings are not inherited on fork

The function Chroot_has_package is currently unused.

It wasn't working properly, the correct line would be:

if Chroot "${CHROOT}" dpkg-query -s ${PACKAGE} 2>&1 | grep -q "^Status: install"

• Pass along ${CHROOT}
• Do not redirect to /dev/null, because grep needs the input

Good point about the missing "${CHROOT}", thanks. I've updated it to your suggestion, except I made it 2>/dev/null, since I think it's more appropriate to grep just the stdout lines in this case.

Ok.

Indeed. Both times the dollar should have been escaped

Nitpick: no semicolon required

changed this line in version 2 of the diff

Good point

Have you tested this?

I've tried to place some random .deb files in the various locations in the configuration:

mkdir -p config/packages
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w1_amd64.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w2_amd64.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w3_all.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w4_all.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w1_amd64.udeb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w2_amd64.udeb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w3_all.udeb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages/w4_all.udeb
mkdir -p config/packages.binary
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w5_amd64.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w6_amd64.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w7_all.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w8_all.deb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w5_amd64.udeb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w6_amd64.udeb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w7_all.udeb
cp -a /var/cache/apt/archives/wireplumber_0.4.4-1_amd64.deb config/packages.binary/w8_all.udeb

When dpkg is called, the files are not (yet?) inside the chroot.

changed this line in version 2 of the diff

You're right, this isn't correct. I think a correct solution could be to change Install_file to just copy the .deb or .udeb file to the top-level DEST, then have a separate function that runs a script in the chroot along the lines of:

for FILE in *.deb *.udeb
do
    : # reorganize based on archive area
done

(similar to how apt-ftparchive is run here and here)

I'm hesitant to change so much about the script though; obviously I'm not super familiar with how it operates. On the other hand, it does seem like a bummer to have dpkg required on the host side solely for use in Install_file

I didn't investigate further, but it looks to me that the line dpkg -I ${FILE} | awk '/^.*Section: / { print $2 }' only extracts the Section: values. Can't this be worked-around? E.g. with a new bash function that either calls dpkg if available or otherwise performs 'some magic'.

This was something I'd considered as well, but I thought it seemed too fragile. Maybe it's worth exploring further, though.

A .deb file (and presumably .udeb as well) is an archive of 3 files, control.tar.xz, data.tar.xz, and debian-binary. The file we are interested in here is part of the control.tar.xz tarball, namely control. So we could do this:

ar p "$deb_file" control.tar.xz | tar -x --xz -f - ./control -O | grep -F 'Section:'

The main issue is that the compression format has not always been .xz (and thus might not always be xz). So there would need to be a step beforehand to list the archive contents and choose the decompression option based on that. At this point, I worry that this will re-implement too much of dpkg's functionality. That said, maybe it's feasible; I'm open to opinions/suggestions.

please don't - simply skip those values if possible, otherwise make the debian-installer support disabled on such setups entirely (it's only a subset of functionality anyway)

There are already 2 installers: debian-installer from the boot menu and Calamares from a live booted environment. Perhaps for your use-case Calamares is sufficient?

added 1 commit

• 87d6b693 - Fixes to support building from non-Debian hosts

Compare with previous version