Stop using origtargz/uscan to download sources during the Salsa CI build (at least if upstream branch/tags exists in project)
While reviewing upload history of https://tracker.debian.org/pkg/xz-utils and commits and tags at https://salsa.debian.org/debian/xz-utils/-/commits/debian/unstable I noticed that the Salsa-CI step to extract sources was failing on gbp error, and then falling back to download source tarball with uscan:
$ if find . -maxdepth 3 -wholename "*/debian/source/format" -exec cat {} \; | grep -q '3.0 (gitarchive)' # collapsed multi-line command
gbp:info: All Orig tarballs 'xz-utils_5.6.1.orig.tar.gz' found at '/builds/debian/xz-utils/debian/output'
gbp:error: upstream/5.6.1 is not a valid treeish
Trying uscan --download --download-current-version ...
Newest version of xz-utils on remote site is 5.6.1, specified download version is 5.6.1
gpgv: Signature made Sat Mar 9 08:19:09 2024 UTC
gpgv: using RSA key 22D465F2B4C173803B20C6DE59FCF207FEA7F445
gpgv: Good signature from "Jia Tan <jiat0218@gmail.com>"
Successfully renamed ../xz-5.6.1.tar.xz to ../xz-utils_5.6.1.orig.tar.xz.
'../xz-utils_5.6.1.orig.tar.xz' -> '/builds/debian/xz-utils/debian/output/xz-utils_5.6.1.orig.tar.xz'
'../xz-utils_5.6.1.orig.tar.xz.asc' -> '/builds/debian/xz-utils/debian/output/xz-utils_5.6.1.orig.tar.xz.asc'
gbp:info: Disabling 'cleaner' hook
gbp:info: Extracting 'xz-utils_5.6.1.orig.tar.xz' to '/builds/debian/xz-utils/debian/output/xz-utils-tmp'
gbp:info: Exporting 'HEAD' to '/builds/debian/xz-utils/debian/output/xz-utils-tmp'
gbp:info: Moving '/builds/debian/xz-utils/debian/output/xz-utils-tmp' to '/builds/debian/xz-utils/debian/output/xz-utils-5.6.1'
gbp:info: Performing the build
Note gbp:error: upstream/5.6.1 is not a valid treeish
. Full log at https://salsa.debian.org/debian/xz-utils/-/jobs/5506680
In the xz-utils did have a upstream branch, and upstream tags, but they were misconfigured. The project used git-buildpackage and had a `debian/gbp.conf with:
[DEFAULT]
debian-branch=debian/unstable
upstream-branch=upstream/v5.5
This was misconfigured in two ways: the package was updated and it should have been pointing to v5.6
and additionally the repository on Salsa was missing branches with the naming pattern upstream/v5.5
and upstream/v5.6
. Based on xz-utils/a9aa9fc2 where the maintainer stopped using upstream git branch and switched to importing upstream tarballs in single commits it seems the maintainer just didn't fully grasp what mistakes they were doing, and thus having Salsa-CI fail on the upstream/5.6.1 is not a valid treeish
error would have probably helped maintainer to fix it (assuming they didn't intentionally want to obfuscate the source code source).
Salsa-CI should have had a hard failure on this and not fall-back to downloading anything.
Use of origtargz
was introduced in dc2db922 4 years ago by @hartge with reference to discussion with @tin with commit message title "Complete rewrite of job logic" and nothing else. This makes me ask do we really need origtargz
? Could the extract-source step run without network access completely to block build-time downloads, or could it at least have prioritization of git-buildpackage commands and fail if there are traces of gbp-buildpackage usage and it fails to run?
Falling back to origtargz
could be limited to only some very specific edge-cases of packages where maintainer does not care about upstream source code supply chain, or completely removed as most maintainers who care about quality and use Salsa-CI most likely also care about supply chain integrity.