Skip to content

Mirroring multiple suites is network-inefficient

The APTMirror task downloads packages if they don't exist in the target suite. For suites that share a large number of packages - for example, trixie and sid - this is very inefficient. The resulting files are automatically deduplicated in the database and the file store (because files are stored based on their hash), but the task still has to download the files from the remote mirror before realizing that they're identical to the ones it already has and so it can just throw them away. It would be good to optimize this.

I think the best approach would be to download all the files matching hashes we expect (based on the suite's index files) from debusine's store into the temporary download directory before calling apt-get source or apt-get download. This way we don't completely trust the hashes (apt-get will at least redownload if there's a file name or size mismatch), and it would straightforwardly deal with the common case in source packages where they share some but not all of their files (especially .orig.tar.*) with other source packages.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information