Signed-off-by: Holger Levsen <holger@layer-acht.org>

Changes-By: lintian-brush

Changes-By: lintian-brush
Fixes: lintian: older-source-format
See-also: https://lintian.debian.org/tags/older-source-format.html

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Michael Prokop authored Feb 16, 2021 and Holger Levsen committed Feb 21, 2021

If the Packages file gets rewritten during `apt-get update`
execution, then piupart might fail with a hashsum mismatch
error, like:

| DEBUG: Starting command: ['chroot', '/var/cache/pbuilder/tmp.3l19K3MgeX_piuparts_tmp/tmp1m34xz', 'apt-get', 'update']
| DUMP:
| [...]
| Err:5 https://deb.example.com/autobuild gerrit_mika_foobar/main amd64 Packages
|   File has unexpected size (1338 != 1346). Mirror sync in progress? [IP: 10.10.42.42 443]
|   Hashes of expected file:
|    - Filesize:1346 [weak]
|    - SHA256:5af5db54c0d80e595a247a2070a1f8381468cbdacea1b62ead353d5bea074697
|    - SHA1:61962e3dcd01055a4baf3ab8e318c17832b6b0fd [weak]
|    - MD5Sum:92f602acb8051cfd90b3e33900a29932 [weak]
|   Release file created at: Tue, 16 Feb 2021 01:13:26 +0000
| [...]
| Reading package lists...
| E: Failed to fetch https://deb.example.com/autobuild/dists/gerrit_mika_foobar/main/binary-amd64/Packages.gz  File has unexpected size (1338 != 1346). Mirror sync in progress? [IP: 10.10.42.42 443]
|    Hashes of expected file:
|     - Filesize:1346 [weak]
|     - SHA256:5af5db54c0d80e595a247a2070a1f8381468cbdacea1b62ead353d5bea074697
|     - SHA1:61962e3dcd01055a4baf3ab8e318c17832b6b0fd [weak]
|     - MD5Sum:92f602acb8051cfd90b3e33900a29932 [weak]
|    Release file created at: Tue, 16 Feb 2021 01:13:26 +0000
| E: Some index files failed to download. They have been ignored, or old ones used instead.
| ERROR: Command failed (status=100): ['chroot', '/var/cache/pbuilder/tmp.3l19K3MgeX_piuparts_tmp/tmp1m34xz', 'apt-get', 'update']

By giving the user an option to retry the update procedure (up to
<num> times, as specified on the command line), we increase
chances for a successful build. This is useful especially in busy
continuous integration/delivery environments, lacking by-hash
support (see
https://www.chiark.greenend.org.uk/~cjwatson/blog/no-more-hash-sum-mismatch-errors.html),
for example all repositories using reprepro, which doesn't support
by-hash yet (see https://bugs.debian.org/820660

).

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

This reverts commit 9abb0e66.

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Changes-By: lintian-brush
Fixes: lintian: debian-changelog-has-wrong-day-of-week
See-also: https://lintian.debian.org/tags/debian-changelog-has-wrong-day-of-week.html

Nis Martensen authored Aug 26, 2020 and Holger Levsen committed Jan 18, 2021

Since commit c3900457

 different dicts
were used for iterating and modifying, so we never modify the dicts we
are currently iterating over. However, the commit was incomplete because
it was longer guaranteed that each key is still part of the dict we are
trying to modify, but no checks were added.
The result are possible crashes such as the one reported in 968842:

    Traceback (most recent call last):
      File "/usr/sbin/piuparts", line 3267, in <module>
        main()
      File "/usr/sbin/piuparts", line 3252, in main
        process_packages(package_list)
      File "/usr/sbin/piuparts", line 3173, in process_packages
        if not install_purge_test(chroot, chroot_state,
      File "/usr/sbin/piuparts", line 2414, in install_purge_test
        return check_results(chroot, chroot_state, file_owners, deps_info=deps_info)
      File "/usr/sbin/piuparts", line 2235, in check_results
        (new, removed, modified) = diff_meta_data(reference_info, current_info)
      File "/usr/sbin/piuparts", line 2041, in diff_meta_data
        del tree2_c[name]
    KeyError: '/var/spool/news/in.coming/tmp/'

Such KeyErrors can be prevented by checking if a key exists in the dict
before trying to drop it. Adding explicit checks is simpler than
adding a try..except clause here because there are already if clauses
that we can adapt.

Closes: #968842
Signed-off-by: Holger Levsen <holger@layer-acht.org>

Nis Martensen authored Sep 06, 2020 and Holger Levsen committed Jan 18, 2021

This should fix the crash reported in
https://alioth-lists.debian.net/pipermail/piuparts-devel/2020-September/009165.html



Signed-off-by: Holger Levsen <holger@layer-acht.org>

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Nis Martensen authored Dec 28, 2020 and Holger Levsen committed Jan 18, 2021

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Nis Martensen authored Dec 27, 2020 and Holger Levsen committed Jan 18, 2021

The existing must-succeed test protects against potential bugs that make
piuparts always fail. Other potential bugs might make piuparts always
succeed, so let's add a must-fail test to cover this case, too.

Signed-off-by: Holger Levsen <holger@layer-acht.org>

Make the summary generation atomic

See merge request !28

Having Depends last confuses apt 2.0.0.

Reported-By: Ferenc Wágner (https://alioth-lists.debian.net/pipermail/piuparts-devel/2020-March/009069.html)

This file, which tracks the current dselect download method, is now handled by
dselect (>= 1.20.0~), but it is also created by debootstrap while it is doing
its apt/dselect setup.

Closes: #953756.

With the performance improvement of querying the BTS in bulk, piuparts-analyze
now spends most of its time waiting for busy sections. Let's wait a bit less.

The BTS SOAP interface struggles when we do a lot of package-based queries; We
even get throttled (and get ConnectionErrors) when doing the queries for a usual
piuparts-analyze run on pejacevic.

Querying the BTS by bug numbers instead doesn't hit the throttle, even when
getting all bugs usertagged for piuparts. This bulk query makes the full
piuparts-analyze run go way faster.

fixes:

    ERROR processing fail/<package>.log
    <class 'TypeError'>

See !10

See merge request !15

Signed-off-by: Holger Levsen <holger@layer-acht.org>