179.md 11.5 KB
Newer Older
layout: new/blog
week: 179
published: 2018-10-01 17:34:43
5 6

Chris Lamb's avatar
Chris Lamb committed
Here's what happened in the [Reproducible Builds](https://reproducible-builds.org) effort between Sunday September 23 and Saturday September 29 2018:

Chris Lamb's avatar
Chris Lamb committed
* Another reminder that the **fourth Reproducible Builds summit will take place in December 11th—13th 2018** at [Mozilla](https://wiki.mozilla.org/Paris)'s offices in Paris, France. If you are interested in attending please register by sending an email to `holger@layer-acht.org`. More details may be found on the [associated event page](https://reproducible-builds.org/events/paris2018/).

Chris Lamb's avatar
Chris Lamb committed
* Holger Levsen is requesting review and comments on [a multi-project syntax notes proposal](https://salsa.debian.org/reproducible-builds/reproducible-notes/blob/multi-project-syntax/README). He hopes to implement this in the next few weeks.
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed

Chris Lamb's avatar
Chris Lamb committed
* [A pull request was proposed](https://github.com/python/cpython/pull/9607) to [CPython](https://github.com/python/cpython) to ensure [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/specs/source-date-epoch/) only influences the default type of `.pyc` files (yet does not enforce it).
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed

Chris Lamb's avatar
Chris Lamb committed
* [Guillem Jover](https://www.hadrons.org/~guillem/) uploaded [dpkg 1.19.1 to Debian unstable](https://tracker.debian.org/news/989886/accepted-dpkg-1191-source-into-unstable/), adding support for dumping database package records in alphabetical order in order to provide reproducible `status` and `available` database files, and further make it possible to output other "[deb822](https://manpages.debian.org/unstable/dpkg-dev/deb822.5.en.html)"-formatted data in a deterministic way. In addition, the `Dpkg::Vendor::Debian` library gained support for a reproducibility-related `fixfilepath` feature.

Chris Lamb's avatar
Chris Lamb committed
* Jelle van der Waa [started a TODO list](https://www.archlinux.org/todo/imagemagicks-convert-not-reproducible-in-pkgbuilds/) to fix unreproducible `PKGBUILD` files in [Arch Linux](https://www.archlinux.org/) which use the [Imagemagick](https://www.imagemagick.org/script/index.php) graphics library's `convert(1)` utility for image manipulation which embeds the `date:create` and `date:modify` file headers.
Daniel Shahaf's avatar
Daniel Shahaf committed

Chris Lamb's avatar
Chris Lamb committed
* `disorderfs` version `0.5.4-1` (our [FUSE](https://github.com/libfuse/libfuse)-based filesystem that deliberately introduces non-determinism into filesystem metadata) was [uploaded to Debian unstable](https://tracker.debian.org/news/989895/accepted-disorderfs-054-1-source-amd64-into-unstable/) by Chris Lamb. It included a number of contributions [covered previously](https://salsa.debian.org/reproducible-builds/disorderfs/commits/debian/0.5.4-1).
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed

Chris Lamb's avatar
Chris Lamb committed
21 22
* A fix to ensure to that [fontconfig](https://www.freedesktop.org/wiki/Software/fontconfig/) — a library for configuring and customizing font access — [landed in Debian unstable](https://bugs.debian.org/864082#96) (via upstream).

Chris Lamb's avatar
Chris Lamb committed
* Chris Lamb added four Debian package reviews. In addition, three were updated and 11 were removed in this week, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html).
Chris Lamb's avatar
Chris Lamb committed
24 25

* Bernhard M. Wiedemann discovered problems from [arch-dependent `noarch` packages](https://bugzilla.opensuse.org/show_bug.cgi?id=1109534) in [openSUSE](https://www.opensuse.org/); there are actually over 1,000 of these.
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed
26 27 28 29 30 31 32

Patches filed

* Bernhard M. Wiedemann:

    * [build-compare](https://github.com/openSUSE/build-compare/pull/30) (better quoting of strings)
Chris Lamb's avatar
Chris Lamb committed
    * [cpio](https://build.opensuse.org/request/show/638582) (drop date using `gettextize --no-changelog`)
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed
    * [foma/malaga-suomi](https://github.com/mhulden/foma/pull/78) (merged, segfault bug, noarch)
Chris Lamb's avatar
Chris Lamb committed
    * [gromacs](https://gerrit.gromacs.org/8151) (merged, drop user+host+date+CPU info)
    * [kubernetes](https://github.com/kubernetes/kubernetes/pull/68983) (merged, sort sets for reproducible manual pages)
Chris Lamb's avatar
Chris Lamb committed
    * [ldc](https://github.com/ldc-developers/ldc/pull/2812) (merged, drop CPU type from manpage)
Chris Lamb's avatar
Chris Lamb committed
    * [mapcrafter](https://github.com/mapcrafter/mapcrafter/pull/283) (use `convert -strip`)
Chris Lamb's avatar
Chris Lamb committed
39 40 41 42 43
    * [nfoview](https://build.opensuse.org/request/show/638547) (version-update, filesyssystem)
    * [obs-build](https://github.com/openSUSE/obs-build/pull/466) (use `virtio-serial`)
    * [plasma5-desktop](https://bugzilla.opensuse.org/show_bug.cgi?id=1109420) (parallelism/race)
    * [post-build-checks](https://bugzilla.opensuse.org/show_bug.cgi?id=1109470) (bug from arch-dependent `noarch` package)
    * [something-for-rabbit](https://build.opensuse.org/request/show/638283) (make it `noarch`)
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed
    * [vit](https://build.opensuse.org/request/show/638282) (date)
Chris Lamb's avatar
Chris Lamb committed
    * [yast-x11](https://github.com/yast/yast-x11/pull/18) (merged, sort hash, `noarch`)
Chris Lamb's avatar
Chris Lamb committed
46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

* Chris Lamb:

    * [#909476](https://bugs.debian.org/909476) filed against [botan](https://tracker.debian.org/pkg/botan).
    * [#909568](https://bugs.debian.org/909568) filed against [infnoise](https://tracker.debian.org/pkg/infnoise).
    * [#909782](https://bugs.debian.org/909782) filed against [libinput](https://tracker.debian.org/pkg/libinput).

diffoscope development

[diffoscope](https://diffoscope.org/) (our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages) was [updated in Debian unstable](https://tracker.debian.org/news/989165/accepted-diffoscope-102-source-into-unstable/) by Mattia Rizzolo. It included [contributions already covered in previous weeks](https://salsa.debian.org/reproducible-builds/diffoscope/commits/102) but also included new changes from:

* Chris Lamb:
    * [Strip trailing whitespace from `ssconvert(1)` output to support gnumeric 1.12.43](https://salsa.debian.org/reproducible-builds/diffoscope/commit/e9d3062),

* Mattia Rizzolo:
    * [Ignore continuous-integration failures in Debian testing](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7f9f0ac).
    * [Adapt tests to the output of gnumeric 1.12.43](https://salsa.debian.org/reproducible-builds/diffoscope/commit/ef5ae49).
    * [In Debian, list `liblz4-tool` as an alternative to the lz4 package](https://salsa.debian.org/reproducible-builds/diffoscope/commit/89e7e67).

Test framework development

There were a huge number of updates to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org/) by Holger Levsen this week, including:
Chris Lamb's avatar
Chris Lamb committed
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121

* [Arch Linux](https://www.archlinux.org/)-specific changes:

    * [Introduce user/group variations](https://salsa.debian.org/qa/jenkins.debian.net/commit/3977ee6a) in order to detect reproducibility issues.
    * [Add a script to manually schedule packages](https://salsa.debian.org/qa/jenkins.debian.net/commit/d1c149f8).
    * [Detect failure to download from Git repositories](https://salsa.debian.org/qa/jenkins.debian.net/commit/9d91a3fb).
    * Perform a [`chown(1)` call on the build directory to the `build2` user](https://salsa.debian.org/qa/jenkins.debian.net/commit/30d9ce83) to ensure it becomes writeable.
    * [Announce summary of scheduled packages again](https://salsa.debian.org/qa/jenkins.debian.net/commit/41620d4a).
    * [Fix scheduling of old packages](https://salsa.debian.org/qa/jenkins.debian.net/commit/2738c0f2).
    * [Fix `CSV` statistic storage](https://salsa.debian.org/qa/jenkins.debian.net/commit/a4406fec).
    * Reschedule ["depwait" and "404" packages after 24 hours has elapsed](https://salsa.debian.org/qa/jenkins.debian.net/commit/0bf88f0f), [packages in 404\_ state after two days](https://salsa.debian.org/qa/jenkins.debian.net/commit/c0f02102) and [`DEPWAIT` state after two days](https://salsa.debian.org/qa/jenkins.debian.net/commit/0db4c960).
    * [Ignore long-living schroots in maintenance job](https://salsa.debian.org/qa/jenkins.debian.net/commit/1ff6d6ec) which was finding old schroots by accident.
    * [Correctly detect and report if pacman cannot lock its database](https://salsa.debian.org/qa/jenkins.debian.net/commit/0810f04c).
    * [Detect "The requested URL returned error: 504" (gateway timeout)](https://salsa.debian.org/qa/jenkins.debian.net/commit/0be9051e) build failures.
    * [Create pages of packages in "repository X in Y" states](https://salsa.debian.org/qa/jenkins.debian.net/commit/b62d74f4).
    * [Create a job to update Archlinux webpages every thirty minutes](https://salsa.debian.org/qa/jenkins.debian.net/commit/0bc16c61) as well as [create links between pages](https://salsa.debian.org/qa/jenkins.debian.net/commit/777ffcdf).
    * [Use the `${PKG_ID}` variable consistently](https://salsa.debian.org/qa/jenkins.debian.net/commit/e9d77190)

* [Debian GNU/Linux](https://www.debian.org/)-specific changes:

    * Mark [odxu4a as offline](https://salsa.debian.org/qa/jenkins.debian.net/commit/44c1c9da) and [then back up](https://salsa.debian.org/qa/jenkins.debian.net/commit/78b8f1e9) (!)
    * [Extend the `check_node_is_up()` utility](https://salsa.debian.org/qa/jenkins.debian.net/commit/72a6c362). This ensures that nodes not accidentally marked as offline
    * [Set `man-db/auto-update` debconf to false and configure `dpkg` to use the `force-unsafe-io` flag](https://salsa.debian.org/qa/jenkins.debian.net/commit/82e2d938) to speed up builds.

* Misc changes:

    * Shorten IRC messages ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/ba3eaa65) & [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/f76ce377)).
    * Provide overviews of [monthly](https://salsa.debian.org/qa/jenkins.debian.net/commit/4a8a2e23) graphs from [Munin](http://munin-monitoring.org/) as well as [yearly](https://salsa.debian.org/qa/jenkins.debian.net/commit/34df443b).
    * [Mark nodes as offline after hours of failures](https://salsa.debian.org/qa/jenkins.debian.net/commit/f8d3d67b).
    * [Configure iptables to drop incoming UDP packets](https://salsa.debian.org/qa/jenkins.debian.net/commit/ae6b92f1) - thanks, Bernhard M. Wiedemann!
    * [Ignore Archlinux builds](https://salsa.debian.org/qa/jenkins.debian.net/commit/2f701ff7)  when looking for breakages.
    * [Fix the MacPorts diffoscope check](https://salsa.debian.org/qa/jenkins.debian.net/commit/2af0e998).
    * [Add a new "job health overview" page](https://salsa.debian.org/qa/jenkins.debian.net/commit/9b48b877).
    * [Split `html_nodes_info` job into two jobs](https://salsa.debian.org/qa/jenkins.debian.net/commit/4f98a241) for flexibility.
    * [Keep fewer log files](https://salsa.debian.org/qa/jenkins.debian.net/commit/b06082e9).
    * [Add notes about which tables Debian is using](https://salsa.debian.org/qa/jenkins.debian.net/commit/cc7db6f7) as some might be useful for Arch Linux too.

In addition, Eli Schwartz [refactored the grepping of build logs into a helper function](https://salsa.debian.org/qa/jenkins.debian.net/commit/6af7b40b), and Mattia Rizzolo made the following changes:

* [Enable all the reproducible-related build flags from `dpkg` by exporting `DEB_BUILD_OPTIONS=reproducible=+all`](https://salsa.debian.org/qa/jenkins.debian.net/commit/48607530).
* [Fix the check for patched packages since we moved from Alioth to Salsa](https://salsa.debian.org/qa/jenkins.debian.net/commit/1cc7bcd6).
* [Don't consider "unknown" suites while building the history page](https://salsa.debian.org/qa/jenkins.debian.net/commit/eb1d60f0).
* [Aid debugging by defining a `__str__` Python "magic" method](https://salsa.debian.org/qa/jenkins.debian.net/commit/4d9ea9e5).
* [Fix a typo in `build.sh`](https://salsa.debian.org/qa/jenkins.debian.net/commit/6fd46e4f).


This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.