Commit a6f26ccb authored by Chris Lamb's avatar Chris Lamb 💬

179: Initial draft.

parent 6160f2a3
......@@ -3,40 +3,117 @@ layout: blog
week: 179
---
* [fontconfig fix landed in Debian](https://bugs.debian.org/864082)
Here's what happened in the [Reproducible Builds](https://reproducible-builds.org) effort between Sunday September 23 and Saturday September 29 2018:
* FIXME: Guillem uploaded dpkg 1.19.1 with these changes relevant for Reproducible Debian:
* Dump database package records in alphabetical order. This will give
reproducible status and available database files, and make it possible
to output other deb822 formatted data in a deterministic way.
* Dpkg::Vendor::Debian: Add fixfilepath support to reproducible feature.
* Another reminder that the **fourth Reproducible Builds summit will take place in December 11th—13th 2018** at [Mozilla](https://wiki.mozilla.org/Paris)'s offices in Paris, France. If you are interested in attending please register by sending an email to `holger@layer-acht.org`. More details may be found on the [associated event page](https://reproducible-builds.org/events/paris2018/).
* Bernhard M. Wiedemann discovered problems from [arch-dependent noarch packages](https://bugzilla.opensuse.org/show_bug.cgi?id=1109534) and there are actually over 1000 of these in openSUSE.
* Holger Levsen is requesting a for review and comments on [a on a multi-project syntax notes proposal](https://salsa.debian.org/reproducible-builds/reproducible-notes/blob/multi-project-syntax/README). He hopes to implement this in the next few weeks.
* [A change](https://github.com/python/cpython/pull/9607) is proposed to python-3.7+ so that `SOURCE_DATE_EPOCH` only influences the default type of .pyc but does not force it.
* [A pull request was proposed](https://github.com/python/cpython/pull/9607) to [CPython](https://github.com/python/cpython) to ensure [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/specs/source-date-epoch/) only influences the default type of `.pyc` files yet does not enforce it.
* Jelle van der Waa started a [To Do list](https://www.archlinux.org/todo/imagemagicks-convert-not-reproducible-in-pkgbuilds/) to fix unreproducible PKGBUILD's in Arch Linux which use imagemagick's convert for image conversion which embeds the date:create and date:modify.
* [Guillem Jover](https://www.hadrons.org/~guillem/) uploaded [dpkg 1.19.1 to Debian unstable](https://tracker.debian.org/news/989886/accepted-dpkg-1191-source-into-unstable/), adding support for dumping database package records in alphabetical order in order to provide reproducible status and available database files, and furhte rmake it possible to output other "deb822"-formatted data in a deterministic way. In addition, the `Dpkg::Vendor::Debian` library gained support for a `fixfilepath` feature.
How you can help
----------------
* Jelle van der Waa [started a TODO list](https://www.archlinux.org/todo/imagemagicks-convert-not-reproducible-in-pkgbuilds/) to fix unreproducible `PKGBUILD` files in [Arch Linux](https://www.archlinux.org/) which use the [Imagemagick](https://www.imagemagick.org/script/index.php) graphics library's `convert(1)` utility for image manipulation which embeds the `date:create` and `date:modify` file headers.
* Holger is asking for review and comments on https://salsa.debian.org/reproducible-builds/reproducible-notes/blob/multi-project-syntax/README compared to the README in git master, as he hopes to implement that in the next 1-3 weeks.
* `disorderfs` version `0.5.4-1` (our [FUSE](https://github.com/libfuse/libfuse)-based filesystem that deliberately introduces non-determinism into filesystem metadata) was [uploaded to Debian unstable](https://tracker.debian.org/news/989895/accepted-disorderfs-054-1-source-amd64-into-unstable/) by Chris Lamb. It included a number of contributions [already covered in previous weeks](https://salsa.debian.org/reproducible-builds/disorderfs/commits/debian/0.5.4-1).
* A fix to ensure to that [fontconfig](https://www.freedesktop.org/wiki/Software/fontconfig/) — a library for configuring and customizing font access — [landed in Debian unstable](https://bugs.debian.org/864082#96) (via upstream).
* Chris Lamb added four Debian package reviews. In addition, three were updated and eleven were removed in this week, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html).
* Bernhard M. Wiedemann discovered problems from [arch-dependent `noarch` packages](https://bugzilla.opensuse.org/show_bug.cgi?id=1109534) in [openSUSE](https://www.opensuse.org/); there are actually over 1,000 of these.
Patches filed
-------------
* Bernhard M. Wiedemann:
* [ldc](https://github.com/ldc-developers/ldc/pull/2812) (merged, drop CPU type from man-page)
* [plasma5-desktop](https://bugzilla.opensuse.org/show_bug.cgi?id=1109420) (parallelism/race)
* [obs-build](https://github.com/openSUSE/obs-build/pull/466) (use virtio-serial)
* [build-compare](https://github.com/openSUSE/build-compare/pull/30) (better quoting of strings)
* [kubernetes](https://github.com/kubernetes/kubernetes/pull/68983) (sort sets for reproducible man pages)
* [cpio](https://build.opensuse.org/request/show/638582) (drop date using `gettextize --no-changelog`)
* [foma/malaga-suomi](https://github.com/mhulden/foma/pull/78) (merged, segfault bug, noarch)
* [post-build-checks](https://bugzilla.opensuse.org/show_bug.cgi?id=1109470) (bug from arch-dependent noarch pkg)
* [yast-x11](https://github.com/yast/yast-x11/pull/18) (sort hash, noarch)
* [something-for-rabbit](https://build.opensuse.org/request/show/638283) (make it noarch)
* [nfoview](https://build.opensuse.org/request/show/638547) (version-update, filesys)
* [cpio](https://build.opensuse.org/request/show/638582) (drop date using gettextize --no-changelog)
* [kubernetes](https://github.com/kubernetes/kubernetes/pull/68983) (sort sets for reproducible manual pages)
* [ldc](https://github.com/ldc-developers/ldc/pull/2812) (merged, drop CPU type from manpage)
* [nfoview](https://build.opensuse.org/request/show/638547) (version-update, filesyssystem)
* [obs-build](https://github.com/openSUSE/obs-build/pull/466) (use `virtio-serial`)
* [plasma5-desktop](https://bugzilla.opensuse.org/show_bug.cgi?id=1109420) (parallelism/race)
* [post-build-checks](https://bugzilla.opensuse.org/show_bug.cgi?id=1109470) (bug from arch-dependent `noarch` package)
* [something-for-rabbit](https://build.opensuse.org/request/show/638283) (make it `noarch`)
* [vit](https://build.opensuse.org/request/show/638282) (date)
* [yast-x11](https://github.com/yast/yast-x11/pull/18) (sort hash, noarch)
* Chris Lamb:
* [#909476](https://bugs.debian.org/909476) filed against [botan](https://tracker.debian.org/pkg/botan).
* [#909568](https://bugs.debian.org/909568) filed against [infnoise](https://tracker.debian.org/pkg/infnoise).
* [#909782](https://bugs.debian.org/909782) filed against [libinput](https://tracker.debian.org/pkg/libinput).
diffoscope development
----------------------
[diffoscope](https://diffoscope.org/) (our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages) was [updated in Debian unstable](https://tracker.debian.org/news/989165/accepted-diffoscope-102-source-into-unstable/) by Mattia Rizzolo. It included [contributions already covered in previous weeks](https://salsa.debian.org/reproducible-builds/diffoscope/commits/102) but also included new changes from:
* Chris Lamb:
* [Strip trailing whitespace from `ssconvert(1)` output to support gnumeric 1.12.43](https://salsa.debian.org/reproducible-builds/diffoscope/commit/e9d3062),
* Mattia Rizzolo:
* [Ignore continuous-integration failures in Debian testing](https://salsa.debian.org/reproducible-builds/diffoscope/commit/7f9f0ac).
* [Adapt tests to the output of gnumeric 1.12.43](https://salsa.debian.org/reproducible-builds/diffoscope/commit/ef5ae49).
* [In Debian, list `liblz4-tool` as an alternative to the lz4 package](https://salsa.debian.org/reproducible-builds/diffoscope/commit/89e7e67).
Test framework development
--------------------------
There were a huge number of updates to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](tests.reproducible-builds.org) by Holger Levsen this month, including:
* [Arch Linux](https://www.archlinux.org/)-specific changes:
* [Introduce user/group variations](https://salsa.debian.org/qa/jenkins.debian.net/commit/3977ee6a) in order to detect reproducibility issues.
* [Add a script to manually schedule packages](https://salsa.debian.org/qa/jenkins.debian.net/commit/d1c149f8).
* [Detect failure to download from Git repositories](https://salsa.debian.org/qa/jenkins.debian.net/commit/9d91a3fb).
* Perform a [`chown(1)` call on the build directory to the `build2` user](https://salsa.debian.org/qa/jenkins.debian.net/commit/30d9ce83) to ensure it becomes writeable.
* [Announce summary of scheduled packages again](https://salsa.debian.org/qa/jenkins.debian.net/commit/41620d4a).
* [Fix scheduling of old packages](https://salsa.debian.org/qa/jenkins.debian.net/commit/2738c0f2).
* [Fix `CSV` statistic storage](https://salsa.debian.org/qa/jenkins.debian.net/commit/a4406fec).
* Reschedule ["depwait" and "404" packages after 24 hours has elapsed](https://salsa.debian.org/qa/jenkins.debian.net/commit/0bf88f0f), [packages in 404\_ state after two days](https://salsa.debian.org/qa/jenkins.debian.net/commit/c0f02102) and [`DEPWAIT` state after two days](https://salsa.debian.org/qa/jenkins.debian.net/commit/0db4c960).
* [Ignore long-living schroots in maintenance job](https://salsa.debian.org/qa/jenkins.debian.net/commit/1ff6d6ec) which was finding old schroots by accident.
* [Correctly detect and report if pacman cannot lock its database](https://salsa.debian.org/qa/jenkins.debian.net/commit/0810f04c).
* [Detect "The requested URL returned error: 504" (gateway timeout)](https://salsa.debian.org/qa/jenkins.debian.net/commit/0be9051e) build failures.
* [Create pages of packages in "repository X in Y" states](https://salsa.debian.org/qa/jenkins.debian.net/commit/b62d74f4).
* [Create a job to update Archlinux webpages every thirty minutes](https://salsa.debian.org/qa/jenkins.debian.net/commit/0bc16c61) as well as [create links between pages](https://salsa.debian.org/qa/jenkins.debian.net/commit/777ffcdf).
* [Use the `${PKG_ID}` variable consistently](https://salsa.debian.org/qa/jenkins.debian.net/commit/e9d77190)
* [Debian GNU/Linux](https://www.debian.org/)-specific changes:
* Mark [odxu4a as offline](https://salsa.debian.org/qa/jenkins.debian.net/commit/44c1c9da) and [then back up](https://salsa.debian.org/qa/jenkins.debian.net/commit/78b8f1e9) (!)
* [Extend the `check_node_is_up()` utility](https://salsa.debian.org/qa/jenkins.debian.net/commit/72a6c362). This ensures that nodes not accidentally marked as offline
* [Set `man-db/auto-update` debconf to false and configure `dpkg` to use the `force-unsafe-io` flag](https://salsa.debian.org/qa/jenkins.debian.net/commit/82e2d938) to speed up builds.
* Misc changes:
* Shorten IRC messages ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/ba3eaa65) & [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/f76ce377)).
* Provide overviews of [monthly](https://salsa.debian.org/qa/jenkins.debian.net/commit/4a8a2e23) graphs from [Munin](http://munin-monitoring.org/) as well as [yearly](https://salsa.debian.org/qa/jenkins.debian.net/commit/34df443b).
* [Mark nodes as offline after hours of failures](https://salsa.debian.org/qa/jenkins.debian.net/commit/f8d3d67b).
* [Configure iptables to drop incoming UDP packets](https://salsa.debian.org/qa/jenkins.debian.net/commit/ae6b92f1) - thanks, Bernhard M. Wiedemann!
* [Ignore Archlinux builds](https://salsa.debian.org/qa/jenkins.debian.net/commit/2f701ff7) when looking for breakages.
* [Fix the MacPorts diffoscope check](https://salsa.debian.org/qa/jenkins.debian.net/commit/2af0e998).
* [Add a new "job health overview" page](https://salsa.debian.org/qa/jenkins.debian.net/commit/9b48b877).
* [Split `html_nodes_info` job into two jobs](https://salsa.debian.org/qa/jenkins.debian.net/commit/4f98a241) for flexibility.
* [Keep fewer log files](https://salsa.debian.org/qa/jenkins.debian.net/commit/b06082e9).
* [Add notes about which tables Debian is using](https://salsa.debian.org/qa/jenkins.debian.net/commit/cc7db6f7) as some might be useful for Arch Linux too.
In addition, Eli Schwartz [refactored the grepping of build logs into a helper function](https://salsa.debian.org/qa/jenkins.debian.net/commit/6af7b40b), and Mattia Rizzolo made the following changes:
* [Enable all the reproducible-related build flags from `dpkg` by exporting `DEB_BUILD_OPTIONS=reproducible=+all`](https://salsa.debian.org/qa/jenkins.debian.net/commit/48607530).
* [Fix the check for patched packages since we moved from Alioth to Salsa](https://salsa.debian.org/qa/jenkins.debian.net/commit/1cc7bcd6).
* [Don't consider "unknown" suites while building the history page](https://salsa.debian.org/qa/jenkins.debian.net/commit/eb1d60f0).
* [Aid debugging by defining a `__str__` Python "magic" method](https://salsa.debian.org/qa/jenkins.debian.net/commit/4d9ea9e5).
* [Fix a typo in `build.sh`](https://salsa.debian.org/qa/jenkins.debian.net/commit/6fd46e4f).
Misc.
-----
This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment