181.md 11.2 KB
Newer Older
Chris Lamb's avatar
Chris Lamb committed
1
---
2
layout: new/blog
Chris Lamb's avatar
Chris Lamb committed
3
week: 181
4
published: 2018-10-16 20:23:24
Chris Lamb's avatar
Chris Lamb committed
5
---
6

Chris Lamb's avatar
Chris Lamb committed
7 8
Here's what happened in the [Reproducible Builds](https://reproducible-builds.org) effort between Sunday October 7 and Saturday October 13 2018:

9
* Holger gave a talk about [diffoscope](https://diffoscope.org/) at the [MirageOS](https://mirage.io/) [hacker retreat](http://retreat.mirage.io). This resulted in three bugs being filed ([#910540](https://bugs.debian.org/910540) (*backtrace when using diffoscope on MacOS*, [#910542](https://bugs.debian.org/910542) (*Please add better support for OCaml object files*), & [#910541](https://bugs.debian.org/910541) (*Filing bugs on diffoscope is cumbersome for non-Debian contributors*).
Chris Lamb's avatar
Chris Lamb committed
10

11
* In the upcoming weeks, Chris Lamb and kpcyrd will be attending [New York University](https://www.nyu.edu/) to work with students from the *Application Security* course on the topic of reproducible builds.
Chris Lamb's avatar
Chris Lamb committed
12 13 14 15 16 17 18 19 20 21 22

* Richard Purdie reported that the [Yocto Project](https://www.yoctoproject.org/) now offers [reproducible builds for arbitrary paths and build users](https://lists.reproducible-builds.org/pipermail/rb-general/2018-October/001203.html). This is only tested with `core-image-sato`, an [X-based](https://en.wikipedia.org/wiki/X_Window_System) graphical image but means the core system components are reproducible, providing a solid base that further improvements can be made from.

* [Mes](https://gitlab.com/janneke/mes) (a Scheme-based compiler for our "sister" [bootstrappable builds](http://bootstrappable.org) effort) [announced their 0.18 release](http://lists.gnu.org/archive/html/info-gnu/2018-10/msg00000.html).

* Mattia Rizzolo updated the [Contribute](https://reproducible-builds.org/contribute/) section on [our website](https://reproducible-builds.org/), removing some references from the now-deprecated [Alioth](https://wiki.debian.org/Alioth) Debian service. He also [added a handy link to file a new bug report](https://salsa.debian.org/reproducible-builds/diffoscope-website/commit/d1859e5) on the [diffoscope.org](https://diffoscope.org/) website.

* disorderfs (our [FUSE](https://github.com/libfuse/libfuse)-based filesystem that deliberately introduces non-determinism into filesystem metadata) version `0.5.4-2` was [uploaded to Debian unstable](https://tracker.debian.org/news/995207/accepted-disorderfs-054-2-source-into-unstable/) by Holger Levsen. It [included contributions already covered in previous weeks](https://salsa.debian.org/reproducible-builds/disorderfs/commits/debian/0.5.4-2) but also include some fixes to the the `debian/watch` file ([1](https://salsa.debian.org/reproducible-builds/disorderfs/commit/494a411), [2](https://salsa.debian.org/reproducible-builds/disorderfs/commit/e099a4e) & [3](https://salsa.debian.org/reproducible-builds/disorderfs/commit/4c803a4)) and was migrated to [use the new `debhelper-compat (= 11)`](https://salsa.debian.org/reproducible-builds/disorderfs/commit/9c648d3) notation.

* Chris Lamb reviewed 25 Debian packages and 28 were removed in this week, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html).Chris also added a new [`timestamps_in_episode_files_generated_by_jxc`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/0eab9dbe) toolchain issue and [expanded the comments](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/92bdc5bb) for [`translations_missing_in_desktop_files`](https://tests.reproducible-builds.org/debian/issues/unstable/translations_missing_in_desktop_files_issue.html).

23
* strip-nondeterminism version `0.043-2` was [uploaded to Debian unstable](https://tracker.debian.org/news/995203/accepted-strip-nondeterminism-0043-2-source-into-unstable/) by Mattia Rizzolo. It [included contributions already covered in previous weeks](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commits/debian/0.043-2) as well [using the new `debhelper-compat (= 11)` notation](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/deba9e1) and [fixing the `debian/watch` file](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/commit/8c7200b) after moving from the now-deprecated [Alioth](https://wiki.debian.org/Alioth) Debian infrastructure.
Chris Lamb's avatar
Chris Lamb committed
24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

Another brief reminder that another Reproducible Builds summit will be taking place between 11th—13th December 2018 in [Mozilla](https://wiki.mozilla.org/Paris)'s offices in Paris. If you are interested in attending please send an email to `holger@layer-acht.org`. More details can also be found on the [corresponding event page](https://reproducible-builds.org/events/paris2018/) of our website.


diffoscope development
----------------------

[diffoscope](https://diffoscope.org/) is our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages) was updated this week, including contributions from:

* Chris Lamb:
    * [Use `str.startswith(...)` over `str.index(...) == 0` in the Macho comparator to prevent tracebacks if text cannot be found on the line](https://salsa.debian.org/reproducible-builds/diffoscope/commit/56f8215). ([#910540](https://bugs.debian.org/910540)).

* Marek Marczykowski-Górecki:
    * [Handle FAT filesystems with the `fsimage` comparator](https://salsa.debian.org/reproducible-builds/diffoscope/commit/da3b97a).
    * [Add a test for FAT `fsimage` comparator](https://salsa.debian.org/reproducible-builds/diffoscope/commit/575f99b).

* Mattia Rizzolo:
    * Adjust some build dependencies: Add [`linux-image-amd64 [amd64]` to allow the SquashFS tests to run](https://salsa.debian.org/reproducible-builds/diffoscope/commit/225a30b) and [`linux-image-generic`](https://salsa.debian.org/reproducible-builds/diffoscope/commit/b90bd2f).
    * Adjust "autopkgtest" dependencies: Add [`linux-image-amd64`](https://salsa.debian.org/reproducible-builds/diffoscope/commit/c6d2511) as well as [the optional Python modules](https://salsa.debian.org/reproducible-builds/diffoscope/commit/71eb7e3).
    * [Move the list of optional Python packages to `setup.py` and have `dh_python3` check the generated `requires.txt](https://salsa.debian.org/reproducible-builds/diffoscope/commit/1e109bc).
    * [Add a `Build-Conflicts` against `graphicsmagick-imagemagick-compat`](https://salsa.debian.org/reproducible-builds/diffoscope/commit/dc3634e).
    * [Temporarily drop `{Build,Test}-Depends` for `apktool` and `oggvideotools`; they are not available in `buster` at the moment](https://salsa.debian.org/reproducible-builds/diffoscope/commit/018b7ae).
    * [Rename `procyon-decompiler` to `procyon`](https://salsa.debian.org/reproducible-builds/diffoscope/commit/d7ec996),
47

Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed
48 49 50 51 52

Packages reviewed and fixed, and bugs filed
-------------------------------------------

* Bernhard M. Wiedemann:
Chris Lamb's avatar
Chris Lamb committed
53 54
    * [hub](https://github.com/github/hub/pull/1892) (`Makefile` fix)
    * [imagemagick](https://github.com/ImageMagick/ImageMagick/issues/1332) (`noarch` diff in png + gif)
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed
55
    * [ninja](https://github.com/ninja-build/ninja/commit/5fcdcf95cb62ab3d593c36ef90df27cef63874a1#r30852768) (discovered unreleased fix for FTBFS-2038)
Chris Lamb's avatar
Chris Lamb committed
56
    * [perl-MooseX-Role-Cmd](https://build.opensuse.org/request/show/640849) (make `noarch`)
Bernhard M. Wiedemann's avatar
Bernhard M. Wiedemann committed
57
    * [perl-Term-Size-Perl](https://rt.cpan.org/Public/Bug/Display.html?id=127333) (date)
58
    * [pocl](https://github.com/pocl/pocl/pull/667) (fixed, compile-time CPU-detection)
Chris Lamb's avatar
Chris Lamb committed
59 60 61 62 63 64 65 66 67 68 69 70 71
    * [python-keystoneauth1](https://bugs.launchpad.net/keystoneauth/+bug/1796899) (randomness in docs)
    * [ronn](https://github.com/kamontat/ronn/pull/3) (date)

* Chris Lamb:
    * [#910490](https://bugs.debian.org/910490) filed against [libhandy](https://tracker.debian.org/pkg/libhandy).
    * [#910655](https://bugs.debian.org/910655) filed against [fast5](https://tracker.debian.org/pkg/fast5).
    * [#910752](https://bugs.debian.org/910752) filed against [syndie](https://tracker.debian.org/pkg/syndie).
    * [#910789](https://bugs.debian.org/910789) filed against [opari2](https://tracker.debian.org/pkg/opari2).


Test framework development
--------------------------

72
There were a large number of updates to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org/) by Holger Levsen this week, including:
Chris Lamb's avatar
Chris Lamb committed
73 74 75 76

* [Arch Linux](https://www.archlinux.org/)-specific changes:
    * [Temporarily disable building with `build2` and `build` as the jenkins user](https://salsa.debian.org/qa/jenkins.debian.net/commit/e76b96c6). This was an attempt to diagnose a path-variation issue.
    * [Also call `printenv` before the builds to diagnose path issue](https://salsa.debian.org/qa/jenkins.debian.net/commit/bed37f55).
77
    * [Always force PATH to match `makechrootpkg`](https://salsa.debian.org/qa/jenkins.debian.net/commit/8d2d1b29), else Arch Linux builds vary as the include the current PATH in the build results.
Chris Lamb's avatar
Chris Lamb committed
78
    * [Increase timeout for running `diffoscope` to one hour](https://salsa.debian.org/qa/jenkins.debian.net/commit/628e4da5).
79
    * [Trap "exit" so we properly end the `schroot` sessions on failures](https://salsa.debian.org/qa/jenkins.debian.net/commit/43fa8d16).
Chris Lamb's avatar
Chris Lamb committed
80 81 82 83 84
    * [Include the number of recent builds on the `recent_builds.html` page](https://salsa.debian.org/qa/jenkins.debian.net/commit/c30aaa7e).
    * [Drop an unused variable](https://salsa.debian.org/qa/jenkins.debian.net/commit/53ab78ff).

* [Debian GNU/Linux](https://www.debian.org/)-specific changes:

85
    * Re-add [8 `i386` builders](https://salsa.debian.org/qa/jenkins.debian.net/commit/af220237) and [8 `arm64` builders](https://salsa.debian.org/qa/jenkins.debian.net/commit/0cb66d2b) as the relevant build nodes are not using all the memory they have available to them.
Chris Lamb's avatar
Chris Lamb committed
86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102

* Misc/generic changes:

    * Add a new [job health page](https://jenkins.debian.net/userContent/jenkins_job_health.html) ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/e41356e1), [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/004ab3c3), [3](https://salsa.debian.org/qa/jenkins.debian.net/commit/7640a773), [4](https://salsa.debian.org/qa/jenkins.debian.net/commit/7836b079), etc.)
    * [Warn on  files  greater than 8G in `/var/log` and fail maintenance job if files greater than 32G are found](https://salsa.debian.org/qa/jenkins.debian.net/commit/ec09e960).
    * [Add health-check and maintenance jobs for the `pb10` node](https://salsa.debian.org/qa/jenkins.debian.net/commit/2716e823).
    * [Only check for outdated running kernels once per day; the maintenance job also checks this](https://salsa.debian.org/qa/jenkins.debian.net/commit/4d6b2a77).
    * [Fix path to `/var/spool/squid/` for compatibility with `stretch`](https://salsa.debian.org/qa/jenkins.debian.net/commit/a492bd5b).
    * Perform some node maintenance. ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/1332da65), [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/9993e59c), [3](https://salsa.debian.org/qa/jenkins.debian.net/commit/bd143021), etc.).

In addition, Mattia Rizzolo performed some node administration ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/c8731d91), [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/daeeb033)).


Misc.
-----

This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.