Skip to content
Commits on Source (2)
......@@ -3,41 +3,84 @@ layout: new/blog
week: 196
---
* FIXME: tracker.debian.org/nix is finally available in Debian
Here's what happened in the [Reproducible Builds](https://reproducible-builds.org) effort between Sunday January 20 and Saturday January 26 2019:
* [FIXME](https://github.com/pytroll/satpy/pull/579#issuecomment-455991066)
* There was considerable progresss in making the [Debian Installer](https://www.debian.org/devel/debian-installer/) images reproducible with a [number of rounds of code review](https://alioth-lists.debian.net/pipermail/reproducible-builds/Week-of-Mon-20190114/011054.html), a subsequent merge of [Chris Lamb](https://chris-lamb.co.uk/)'s [merge request](https://salsa.debian.org/installer-team/debian-installer/merge_requests/3) and closing of the [corresponding bug report](https://bugs.debian.org/900918#29) for the time being, pending further testing.
* [FIXME](https://linux.conf.au/schedule/presentation/185/)
* At [linux.conf.au](https://linux.conf.au/) 2019 in Christchurch, New Zealand there were (at least!) two talks that touched on Reproducible Builds. First, [Benno Rice](https://twitter.com/jeamland) gave a talk titled "[How Much Do You Trust That Package? Understanding The Software Supply Chain](https://2019.linux.conf.au/schedule/presentation/237/)" ([YouTube link](https://www.youtube.com/watch?v=fnELtqE6mMM)). In addition, [Aleksandra Pawlik](https://github.com/apawlik) presented on "[Building reproducible computing environments: a workshop for non-experts](https://2019.linux.conf.au/schedule/presentation/185/)" ([YouTube link](https://www.youtube.com/watch?v=B2xzQFQ2hg0)).
* [etc.](https://alioth-lists.debian.net/pipermail/reproducible-builds/Week-of-Mon-20190114/011054.html)
* [diffoscope](https://diffoscope.org/) is our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages. There were a few updates this week from Chris Lamb, including not crashing if we were unable to successfully extract a [guestfs](http://libguestfs.org/) filesystem [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/dda7713)][[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/6b8e47d)] ([#901982](https://bugs.debian.org/901982)), avoid clumsy profiling title length calculations by using [Markdown](https://en.wikipedia.org/wiki/Markdown) syntax [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/ef2aa62)] and drop the printing out [`dpkg-query(1)`](https://linux.die.net/man/1/dpkg-query) output when running tests. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/d98c4b0)]
* Mattia updated the expiration of the GPG key used to sign our [experimental debian archive](https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain)
* The [compiler for the Elixir language](https://github.com/elixir-lang/elixir) received a number of updates [to make it compile in 2019](https://github.com/elixir-lang/elixir/issues/8702) and [in 2020](https://github.com/elixir-lang/elixir/pull/8688) and to [create all its `.beam` files in a reproducible manner](https://github.com/elixir-lang/elixir/issues/8689) which permitted the creation of reproducible [openSUSE](https://www.opensuse.org/) packages. They also are [adding reproducibility tests](https://github.com/elixir-lang/elixir/pull/8701) to their continuous-integration system so that it does not regress in the future.
* [How Much Do You Trust That Package- Understanding The Software Supply Chain](https://www.youtube.com/watch?v=fnELtqE6mMM)
* Chris Lamb's [historical summary and a request for action](https://lists.freedesktop.org/archives/fontconfig/2019-January/006420.html) posted on [Fontconfig](https://www.freedesktop.org/wiki/Software/fontconfig/)'s [mailing list](https://lists.freedesktop.org/archives/fontconfig/) in order that a solution may be found and included in Debian *buster*, has resulted in a [considerable process and discussion](https://lists.freedesktop.org/archives/fontconfig/2019-January/006464.html) on the upstream mailing list.
* [FIXME](https://ssl.engineering.nyu.edu/blog/2019-01-18-in-toto-paris)
* Hervé Boutemy made more updates to the [reproducible-builds.org](https://reproducible-builds.org) project website this week, including adding section on auditing a JVM build. [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/c7191b9)], defining `build.setup` as an optional field [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/6c01120)], explaining the distinction between build instructions vs effective environment [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ba2f561)] and detailed the [Maven](https://maven.apache.org/) rebuild instructions [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e587f04)].:w
* [FIXME](https://2019.linux.conf.au/schedule/presentation/185/)
* Marvin Humphrey started a thread on [our mailing list](https://lists.reproducible-builds.org/pipermail/rb-general/) this week on the [Definition of "reproducible build"](https://lists.reproducible-builds.org/pipermail/rb-general/2019-January/001426.html), referencing a thread thread on the [Apache Foundation](https://apache.org)'s [legal-discuss mailing list](https://lists.reproducible-builds.org/pipermail/rb-general/2019-January/001428.html).
* [forwarded 920409 upstream](https://github.com/benjsc/splitpatch/pull/10)
* Bernhard M. Wiedemann posted his monthly [Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2019-01/msg00393.html) for the [openSUSE](https://opensuse.org/) distribution.
* [FIXME: Next steps for a reproducible Fontconfig?](https://lists.freedesktop.org/archives/fontconfig/2019-January/006464.html)
* Reproducible builds were mentioned in [Episode 9](https://librelounge.org/episodes/episode-9-funding-free-software-development-pt2.html) of the [Libre Lounge](https://librelounge.org/) podcast in a more-general discussion about funding free software development. ([Direct link](https://pca.st/1MUC#t=23m00s))
* [The compiler for the elixir language](https://github.com/elixir-lang/elixir) got a number of updates to [make it compile in 2019](https://github.com/elixir-lang/elixir/issues/8702) and [2020](https://github.com/elixir-lang/elixir/pull/8688) and to [create all its .beam files in a reproducible way](https://github.com/elixir-lang/elixir/issues/8689). This allowed to build reproducible openSUSE packages. They also are [adding some reproducibility tests to their CI](https://github.com/elixir-lang/elixir/pull/8701) so that it does not break as easily in the future.
* The [Nix](https://nixos.org/nix) "purely functional package manager" was uploaded to Debian as version `2.2.1-2`, pending [processing in the NEW queue](https://ftp-master.debian.org/new/nix_2.2.1-2.html).
* Bernhard M. Wiedemann posted his [monthly openSUSE r-b status update](https://lists.opensuse.org/opensuse-factory/2019-01/msg00393.html)
* Lukas Pühringer [posted a report](https://ssl.engineering.nyu.edu/blog/2019-01-18-in-toto-paris) from the [in-toto](https://in-toto.github.io/) project's partication in the recent [Reproducible Builds summit in Paris](https://reproducible-builds.org/events/paris2018/).
* 10 Debian package reviews were added, 9 were updated and 20 were removed in this week, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). Two new issue types were added: [`randomness_in_ids_generated_by_org-html-publish-to-html`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/0baeb163) and [`ftbfs_due_to_f_file_prefix_map`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/a868eb81) by Chris Lamb and Mattia Rizzolo respectfully.
## Packages reviewed and fixed, and bugs filed
* Bernhard M. Wiedemann:
* [libqt5-qtwebengine](https://build.opensuse.org/request/show/667729) (date, already upstream)
* [pcre2](https://build.opensuse.org/request/show/668144) (PGO/parallelism)
* [perl](https://build.opensuse.org/request/show/668211) (ASLR, FTBFS-2020)
* [myman](https://build.opensuse.org/request/show/668574) (date+time)
* [nsnake](https://build.opensuse.org/request/show/668628) (date, filesys ; also added in [distropatches](https://github.com/distropatches/nSnake))
* [nDPI](https://github.com/ntop/nDPI/pull/662) (use changelog date)
* [python-IMDbPY](https://github.com/alberanid/imdbpy/pull/202) (sort python glob)
* [libqt5-qtwebengine](https://build.opensuse.org/request/show/667729): Date, already upstream.
* [myman](https://build.opensuse.org/request/show/668574): Date & time.
* [nDPI](https://github.com/ntop/nDPI/pull/662): Use changelog date.
* [nsnake](https://build.opensuse.org/request/show/668628): date, filesystem ordering, also added in [`distropatches.git`](https://github.com/distropatches/nSnake))
* [pcre2](https://build.opensuse.org/request/show/668144): [Profile-guided optimisation](https://en.wikipedia.org/wiki/Profile-guided_optimization) (PGO) / parallelism)
* [perl](https://build.opensuse.org/request/show/668211): [Address space layout randomization](https://en.wikipedia.org/wiki/Address_space_layout_randomization) (ASLR), fails to build in 2020.
* [python-IMDbPY](https://github.com/alberanid/imdbpy/pull/202): sort call to Python `glob.glob()`
* Chris Lamb:
* [#919566](https://bugs.debian.org/919566) filed against [satpy](https://tracker.debian.org/pkg/satpy) ([merged upstream](https://github.com/pytroll/satpy/pull/579#issuecomment-455991066)).
* [#920409](https://bugs.debian.org/920409) filed against [splitpatch](https://tracker.debian.org/pkg/splitpatch) ([forwarded upstream](https://github.com/benjsc/splitpatch/pull/10))
* [#920411](https://bugs.debian.org/920411) filed against [mongo-c-driver](https://tracker.debian.org/pkg/mongo-c-driver).
* [#920591](https://bugs.debian.org/920591) filed against [lambda-align2](https://tracker.debian.org/pkg/lambda-align2).
* [#920592](https://bugs.debian.org/920592) filed against [roaraudio](https://tracker.debian.org/pkg/roaraudio).
* [#920594](https://bugs.debian.org/920594) filed against [papi](https://tracker.debian.org/pkg/papi).
* [#920595](https://bugs.debian.org/920595) filed against [ukui-themes](https://tracker.debian.org/pkg/ukui-themes).
## Test framework development
We operate a comprehensive [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org). This week:
* Eli Schwartz:
* Fix the "preseed" of [Arch Linux](https://www.archlinux.org/)'s [PGP](https://www.gnupg.org/) keys by sending output to `stdout`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/da198a0d)]
* Holger Levsen:
* [Arch Linux](https://www.archlinux.org/)-specific changes:
* Refactor the scheduler's usage of `$repo` and `$REPO` variables "against insanity". [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/fa434bad)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/26e9cf88)]
* Correct a [fencepost error](https://en.wikipedia.org/wiki/Off-by-one_error#Fencepost_error) in the scheduler; if we want to request `n` packages we need to set a limit of `n + 1`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6ad77ebf)]
* Include a `n builds in the last 3h` statistic in the IRC notifications. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ca89fba0)]
* Schedule packages six times a day instead of eight. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/c5002281)]
* [F-Droid](https://f-droid.org/)-specific changes:
* Run the setup job three times a week now, building all apps daily. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/44e71a2d)]
* [LEDE](https://en.wikipedia.org/wiki/LEDE)/OpenWrt, [coreboot](https://www.coreboot.org/) and [NetBSD](https://www.netbsd.org/) changes:
* Test three times per week instead of just once. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/93fa99fa)]
* Move building to [OSUOSL](https://osuosl.org/) nodes 171 & 172, from `pb3` & `pb4`. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/2ddfd5b9)]
* Build at different times to not interfere. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f6fdcca8)]
* Misc/generic changes:
* Update status of the deployment of the new [OSUOSL](https://osuosl.org/) nodes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/4f7f5029)]
* Fix the [Debian](https://www.debian.org/) `dsa-check-running-kernel` to deal with the [Ubuntu](https://www.ubuntu.com/) LTS changes. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/eb9c7401)]
* Correct [KGB IRC interface](https://salsa.debian.org/kgb-team/kgb/wikis/Home)'s directory permissions and create it if it does not exist. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/88f1965f)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6b93ad9a)]
* Fix a bug that was preventing [OSUOSL](https://osuosl.org/) hosts from running correctly in the future. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/ee369979)]
* Set the correct permissions on the `jenkins` user's `~/.ssh` directory. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7d4e095a)]
* Node maintenance. ([[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/1eb162c4)], [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/46db251b)], [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/95d6582e)], etc.)
* Mattia Rizzolo:
* Update the expiration of the [GPG](https://www.gnupg.org/) key used to sign [our experimental Debian archive](https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f2d42e18)]
* In our [pbuilder](https://wiki.debian.org/PbuilderTricks) configuration, use the APT dependency resolver [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/cf6c8577)] simplify the section for `i386`/`armhf` hosts [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/0b7d2b9f)] and [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) the `MIRRORSITE` configuration, now that is the same for everything. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/d86fdb4d)]
* Node maintenance. ([[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/07021252)], [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/42cb05a4)], [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a8fe9887)], etc)
* [Mentioned in LibreLounge](https://pca.st/1MUC#t=23m00s) podcast
---
* [youtube video of linux.conf.au talk](https://www.youtube.com/watch?v=B2xzQFQ2hg0)
This week's edition was written by [Bernhard M. Wiedemann](https://lizards.opensuse.org/author/bmwiedemann/), [Chris Lamb](https://chris-lamb.co.uk/), [Holger Levsen](http://layer-acht.org/thinking/), Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.