_reports/2019-08.md

@@ -2,82 +2,237 @@
 layout: report
 year: "2019"
 month: "08"
-month_name: "August"
-title: "Reproducible builds in August 2019"
+title: "Reproducible Builds in August 2019"
 draft: true
 ---
 
+[![]({{ "/images/reports/2019-08/reproducible-builds.png#right" | prepend: site.baseurl }})](https://reproducible-builds.org/)
+
+**Welcome to the August 2019 report from the [Reproducible Builds](https://reproducible-builds.org) project!**
+{: .lead}
+
+In these monthly reports we outline the most important things that have happened in the world of Reproducible Builds and we have been up to.
+
+As a quick recap of our project, whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed to end users or systems as precompiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during these compilation processes by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
+
+In this month's report, we will cover:
+
+* **Media coverage & events** — *Webmin, CCCamp, etc.*
+* **Distribution work** — *The first fully-reproducible package sets, openSUSE update, etc*
+* **Upstream news** — *libfaketime updates and ensuring good definitions, etc.*
+* **Software development** — *More work on diffoscope and new variations in our testing framework, etc.*
+* **Misc news** — *From our mailing list, etc.*
+* **Getting in touch** — *How to contribute, etc*
+
+If you are interested in contributing to our project, please visit our [*Contribute*]({{ "/contribute/" | prepend: site.baseurl }}) page on our website.
+
+---
+
+## Media coverage & events
+
+[![]({{ "/images/reports/2019-08/webmin.png#right" | prepend: site.baseurl }})](http://www.webmin.com/)
+
+A backdoor was found in [Webmin](http://www.webmin.com/exploit.html), the web-based application used by sysadmins to remotely manage Unix-based systems. Whilst more details can be found on [upstream's dedicated exploit page](http://www.webmin.com/exploit.html) it appears that the build toolchain was compromised. Note especially that the exploit "did not show up in any Git diffs" and thus would not have been found via an audit of the source code. The backdoor would allow a remote attacker to execute arbitrary commands with superuser privileges on the machine running Webmin. Once a machine is compromised, an attacker could then use it to launch attacks on other systems managed through Webmin or indeed any other connected system. Techniques such as reproducible builds can help detect exactly these kinds of attacks that can lay dormant for years. ([LWN comments](https://lwn.net/Articles/796951/))
+
+In a talk titled [*There and Back Again, Reproducibly!*](https://cfp.linuxdev-br.net/2019/talk/VH9CCY/), Holger Levsen and Vagrant Cascadian presented at the 2009 edition of the [Linux Developer Conference](https://linuxdev-br.net/) in São Paulo, Brazil on Reproducible Builds.
+
+[LWN](https://lwn.net) posted and hosted an an interesting summary and discussion on [*Hardening the `file` utility for Debian*](https://lwn.net/Articles/796108). In July, Chris Lamb had cross-posted his reply to the "[Re: file(1) now with seccomp support enabled](https://lists.reproducible-builds.org/pipermail/rb-general/2019-July/001612.html) thread that was [originally started on the `debian-devel`](https://lists.debian.org/debian-devel/2019/07/msg00391.html) mailing list - in this post, Chris refers to our `strip-nondeterminism` tool not being able to accommodate the additional security hardening in [`file(1)`](http://darwinsys.com/file/) and the changes made to the tool in order to do fix this issue which was causing a huge number of regressions in [our testing framework](http://tests.reproducible-builds.org/).
+
+[![]({{ "/images/reports/2019-08/cccamp.png#right" | prepend: site.baseurl }})](https://events.ccc.de/camp/2019/)
+
+The Chaos Communication Camp — an international, five-day open-air event for hackers that provides a relaxed atmosphere for free exchange of technical, social, and political ideas — [hosted its 2019 edition](https://events.ccc.de/camp/2019/) where there were many discussions and meet-ups at least partly related to Reproducible Builds. This including the titular [Reproducible Builds Meetup](https://events.ccc.de/camp/2019/wiki/Session:Reproducible_Builds_Meetup) session which was attended by around twenty-five people where half of them were fresh to the project as well as [a session dedicated to all ArchLinux related issues](https://events.ccc.de/camp/2019/wiki/Session:Arch_Linux_Meetup).
+
+---
+
+### Distribution work
+
+[![]({{ "/images/reports/2019-08/debian.png#right" | prepend: site.baseurl }})](https://debian.org/)
+
+In Debian, the first "package sets" — ie. defined subsets of the entire archive — have become 100% reproducible, including as the so-called "essential" set for the [bullseye distribution on the `amd64`](https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_essential.html) and the [`armhf`](https://tests.reproducible-builds.org/debian/bullseye/armhf/pkg_set_essential.html) architectures, thanks to work by Chris Lamb on [`bash`](https://bugs.debian.org/935127), [`readline`](https://bugs.debian.org/935363) and other low-level libraries and tools. Perl still has issues on [`i386`](https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/i386/diffoscope-results/perl.html) and [`arm64`](https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/arm64/diffoscope-results/perl.html), however.
+
+Dmitry Shachnev [filed a bug report](https://bugs.debian.org/934405) against the `debhelper` utility that speaks to issues around using the date from the `debian/changelog` file as the source for the [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/) environment variable as this can lead to non-intuitive results when package is automatically rebuilt via so-called binary NMUs (NB. not ["source" NMUs](https://wiki.debian.org/NonMaintainerUpload)). A related issue was [later filed against qtbase5-dev](https://bugs.debian.org/934511) by Helmut Grohne as this exact issue led to an issue with co-installability across architectures.
+
+Lastly, 115 reviews of Debian packages were added, 45 were updated and 244 were removed this month, appreciably adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). Many issue types were updated by Chris Lamb, including [`embeds_build_data_via_node_preamble`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/5d91c741), [`embeds_build_data_via_node_rollup`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/e6b686f3), [`captures_build_path_in_beam_cma_cmt_files`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/850df406), [`captures_varying_number_of_build_path_directory_components`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/c0c72250), [`timezone_specific_files_due_to_haskell_devscripts`](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/a1a65bba), etc.
+
+[![]({{ "/images/reports/2019-08/opensuse.png#right" | prepend: site.baseurl }})](https://www.opensuse.org/)
+
+Bernhard M. Wiedemann posted his [monthly Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2019-08/msg00186.html) for the [openSUSE](https://opensuse.org/) distribution. New issues were found from enabling [Link Time Optimization](https://gcc.gnu.org/wiki/LinkTimeOptimization) (LTO) in this distribution's "[Tumbleweed](https://software.opensuse.org/distributions/tumbleweed)" branch. This affected, for example, [nvme-cli](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91307) as well as [`perl-XML-Parser` and `pcc`](https://bugzilla.opensuse.org/show_bug.cgi?id=1146634) with packaging issues.
+
+---
+
+## Upstream news
+
+* [`libfaketime`](https://github.com/wolfcw/libfaketime) is a tool to trick programs into believing that the current system time is actually one specified by the user. This month, Bernhard M. Wiedemann requested [the ability to track and intercept calls that change file timestamps](https://github.com/wolfcw/libfaketime/issues/183) which can help better debug or fix reproducibility issues in software.
+
+* Chris Lamb requested that the [molior build tool](https://github.com/molior-dbs/molior) prefers to [use the term "repeatable build"](https://github.com/molior-dbs/molior/issues/3) in order to avoid confusion over the term "reproducible."
+
+* There was more progress on ensuring that the [`gem` tool in rubygems respects](https://github.com/rubygems/rubygems/issues/2290#issuecomment-522206365) the [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/) environment variable.
+
+[![]({{ "/images/reports/2019-08/openwrt.png#right" | prepend: site.baseurl }})](https://openwrt.org/)
+
+* A [request to include `.buildinfo` files](https://github.com/openwrt/openwrt/pull/2121) in the [OpenWRT](https://openwrt.org/) operating system that targets embedded devices such as routes, etc. was accepted and merged upstream.
+
+---
+
 ## Software development
 
 #### Upstream patches
 
-The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible.
-We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
+The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
 
 * Bernhard M. Wiedemann:
-    * [fwupd](https://bugzilla.opensuse.org/show_bug.cgi?id=1143905) (report hash over unreproducible LTO data)
-    * [kernel-vanilla](https://lists.opensuse.org/opensuse-kernel/2019-08/msg00000.html) (drop number of CPUs)
-    * [kernel-obs-build](https://lists.opensuse.org/opensuse-kernel/2019-08/msg00001.html) (date from /etc/shadow)
-    * [dracut](https://github.com/dracutdevs/dracut/issues/617) (report CPU influencing build result)
-    * [katacontainers-image-initrd/osbuilder](https://github.com/kata-containers/osbuilder/pull/340) (merged ; shell date - new variant with nanoseconds)
     * [buildad](https://github.com/containers/buildah/pull/1805) (date)
-    * [nethack](https://build.opensuse.org/request/show/722212) (date (not submitted upstream); tar)
-    * [python-python3-saml](https://github.com/onelogin/python3-saml/pull/156) (fix FTBFS-2020 - more issues further in the future)
+    * [dracut](https://github.com/dracutdevs/dracut/issues/617) (CPU influences build result)
+    * [fwupd](https://bugzilla.opensuse.org/show_bug.cgi?id=1143905) (unreproducible [LTO](https://gcc.gnu.org/wiki/LinkTimeOptimization) data)
     * [gnutls](https://gitlab.com/gnutls/gnutls/merge_requests/1058) (date / copyright year)
-    * [pcc](https://bugzilla.opensuse.org/show_bug.cgi?id=1146634) (report unreproducibility when building with Link Time Optimization)
-    * [libfaketime](https://github.com/wolfcw/libfaketime/issues/183) (toolchain: fix various builds under libfaketime)
-    * [python-pytest-httpserver](https://github.com/csernazs/pytest-httpserver/pull/22) (renew SSL certs to fix FTBFS after 2019-09-03)
-    * [python-ipyparallel](https://github.com/ipython/ipyparallel/issues/380) (report FTFBS-j1)
-    * [sblim-cmpi-base](https://build.opensuse.org/request/show/726294) (Disable parallel make because of broken build system deps)
+    * [katacontainers-image-initrd/osbuilder](https://github.com/kata-containers/osbuilder/pull/340) (shell date; new variant with nanoseconds)
+    * [kernel-obs-build](https://lists.opensuse.org/opensuse-kernel/2019-08/msg00001.html) (date from `/etc/shadow`)
+    * [kernel-vanilla](https://lists.opensuse.org/opensuse-kernel/2019-08/msg00000.html) (drop number of CPUs)
+    * [libfaketime](https://github.com/wolfcw/libfaketime/issues/183) (toolchain: fix various builds under [`libfaketime`](https://github.com/wolfcw/libfaketime))
+    * [nethack](https://build.opensuse.org/request/show/722212) (date and [`tar(1)`](https://en.wikipedia.org/wiki/Tar_(computing)))
+    * [pcc](https://bugzilla.opensuse.org/show_bug.cgi?id=1146634) (unreproducible when building with [LTO](https://gcc.gnu.org/wiki/LinkTimeOptimization))
+    * [python-ipyparallel](https://github.com/ipython/ipyparallel/issues/380) (Fails to build with a single CPU / `-j1`)
+    * [python-pytest-httpserver](https://github.com/csernazs/pytest-httpserver/pull/22) (renew SSL certs to fix FTBFS after September 2019)
+    * [python-python3-saml](https://github.com/onelogin/python3-saml/pull/156) (Fails to build in 2020)
+    * [sblim-cmpi-base](https://build.opensuse.org/request/show/726294) (Disable parallel [`make`](https://en.wikipedia.org/wiki/Make_(software)) due to broken build dependencies)
 * Chris Lamb:
-    * [desktop-file-utils](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872728) (sort file list)
-    * [libchamplain](https://gitlab.gnome.org/GNOME/libchamplain/merge_requests/9) (path)
-    * [re2c](https://github.com/skvadrik/re2c/pull/258) (shell date)
+    * [#872728](https://bugs.debian.org/872728) filed against [`desktop-file-utils`](https://tracker.debian.org/pkg/desktop-file-utils) (closed)
+    * [#933783](https://bugs.debian.org/933783) filed against [`virulencefinder`](https://tracker.debian.org/pkg/virulencefinder).
+    * [#933790](https://bugs.debian.org/933790) filed against [`norsnet`](https://tracker.debian.org/pkg/norsnet).
+    * [#933834](https://bugs.debian.org/933834) filed against [`haskell-devscripts`](https://tracker.debian.org/pkg/haskell-devscripts).
+    * [#933838](https://bugs.debian.org/933838) filed against [`superlu-dist`](https://tracker.debian.org/pkg/superlu-dist).
+    * [#934120](https://bugs.debian.org/934120) filed against [`python-bleach`](https://tracker.debian.org/pkg/python-bleach).
+    * [#934697](https://bugs.debian.org/934697) filed against [`re2c`](https://tracker.debian.org/pkg/re2c) ([filed upstream](https://github.com/skvadrik/re2c/pull/258)).
+    * [#934698](https://bugs.debian.org/934698) filed against [`libchamplain`](https://tracker.debian.org/pkg/libchamplain) ([filed upstream](https://gitlab.gnome.org/GNOME/libchamplain/merge_requests/9))
+    * [#934699](https://bugs.debian.org/934699) filed against [`scons`](https://tracker.debian.org/pkg/scons).
+    * [#934767](https://bugs.debian.org/934767) filed against [`ecbuild`](https://tracker.debian.org/pkg/ecbuild).
+    * [#934918](https://bugs.debian.org/934918) filed against [`python-etcd3gw`](https://tracker.debian.org/pkg/python-etcd3gw).
+    * [#934919](https://bugs.debian.org/934919) filed against [`omnidb`](https://tracker.debian.org/pkg/omnidb).
+    * [#935127](https://bugs.debian.org/935127) filed against [`bash`](https://tracker.debian.org/pkg/bash).
+    * [#935361](https://bugs.debian.org/935361) filed against [`node-autoprefixer`](https://tracker.debian.org/pkg/node-autoprefixer).
+    * [#935362](https://bugs.debian.org/935362) filed against [`gdbm`](https://tracker.debian.org/pkg/gdbm).
+    * [#935363](https://bugs.debian.org/935363) filed against [`readline`](https://tracker.debian.org/pkg/readline).
+    * [#935790](https://bugs.debian.org/935790) filed against [`node-package-preamble`](https://tracker.debian.org/pkg/node-package-preamble).
+    * [#935846](https://bugs.debian.org/935846) filed against [`musescore-snapshot`](https://tracker.debian.org/pkg/musescore-snapshot).
+    * [#936452](https://bugs.debian.org/936452) filed against [`ust-fs-extra`](https://tracker.debian.org/pkg/rust-fs-extra).
+    * [#936453](https://bugs.debian.org/936453) filed against [`litl`](https://tracker.debian.org/pkg/litl).
 * Mathieu Parent:
-    * [php-pear](https://github.com/pear/pear-core/pull/96) (date) Fixes more than 165 FTBR
+    * [php-pear](https://github.com/pear/pear-core/pull/96) — Fixes over 150 packages with date issues.
 
 #### diffoscope
 
-* Vagrant Cascadian updated to diffoscope [120](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c91364d36cf6c8fc4c696d151eb9fca7832cf898), [121](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8c1379ba404b4db2f0afcf431a4ff720b72a7a19) and [122](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b126f41b301a5ac13835bf20026ae6d1d5ae2bee) in GNU Guix.
+[![]({{ "/images/reports/2019-08/diffoscope.svg#right" | prepend: site.baseurl }})](https://diffoscope.org)
+
+[diffoscope](https://diffoscope.org) is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. It is run countless times a day on [our testing infrastructure](https://tests.reproducible-builds.org/debian/reproducible.html) and is essential for identifying fixes and causes of non-deterministic behaviour.
+
+This month, Chris Lamb made the following changes:
+
+* Improvements:
+    * Don't fallback to an unhelpful raw hexdump when, for example, `readelf(1)` reports an minor issue in a section in an ELF binary. For example, when the `.frames` section is of the `NOBITS` type its contents are apparently "unreliable" and thus `readelf(1)` returns 1. ([#58](https://salsa.debian.org/reproducible-builds/diffoscope/issues/58), [#931962](https://bugs.debian.org/931962))
+    * Include either standard error or standard output (not just the latter) when an external command fails. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/4689755)]
+* Bug fixes:
+    * Skip calls to `unsquashfs` when we are neither root nor running under `fakeroot`. ([#63](https://salsa.debian.org/reproducible-builds/diffoscope/issues/63))
+    * Ensure that all of our artificially-created [`subprocess.CalledProcessError`](https://docs.python.org/3/library/subprocess.html) instances have `output` instances that are `bytes` objects, not `str`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/eb02809)]
+    * Correct a reference to `parser.diff`; `diff` in this context is a Python function in the module. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/8eb9e39)]
+    * Avoid a possible traceback caused by a `str`/`bytes` type confusion when handling the output of failing external commands. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/b803d43)]
+* Testsuite improvements:
+
+    * Test for `4.4` in the output of `squashfs -version`, even though the Debian package version is `1:4.3+git190823-1`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/7cecd8a)]
+    * Apply a patch from László Böszörményi to update the `squashfs` test output and additionally bump the required version for the test itself. ([#62](https://salsa.debian.org/reproducible-builds/diffoscope/issues/62) & [#935684](https://bugs.debian.org/935684))
+    * Add the `wabt` Debian package to the test-dependencies so that we run the [WebAssembly](https://webassembly.org/) tests on our continuous integration platform, etc. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/84ad96d)]
+* Improve debugging:
+    * Add the containing module name to the (eg.) `Using StaticLibFile for ...` debugging messages. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/2f101b8)]
+    * Strip off trailing "`original size modulo 2^32 671`" (etc.) from `gzip` compressed data as this is just a symptom of the contents itself changing that will be reflected elsewhere. ([#61](https://salsa.debian.org/reproducible-builds/diffoscope/issues/61))
+    * Avoid a lack of space between "`... with return code 1`" and "`Standard output`". [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/ffa22f8)]
+    * Improve debugging output when instantantiating our `Comparator` object types. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/1647da8)]
+    * Add a literal "eg." to the comment on stripping "`original size modulo...`" text to emphasise that the actual numbers are not fixed. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/18e3526)]
+* Internal code improvements:
+    * No need to parse the section group from the class name; we can pass it via `type` built-in `kwargs` argument. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/5261096)]
+    * Add support to `Difference.from_command_exc` and friends to ignore specific returncodes from the called program and treat them as "no" difference. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/d3c7ac8)]
+    * Simplify parsing of optional `command_args` argument to `Difference.from_command_exc`. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/cc9a730)]
+    * Set `long_description_content_type` to `text/x-rst` to appease the [PyPI.org](https://pypi.org/) linter. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/7583af2)]
+    * Reposition a comment regarding an exception within the indented block to match Python code convention. [[...](https://salsa.debian.org/reproducible-builds/diffoscope.git/commit/ec86443)]
+
+In addition, Mattia Rizzolo made the following changes:
+
+* Now that we install wabt, expect its tools to be available. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/f2e72a8)]
+* Bump the Debian backport check. [[...](https://salsa.debian.org/reproducible-builds/diffoscope/commit/9591cfb)]
+
+Lasty, Vagrant Cascadian updated diffoscope to versions [120](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c91364d36cf6c8fc4c696d151eb9fca7832cf898), [121](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8c1379ba404b4db2f0afcf431a4ff720b72a7a19) and [122](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b126f41b301a5ac13835bf20026ae6d1d5ae2bee) in the [GNU Guix](https://guix.gnu.org/) distribution.
+
+#### strip-nondeterminism
 
-#### libfaketime
-A [fix to libfaketime](https://github.com/wolfcw/libfaketime/issues/183) can now help better debug or fix reproducibility issues.
+[strip-nondeterminism](https://tracker.debian.org/pkg/strip-nondeterminism) is our tool to remove specific non-deterministic results from a completed build. This month, Chris Lamb made the following changes.
 
-## FIXME
+* Add support for enabling and disabling specific normalizers via the command line. ([#10](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/issues/10))
+* Drop accidentally-committed warning emitted on every fixture-based test. [[...](https://salsa.debian.org/reproducible-builds/strip-nondeterminism.git/commit/e1def58)]
+* Reintroduce the `.ar` normalizer [[...](https://salsa.debian.org/reproducible-builds/strip-nondeterminism.git/commit/bb13f8b)] but disable it by default so that it can be enabled with `--normalizers=+ar` or similar. ([#3](https://salsa.debian.org/reproducible-builds/strip-nondeterminism#3))
+* In verbose mode, print the normalizers that `strip-nondeterminism` will apply. [[...](https://salsa.debian.org/reproducible-builds/strip-nondeterminism.git/commit/2637e1c)]
 
-* [FIXME](https://github.com/molior-dbs/molior/issues/3) - molior: use "repeatable build" instead of "reproducible build"
+In addition, there was some movement on an issue in the [`Archive::Zip` Perl module](https://metacpan.org/pod/Archive::Zip) that `strip-nondeterminism` uses regarding the lack of support for [`bzip` compression](https://en.wikipedia.org/wiki/Bzip2) that [was originally filed in 2016](https://github.com/redhotpenguin/perl-Archive-Zip/issues/26) by [Andrew Ayer](https://www.agwa.name/).
 
-* The first Debian package sets have become 100% reproducible: [Debian essential package set for bullseye/amd64](https://tests.reproducible-builds.org/debian/bullseye/amd64/pkg_set_essential.html) and [armhf](https://tests.reproducible-builds.org/debian/bullseye/armhf/pkg_set_essential.html). perl still has isses on [i386](https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/i386/diffoscope-results/perl.html) and [arm64](https://tests.reproducible-builds.org/debian/rb-pkg/bullseye/arm64/diffoscope-results/perl.html)
+#### Test framework
 
-Bernhard M. Wiedemann posted his [monthly Reproducible Builds status update](https://lists.opensuse.org/opensuse-factory/2019-08/msg00186.html) for the [openSUSE](https://opensuse.org/) distribution where new issues were found from enabling [Link Time Optimization](https://gcc.gnu.org/wiki/LinkTimeOptimization) (LTO) in this distribution's "[Tumbleweed](https://software.opensuse.org/distributions/tumbleweed)" branch. This affected, for example, [nvme-cli](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91307) as well as [perl-XML-Parser and pcc](https://bugzilla.opensuse.org/show_bug.cgi?id=1146634) with packaging issues.
+[![]({{ "/images/reports/2019-08/testframework.png#right" | prepend: site.baseurl }})](https://tests.reproducible-builds.org/)
 
-* [FIXME](https://bugs.debian.org/934511)
+We operate a comprehensive [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org).
 
-* [FIXME](https://bugs.debian.org/934405) - a report on how Debian binary non-maintainer uploads influence `SOURCE_DATE_EPOCH`
+This month Vagrant Cascadian [suggested and subsequently implemented](https://salsa.debian.org/qa/jenkins.debian.net/commit/94469490) that we additionally test a varying build directory of different string lengths (eg. `/path/to/123` vs `/path/to/123456` but we also vary the number of directory *components* within this, eg. `/path/to/dir` vs. `/path/to/parent/subdir`. Curiously, whilst it was *a prior* believed that was rather unlikely to yield differences, Chris Lamb [has managed to identify approximately twenty packages](https://tests.reproducible-builds.org/debian/issues/unstable/captures_varying_number_of_build_path_directory_components_issue.html) that are affected by this issue.
 
-* [FIXME](https://github.com/redhotpenguin/perl-Archive-Zip/issues/26#issuecomment-521408412) - perl-Archive-Zip still has broken bzip handling - [affects strip-nondeterminism](https://salsa.debian.org/reproducible-builds/strip-nondeterminism/issues/1)
+[![]({{ "/images/reports/2019-08/coreboot.png#right" | prepend: site.baseurl }})](https://www.coreboot.org/)
 
-* [Hardening the "file" utility for Debian -- nb. paywalled for now but will not be by report publication date](https://lwn.net/Articles/796108)
+It was also noticed that [our testing of the Coreboot free software firmware](https://tests.reproducible-builds.org/coreboot/coreboot.html) fails to build the toolchain since we switched to building on the Debian `buster` distribution. The [last successful build was on August 7th](https://jenkins.debian.net/job/reproducible_coreboot/356 was t) but all newer builds have failed.
 
-* [FIXME](https://github.com/skvadrik/re2c/pull/258)
+In addition, the following code changes were performed in the last month:
 
-* [FIXME](https://github.com/rubygems/rubygems/issues/2290#issuecomment-522206365)
+* Chris Lamb: Ensure that the size the log for the second build in HTML pages was also correctly formatted (eg. "`12KB`" vs "`12345`"). [[...](https://salsa.debian.org/reproducible-builds/jenkins.debian.net.git/commit/080d7ba3)]
 
-* [FIXME](https://tests.reproducible-builds.org/debian/issues/unstable/captures_varying_number_of_build_path_directory_components_issue.html)
+* Holger Levsen:
 
-* [FIXME](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=935127)
+    * Many changes related to updating our build nodes to the `buster` distribution for Debian. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/a97c97ec)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/6fb3ee7b)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/1fad75e4)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7fef98af)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/da55be7a)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/28941bc2)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/309a1d66)]
+    * Attempt to automatically fixup spurious build failures. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/82bee189)]
+    * Update the maintainer address for [the Debian team tasked with maintaining](https://wiki.debian.org/Teams/pkg-mate) the [MATE desktop](https://mate-desktop.org/). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/e6f7c6d4)]
+    * Try not to build all the release tags of tools such as [diffoscope](https://diffoscope.org), etc.. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/974b699e)]
+    * Use a newer kernel to support building the latest [Arch Linux](https://www.archlinux.org/) packages. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7e575590)]
+    * Re-add checks for "zombie" and log file size sanity checks. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/f17552ad)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/30049d46)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/9fbf8d2c)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/42d7c71a)]
+    * Vary the choice of kernel on the `amd64` again by using the kernel from [Debian "backports"](https://backports.debian.org/). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/b2870778)]
+    * Drop some ancient Debian `jessie`-related configuration. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/96cbb81e)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7e37c5a4)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/87840dae)]
 
-* [FIXME](https://www.zdnet.com/article/backdoor-found-in-webmin-a-popular-web-based-utility-for-managing-unix-servers/) + [webmin upstream notice](http://www.webmin.com/exploit.html) / https://lwn.net/Articles/796951/
+* Mathieu Parent:
+    * Update the contact details for the [Debian PHP Group](https://wiki.debian.org/Teams/DebianPHPGroup). [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/03510cdf)]
+
+* Mattia Rizzolo:
+    * Update our [Postfix](http://www.postfix.org/) email server configuration. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/61ceaf5d)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/8780a849)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/3b964081)]
+    * Use the `safe_load` function of [PyYAML](https://pyyaml.org/wiki/PyYAMLDocumentation) when parsing [YAML-formatted](https://en.wikipedia.org/wiki/YAML) files. [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/fa720775)]
+
+The usual node maintenance was performed by Holger Levsen [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/961d70a6)][[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/25f295b7)] and Vagrant Cascadian [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/30f567ff)].
+
+---
+
+## Misc news
 
-* [FIXME](https://github.com/redhotpenguin/perl-Archive-Zip/issues/26#issuecomment-524039184)
+[![]({{ "/images/reports/2019-08/website.png#right" | prepend: site.baseurl }})](https://reproducible-builds.org/)
 
-* FIXME Vagrant Cascadian and Holger Levsen presented "There and Back Again, Reproducibly!" at linuxdev-br.net.
+There was a yet more effort put into our [our website](https://reproducible-builds.org/) this month, including misc copyediting by Chris Lamb [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/a911e9d)], Mathieu Parent referencing his fix for `php-pear` [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/e47ade1)] and Vagrant Cascadian updating a link to his homepage. [[...](https://salsa.debian.org/reproducible-builds/reproducible-website/commit/7f8bc7e)].
 
-* tests.r-b.o/coreboot fails to build the toolchain since we switched to building on buster, https://jenkins.debian.net/job/reproducible_coreboot/356 was the last successful build, all newer builds fail. Thus https://tests.reproducible-builds.org/coreboot/coreboot.html was last updated on August 7th.
+On [our mailing list](https://lists.reproducible-builds.org/pipermail/rb-general/) this month Santiago Torres Arias started a [*Setting up a MS-hosted rebuilder with in-toto metadata*](https://lists.reproducible-builds.org/pipermail/rb-general/2019-August/001640.html)" thread regarding Microsoft's interest in setting up a rebuilder for Debian packages touching on issues of transparency logs and the integration of [in-toto](https://in-toto.io/) by the [Secure Systems Lab](https://ssl.engineering.nyu.edu/) at [New York University](https://engineering.nyu.edu/). In addition, [Lars Wirzenius](https://liw.fi/) continued conversation regarding [various questions about reproducible builds](https://lists.reproducible-builds.org/pipermail/rb-general/2019-August/001634.html) and their bearing on building a distributed continuous integration system.
 
-* The fix for https://github.com/openwrt/openwrt/pull/2121 has been cherry picked into the git master branch on August 13th as https://github.com/openwrt/openwrt/commit/454021581f630d5d04afeb8ff6581c1bda295c87 - this adds feeds.buildinfo and version.buildinfo files as output into the build directory.
+Lastly, in a thread titled [*Reproducible Builds technical introduction tutorial*](https://lists.reproducible-builds.org/pipermail/rb-general/2019-August/001639.html) Jathan asked whether anyone had some "easy" Reproducible Builds tutorials in slides, video or written document format.
+
+---
 
-* At [CCCamp 2019](https://events.ccc.de/camp/2019) there were once again several discussions and meet-ups at least partly devoted to reproducible builds:
-  * [FIXME](https://events.ccc.de/camp/2019/wiki/Session:Reproducible_Builds_Meetup) was attended by around 25 people, half of them were new to the project.
-  * [FIXME](https://events.ccc.de/camp/2019/wiki/Session:Arch_Linux_Meetup)
+## Getting in touch
+
+If you are interested in contributing the Reproducible Builds project, please visit our [*Contribute*](https://reproducible-builds.org/contribute/) page on our website. However, you can get in touch with us via:
+
+ * IRC: `#reproducible-builds` on `irc.oftc.net`.
+
+ * Twitter: [@ReproBuilds](https://twitter.com/ReproBuilds)
+
+ * Mailing list: [`rb-general@lists.reproducible-builds.org`](https://lists.reproducible-builds.org/listinfo/rb-general)
+
+<br>
+
+---
 
+This month's report was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Mathieu Parent and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

bin/generate-draft.template

@@ -6,11 +6,16 @@ title: "Reproducible Builds in {{ month_year }}"
 draft: true
 ---
 
-**Welcome to the {{ month_year }} report from the [Reproducible Builds](https://reproducible-builds.org) project!** In our reports we outline the most important things that we have been up over the past month.
+**Welcome to the {{ month_year }} report from the [Reproducible Builds](https://reproducible-builds.org) project!**
+{: .lead}
 
-As a quick recap, whilst anyone can inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
+[![]({{ "/images/reports/{{ title_year }}-{{ title_month }}/reproducible-builds.png#right" | prepend: site.baseurl }})](https://reproducible-builds.org/)
 
-In this month's report, we will cover:
+In these reports we outline the most important things that we have been up over the past month. As a quick recap, whilst anyone can inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries.
+
+The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
+
+In this month's report, we cover:
 
 * **Media coverage** — *FIXME, etc.*
 * **Upstream news** — *FIXME, etc.*
@@ -19,7 +24,9 @@ In this month's report, we will cover:
 * **Misc news** — *From our mailing list, etc.*
 * **Getting in touch** — *How to contribute, etc*
 
+{% raw %}
 If you are interested in contributing to our project, please visit our [*Contribute*]({{ "/contribute/" | prepend: site.baseurl }}) page on our website.
+{% endraw %}
 
 ---
 
@@ -37,6 +44,10 @@ If you are interested in contributing to our project, please visit our [*Contrib
 
 ### Distribution work
 
+[![]({{ "/images/reports/{{ title_year }}-{{ title_month }}/opensuse.png#right" | prepend: site.baseurl }})](https://www.opensuse.org/)
+
+[![]({{ "/images/reports/{{ title_year }}-{{ title_month }}/debian.png#center" | prepend: site.baseurl }})](https://debian.org/)
+
 In Debian, ...
 
 * {{ packages_stats['added'] }} reviews of Debian packages were added, {{ packages_stats['updated'] }} were updated and {{ packages_stats['removed'] }} were removed this month, adding to [our knowledge about identified issues](https://tests.reproducible-builds.org/debian/index_issues.html). FIXME issue types have been updated: {% for _, xs in issues_yml.items()|sort %}{% for x in xs %}[{{ x['title'] }}](https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/{{ x['sha'] }}), {% endfor %}{% endfor %}

images/reports/2019-08/diffoscope.svg

+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:xlink="http://www.w3.org/1999/xlink"
+   version="1.1"
+   width="128"
+   height="128"
+   id="svg2">
+  <defs
+     id="defs4" />
+  <metadata
+     id="metadata7">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title></dc:title>
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <g
+     transform="matrix(1.0692573,0,0,1.0692573,-328.34726,-503.5515)"
+     id="layer1">
+    <g
+       id="g5409">
+      <g
+         transform="translate(5.418238,0)"
+         id="g5386">
+        <rect
+           width="90.304001"
+           height="50.999996"
+           x="316.36414"
+           y="472.80621"
+           id="rect4667-3"
+           style="fill:none;stroke:none" />
+        <g
+           id="text4673-8"
+           style="font-size:64px;font-style:normal;font-variant:normal;font-weight:500;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#008000;fill-opacity:1;stroke:none;font-family:Inconsolata;-inkscape-font-specification:Inconsolata Medium">
+          <path
+             d="m 316.36413,483.82622 0,3.968 26.304,0 0,-3.968"
+             id="path5371"
+             style="fill:#c00000;fill-opacity:1" />
+          <path
+             d="m 348.36413,483.82622 0,3.968 26.304,0 0,-3.968"
+             id="path5373"
+             style="fill:#c00000;fill-opacity:1" />
+          <path
+             d="m 380.36413,483.82622 0,3.968 26.304,0 0,-3.968"
+             id="path5375"
+             style="fill:#c00000;fill-opacity:1" />
+        </g>
+        <g
+           id="text5366"
+           style="font-size:64px;font-style:normal;font-variant:normal;font-weight:500;font-stretch:normal;text-align:start;line-height:125%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#008000;fill-opacity:1;stroke:none;font-family:Inconsolata;-inkscape-font-specification:Inconsolata Medium">
+          <path
+             d="m 327.69213,496.99019 0,10.88 -11.328,0 0,3.968 11.328,0 0,11.968 4.032,0 0,-11.968 10.944,0 0,-3.968 -10.944,0 0,-10.88 -4.032,0"
+             id="path5378" />
+          <path
+             d="m 359.69213,496.99019 0,10.88 -11.328,0 0,3.968 11.328,0 0,11.968 4.032,0 0,-11.968 10.944,0 0,-3.968 -10.944,0 0,-10.88 -4.032,0"
+             id="path5380" />
+          <path
+             d="m 391.69213,496.99019 0,10.88 -11.328,0 0,3.968 11.328,0 0,11.968 4.032,0 0,-11.968 10.944,0 0,-3.968 -10.944,0 0,-10.88 -4.032,0"
+             id="path5382" />
+        </g>
+      </g>
+      <use
+         id="use5399"
+         x="0"
+         y="0"
+         width="744.09448"
+         height="1052.3622"
+         xlink:href="#g5386" />
+      <use
+         transform="matrix(0.8,0,0,0.8,82.417275,133.65028)"
+         id="use5401"
+         style="opacity:0.85"
+         x="0"
+         y="0"
+         width="744.09448"
+         height="1052.3622"
+         xlink:href="#g5386" />
+      <use
+         transform="matrix(0.6,0,0,0.6,164.83455,260.05454)"
+         id="use5403"
+         style="opacity:0.7"
+         x="0"
+         y="0"
+         width="744.09448"
+         height="1052.3622"
+         xlink:href="#g5386" />
+      <use
+         transform="matrix(0.4,0,0,0.4,247.25182,379.25208)"
+         id="use5405"
+         style="opacity:0.55"
+         x="0"
+         y="0"
+         width="744.09448"
+         height="1052.3622"
+         xlink:href="#g5386" />
+    </g>
+  </g>
+</svg>