121.md 8.66 KB
Newer Older
1
---
2
layout: blog
3
week: 121
4
published: 2017-08-25 19:08:21
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
---

Here's what happened in the [Reproducible Builds](https://reproducible-builds.org) effort between Sunday August 13 and Saturday August 19 2017:

Reproducible Builds finally mandated by Debian Policy
-----------------------------------------------------

"Packages should build reproducibly" [was merged into Debian
policy](https://anonscm.debian.org/git/dbnpolicy/policy.git/commit/?id=bf256860fbf9d7dccc05fe1aa85841b7a1b1d712)! The added text is as follows and
has been [included into debian-policy 4.1.0.0](https://tracker.debian.org/news/864773):

<pre>
Reproducibility
---------------

Packages should build reproducibly, which for the purposes of this
document [#]_ means that given

- a version of a source package unpacked at a given path;
- a set of versions of installed build dependencies;
- a set of environment variable values;
- a build architecture; and
- a host architecture,

repeatedly building the source package for the build architecture on
any machine of the host architecture with those versions of the build
dependencies installed and exactly those environment variable values
set will produce bit-for-bit identical binary packages.

It is recommended that packages produce bit-for-bit identical binaries
even if most environment variables and build paths are varied.  It is
intended for this stricter standard to replace the above when it is
easier for packages to meet it.

.. [#]
   This is Debian's precisification of the `reproducible-builds.org
   definition <https://reproducible-builds.org/docs/definition/>`_.

</pre>


* Holger Levsen [wrote a blog post](https://layer-acht.org/thinking/blog/20170812-reproducible-policy/)
Georg Faerber's avatar
Georg Faerber committed
47
  briefly describing the background and implications of this. To quote him: "we are *not 94% done* yet, rather more like half done or so. We still need tools and processes to *enable anyone to independently verify* that a given binary comes from the sources it is said to be coming, this will involve distributing `.buildinfo` files and providing user interfaces in APT and elsewhere and probably also systematic rebuilds by us and other parties. And 6% or 7% of the archive is still a lot of packages, eg. in Buster we currently still have [273 unreproducible key packages](https://tests.reproducible-builds.org/debian/buster/amd64/pkg_set_key_packages.html) and for a large part we don't have patches yet so there is still a lot of work ahead."
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
* There were discussion threads on [Hacker News](https://news.ycombinator.com/item?id=15010438)
  and [Reddit](https://www.reddit.com/r/debian/comments/6touxc/new_debian_policy_packages_should_be_reproducible/).
* Our long-term goal is that Policy mandates that packages "must" be reproducible, but for that we need to show further progress and also reach a consensus on `.buildinfo` files and much more.

Reproducible work in other projects
-----------------------------------

Bernhard M. Wiedemann's [reproducibleopensuse
scripts](https://github.com/bmwiedemann/reproducibleopensuse) now
[work on Debian buster](https://github.com/openSUSE/obs-build/pull/376) on the
openSUSE Build Service with the latest versions of
[osc](https://en.opensuse.org/openSUSE:OSC) and
[obs-build](https://github.com/openSUSE/obs-build).


Toolchain development and fixes
-------------------------------

66
[#872514](https://bugs.debian.org/872514) was opened on [devscripts](https://tracker.debian.org/pkg/devscripts) by Chris Lamb to add a
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82
`reproducible-check` program to report on the reproducibility status of
installed packages.


Packages reviewed and fixed, and bugs filed
-------------------------------------------

Upstream reports:

* Bernhard M. Wiedemann:
  * [qt5/base](https://codereview.qt-project.org/202999), `SOURCE_DATE_EPOCH`
    support.

Debian reports:

* Adrian Bunk:
83 84 85
  * [#872262](https://bugs.debian.org/872262) filed against [jellyfish](https://tracker.debian.org/pkg/jellyfish).
  * [#872675](https://bugs.debian.org/872675) filed against [pdp](https://tracker.debian.org/pkg/pdp).
  * [#872678](https://bugs.debian.org/872678) filed against [gem](https://tracker.debian.org/pkg/gem).
86
* Chris Lamb:
87 88 89
  * [#872453](https://bugs.debian.org/872453) filed against [isa-support](https://tracker.debian.org/pkg/isa-support).
  * [#872459](https://bugs.debian.org/872459) filed against [python-numpy](https://tracker.debian.org/pkg/python-numpy).
  * [#872460](https://bugs.debian.org/872460) filed against [gcab](https://tracker.debian.org/pkg/gcab), forwarded
90
    [upstream](https://bugzilla.gnome.org/show_bug.cgi?id=786435).
91 92 93
  * [#872514](https://bugs.debian.org/872514) filed against [devscripts](https://tracker.debian.org/pkg/devscripts).
  * [#872728](https://bugs.debian.org/872728) filed against [desktop-file-utils](https://tracker.debian.org/pkg/desktop-file-utils).
  * [#872729](https://bugs.debian.org/872729) filed against [gtk+2.0](https://tracker.debian.org/pkg/gtk+2.0), forwarded
94 95 96
    [upstream](https://bugzilla.gnome.org/show_bug.cgi?id=786528), found via a
    [reproducibility issue in Tails](https://labs.riseup.net/code/issues/13440).
* Federico Brega:
97
  * [#872285](https://bugs.debian.org/872285) filed against [pyqt5](https://tracker.debian.org/pkg/pyqt5).
98
* Jeremy Bicha:
99
  * [#871963](https://bugs.debian.org/871963) filed against [birdfont](https://tracker.debian.org/pkg/birdfont).
100
* Philip Rinn:
101
  * [#872184](https://bugs.debian.org/872184) filed against [quilt](https://tracker.debian.org/pkg/quilt).
102 103 104 105 106


Debian non-maintainer uploads:

* Mattia Rizzolo:
107
  * [console-data](https://tracker.debian.org/pkg/console-data) (for bug [#799871](https://bugs.debian.org/799871)).
108 109 110 111 112 113 114 115 116 117


Reviews of unreproducible packages
----------------------------------

47 package reviews have been added, 58 have been updated and 39 have been removed in this week,
adding to our knowledge about [identified issues](https://tests.reproducible-builds.org/debian/index_issues.html).

4 issue types have been updated:

118 119 120 121
- Added [nondeterministic_output_generated_by_gcab toolchain](https://tests.reproducible-builds.org/issues/unstable/nondeterministic_output_generated_by_gcab toolchain_issue.html).
- Added [build_path_captured_by_python_numpy_misc_util](https://tests.reproducible-builds.org/issues/unstable/build_path_captured_by_python_numpy_misc_util_issue.html).
- Added [captures_build_path_in_sphinx_attr_links](https://tests.reproducible-builds.org/issues/unstable/captures_build_path_in_sphinx_attr_links_issue.html).
- Re-added [nondeterminstic_ordering_in_gsettings_glib_enums_xml](https://tests.reproducible-builds.org/issues/unstable/nondeterminstic_ordering_in_gsettings_glib_enums_xml_issue.html).
122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138


Weekly QA work
--------------

During our reproducibility testing, FTBFS bugs have been detected and reported by:

 - Adrian Bunk (59)
 - Bastien Roucariès (1)
 - James Clarke (1)
 - Jeremy Bicha (1)


diffoscope development
----------------------

Development [continued in
139
git](https://salsa.debian.org/reproducible-builds/diffoscope.git/log/),
140 141 142 143
including the following contributions:

- Ximin Luo:
  - presenters: html: Don't traverse children whose parents were already
144
    limited (Closes: [#871413](https://bugs.debian.org/871413))
145
  - On a non-GNU system, prefer tools that start with "g" for certain
146 147
    whitelisted commands. (Closes: [#871029](https://bugs.debian.org/871029))
  - Add a `--tool-prefix-binutils` CLI flag. (Closes: [#869868](https://bugs.debian.org/869868))
148 149 150 151 152 153 154 155 156 157 158 159
- Chris Lamb:
  - Temporarily revert "Bump Standards-Version to 4.0.1" to avoid spurious CI
    test failures.
  - comparators.xml: Use ``name`` attribute over ``path`` to avoid leaking
    comparison full path in output.
  - Code style fixes.


disorderfs development
----------------------

Development [continued in
160
git](https://salsa.debian.org/reproducible-builds/disorderfs.git/log/),
161 162 163 164 165 166 167 168 169 170
including the following contributions:

- Chris Lamb:
  - Add simple autopkgtest.


reprotest development
---------------------

Development [continued in
171
git](https://salsa.debian.org/reproducible-builds/reprotest.git/log/), including
172 173 174
the following contributions:

- Ximin Luo:
175
  - Choose an existent `HOME` for the "control" build. (Closes: [#860428](https://bugs.debian.org/860428))
176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191
  - Update `debian/changelog` with Santiago's changes.
- Santiago Torres:
  - Abstract parts of autopkgtest to support running on non-Debian systems.
  - Add a `--host-distro` flag to support that too.


tests.reproducible-builds.org
-----------------------------

Mattia fixed the script which creates the [HTML representation of our database scheme]([https://tests.reproducible-builds.org/reproducibledb.html) to not append .html twice to the filename.

Misc.
-----

This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen & reviewed by a bunch
of Reproducible Builds folks on IRC & the mailing lists.