_blog/posts/185.md

@@ -108,13 +108,12 @@ Test framework development
 --------------------------
 FIXME: please give me 15min to rework this a bit
 
-There were a large number of updates to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](tests.reproducible-builds.org) by Holger Levsen this week, including:
+There were a large number of updates to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](tests.reproducible-builds.org) by Holger Levsen this week, see below. The most important work was done behind the scenes, outside of git, which was a long debugging session to find out why jenkins java processes were suddenly eating all CPU while the machine had a load of 60-200. This involved (temporarily) removing all 1300 jobs, disabling plugins and other things, all didn't help. In the end it turned out that the underlying SSH/HDD performance was configured poorly, after this was fixed, java/jenkins immediatly ran normal.
 
 * [Debian GNU/Linux](https://www.debian.org/)-specific changes:
     * Merge patch by Simon McVittie to (https://salsa.debian.org/qa/jenkins.debian.net/commit/d04769a7)[perform build2 in merged /usr environment for >= buster] (Closes: (https://bugs.debian.org/901473)[#901473].
     * Document that we vary by installing the `usr-merge` package [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/7902f640)] and add a link to the [corresponding Debian Wiki page](https://wiki.debian.org/UsrMerge]) [[...](https://salsa.debian.org/qa/jenkins.debian.net/commit/fb44311e)].
-    * [Use `pbuilder` from the "backports" repositories everywhere](https://salsa.debian.org/qa/jenkins.debian.net/commit/2081b3a4).
-    * [Force installation of `pbuilder` from bpo on ubuntu 16.04](https://salsa.debian.org/qa/jenkins.debian.net/commit/d28a62fb)
+    * [Use `pbuilder` from the "backports" repositories everywhere](https://salsa.debian.org/qa/jenkins.debian.net/commit/2081b3a4), to achieve that also [force installation of `pbuilder` from bpo on ubuntu 16.04](https://salsa.debian.org/qa/jenkins.debian.net/commit/d28a62fb)
     * Deal with flacky armhf boards. ([1](https://salsa.debian.org/qa/jenkins.debian.net/commit/6121cd22), [2](https://salsa.debian.org/qa/jenkins.debian.net/commit/099a8de5), [3](https://salsa.debian.org/qa/jenkins.debian.net/commit/2bc5747f), [4](https://salsa.debian.org/qa/jenkins.debian.net/commit/410d530b))
     * Remove java and depends from all 49 build nodes manually. Also clean up cruft from the jessie2stretch upgrades on armhf nodes.