Compare revisions

Chris Lamb · Chris Lamb · Chris Lamb · Chris Lamb · 61f37e28 · 61f37e28
--- a/_blog/posts/165.md
+++ b/_blog/posts/165.md
@@ -34,76 +34,21 @@ Packages reviewed and fixed, and bugs filed
 tests.reproducible-builds.org development
 -----------------------------------------

-There were a number of changes to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org/), including:
-
-* Mattia Rizzolo (FIXME):
-    * [rblib.models: take the \_status namedtuple out of the Status enum](https://salsa.debian.org/qa/jenkins.debian.net/commit/f874effd)
-    * [Move from jessie-backports to stretch-backports for pbuilder in the arm64 hosts](https://salsa.debian.org/qa/jenkins.debian.net/commit/42decf0c)
-    * [Use APT pinning instead of using `apt-get install -t`.](https://salsa.debian.org/qa/jenkins.debian.net/commit/67fb58ea)
-    * [update\_jdn: stop installing lintian on our hosts, we really have no use for it](https://salsa.debian.org/qa/jenkins.debian.net/commit/6f915a55)
-    * [simplify apt's sources.list of the arm64 hosts, disable restricted and multiverse](https://salsa.debian.org/qa/jenkins.debian.net/commit/bf8f6144)
-    * [reproducible: common.sh: error out when `psql` fails](https://salsa.debian.org/qa/jenkins.debian.net/commit/3cb06dd7)
-    * [reproducible: html\_breakages: check for dbdtxt and deal with the new timeout status](https://salsa.debian.org/qa/jenkins.debian.net/commit/953c5276)
-    * [reproducible: html\_breakages: remove pbuilder\_dep\_fail(), we had the depwait state for a long time](https://salsa.debian.org/qa/jenkins.debian.net/commit/3030488f)
-    * [\_html\_dashboard: always include depwait and E404 packages in the index\_suite\_arch, even when they are 0](https://salsa.debian.org/qa/jenkins.debian.net/commit/84da4810)
-    * [\_html\_dashboard: also include timeout packages in index\_suite\_arch pages](https://salsa.debian.org/qa/jenkins.debian.net/commit/22a799aa)
-    * [\_html\_dashboard: fix NFU icon](https://salsa.debian.org/qa/jenkins.debian.net/commit/49799666)
-    * [dashboard.sh: fix typo](https://salsa.debian.org/qa/jenkins.debian.net/commit/056eac54)
-    * [reproducible: introduce a new "timeout" status to represent build timeouts](https://salsa.debian.org/qa/jenkins.debian.net/commit/4395f641)
-    * build.sh:
-        * [take the automatic artifacts notifications back to #-changes](https://salsa.debian.org/qa/jenkins.debian.net/commit/0b123b98)
-        * [fix SQL error](https://salsa.debian.org/qa/jenkins.debian.net/commit/e6af59dd)
-        * [do an UPSERT insted of manually checking for presence before UPDATE/INSERT](https://salsa.debian.org/qa/jenkins.debian.net/commit/34570a21)
-        * [reinstate saving the builds in stats\_build](https://salsa.debian.org/qa/jenkins.debian.net/commit/c8b887fe)
-        * [keep the diffoscope errors notification in #-changes for a while to see how many there are](https://salsa.debian.org/qa/jenkins.debian.net/commit/20f72bf3)
-        * [differentiate the irc message between diffoscope errors and timeouts](https://salsa.debian.org/qa/jenkins.debian.net/commit/bae726d3)
-        * [split notification handling to a different function, and have it called only when running in master mode](https://salsa.debian.org/qa/jenkins.debian.net/commit/480cf7a2)
-        * [apply the 120m timeout directly to diffoscope instead of the wrapping schroot](https://salsa.debian.org/qa/jenkins.debian.net/commit/a903dd7b)
-        * [log this in the rbuild](https://salsa.debian.org/qa/jenkins.debian.net/commit/548a9b5c)
-        * [aid debug by quoting the variables](https://salsa.debian.org/qa/jenkins.debian.net/commit/3d4e5fa8)
-        * [drop old case when DURATION could have been undefined](https://salsa.debian.org/qa/jenkins.debian.net/commit/bb4adfb6)
-        * [remove some query\_db fallbacks from the sqlite time](https://salsa.debian.org/qa/jenkins.debian.net/commit/bdaff5af)
-        * [silence the greps](https://salsa.debian.org/qa/jenkins.debian.net/commit/b5b250cd)
-        * [fix depwait case](https://salsa.debian.org/qa/jenkins.debian.net/commit/aadebb76)
-        * [drop spurious set -x](https://salsa.debian.org/qa/jenkins.debian.net/commit/3d2d5ab7)
-        * [fix ftbfs check after the last refactor](https://salsa.debian.org/qa/jenkins.debian.net/commit/bbb991a3)
-        * [drop pipe and console logging during first\_build(): those msgs are lost anyway](https://salsa.debian.org/qa/jenkins.debian.net/commit/69544673)
-        * [copy build artifacts locally before further ascertain the build status](https://salsa.debian.org/qa/jenkins.debian.net/commit/63ced24c)
-        * [also consider the case of NOTIFY=''](https://salsa.debian.org/qa/jenkins.debian.net/commit/601a37d8)
-        * [move the post-build steps into build\_rebuild() instead of the main script body](https://salsa.debian.org/qa/jenkins.debian.net/commit/e7cde572)
-        * [merge filter\_changes\_files() into call\_diffoscope\_on\_changes\_files()](https://salsa.debian.org/qa/jenkins.debian.net/commit/256e1d88)
-        * [turn list of elif into a case](https://salsa.debian.org/qa/jenkins.debian.net/commit/d012e181)
-        * [don't send IRC notification for NOTIFY=0](https://salsa.debian.org/qa/jenkins.debian.net/commit/53e059fd)
-        * [print out the duration at the end of the cleanup](https://salsa.debian.org/qa/jenkins.debian.net/commit/84dffbff)
-        * [change the way ftbfs are handled](https://salsa.debian.org/qa/jenkins.debian.net/commit/fda61fc7)
-        * [let the cleanup be implied by the trap](https://salsa.debian.org/qa/jenkins.debian.net/commit/c94aea7c)
-        * [drop handling of the "version mismatch between master and build node", that is not possible anymore](https://salsa.debian.org/qa/jenkins.debian.net/commit/0470f48c)
-    * [worker.sh: also include the timestamp in the error log](https://salsa.debian.org/qa/jenkins.debian.net/commit/4d3f3c33)
-    * [reproducible: html\_indexes: fixup: drop another status mangling that is now unneeded](https://salsa.debian.org/qa/jenkins.debian.net/commit/975ea2f7)
-    * [reproducible: \_common.sh: drop support for 'not\_for\_us' and 'not for us' status names](https://salsa.debian.org/qa/jenkins.debian.net/commit/a349fad1)
-    * [rblib: remove the now unused get\_status\_icon() helper function](https://salsa.debian.org/qa/jenkins.debian.net/commit/751d4d7b)
-    * [reproducible: \_html\_notes: use the new Status class instead of get\_status\_icon()](https://salsa.debian.org/qa/jenkins.debian.net/commit/6f71ed16)
-    * [reproducible: \_html\_notes: list packages in all status in the issues pages](https://salsa.debian.org/qa/jenkins.debian.net/commit/f6dfd246)
-    * [reproducible: maintenance: reschedule stale builds, instead of dropping them](https://salsa.debian.org/qa/jenkins.debian.net/commit/99cdf49a)
-    * [reproducible: \_html\_pkg\_sets: Use the new Status object instead of get\_status\_icon()](https://salsa.debian.org/qa/jenkins.debian.net/commit/dfffafb2)
-    * [reproducible: \_html\_live\_status: also here, use the new Status enum instead of get\_status\_icon()](https://salsa.debian.org/qa/jenkins.debian.net/commit/6a18da2e)
-    * [reproducible: \_html\_indexes: start to use the new Status enum instead of passing around plain strings](https://salsa.debian.org/qa/jenkins.debian.net/commit/92b6fa82)
-    * [rblib/models: introduce a Status enum](https://salsa.debian.org/qa/jenkins.debian.net/commit/187636a0)
-    * [reproducible: \_html\_packages: use the new Status class instead of get\_status\_icon()](https://salsa.debian.org/qa/jenkins.debian.net/commit/3d923465)
-    * [reproducible: \_html\_packages: flake8 fixes](https://salsa.debian.org/qa/jenkins.debian.net/commit/01aaa463)
-    * [reproducible: \_html\_indexes: print the fully-compiled query with all the binded parameters](https://salsa.debian.org/qa/jenkins.debian.net/commit/9ceca98a)
-    * [reproducible: \_db\_maintenance: fixup after a broken rebase](https://salsa.debian.org/qa/jenkins.debian.net/commit/9a9923e9)
-    * [reproducible: rename status '404' to 'E404' to make a future planned change easier](https://salsa.debian.org/qa/jenkins.debian.net/commit/ca3416d7)
-    * [reproducible: normalize 'unreproducible' to 'FTBR' in computer-oriented strings](https://salsa.debian.org/qa/jenkins.debian.net/commit/ccc94fcf)
-    * [reproducible: rename the inconsistent 'not for us' and 'not\_for\_us' to NFU](https://salsa.debian.org/qa/jenkins.debian.net/commit/09012516)
-    * [reproducible: \_db\_maintenance: automatically bump the value in rb\_schema without having to specify the query at each update](https://salsa.debian.org/qa/jenkins.debian.net/commit/01ac367e)
-    * [reproducible: \_db\_maintenance: run the update queries in a single transaction](https://salsa.debian.org/qa/jenkins.debian.net/commit/cb6b2f90)
-    * [reproducible: \_db\_maintenance: fix typo in function name](https://salsa.debian.org/qa/jenkins.debian.net/commit/26739d90)
-    * [rblib: get\_status\_icon(): fail when looking up an unknown status](https://salsa.debian.org/qa/jenkins.debian.net/commit/457b5456)
+There were a large number of changes to our [Jenkins](https://jenkins.io/)-based testing framework that powers [tests.reproducible-builds.org](https://tests.reproducible-builds.org/), including:

 * Chris Lamb:
    * [Correct "which build timed out" grammar](https://salsa.debian.org/qa/jenkins.debian.net/commit/30a24438).

+* Mattia Rizzolo:
+    * [Introduce a new "timeout" status to represent build timeouts](https://salsa.debian.org/qa/jenkins.debian.net/commit/4395f641), requiring multiple changes including [appling the timeout directly to diffoscope](https://salsa.debian.org/qa/jenkins.debian.net/commit/a903dd7b), adding a [new icon to the HTML dashboard](https://salsa.debian.org/qa/jenkins.debian.net/commit/49799666), [differentiating the IRC messages](https://salsa.debian.org/qa/jenkins.debian.net/commit/bae726d3), [listing packages on the issues page](https://salsa.debian.org/qa/jenkins.debian.net/commit/f6dfd246) and mulitple other fixups.
+    * Don't install [Lintian][https://lintian.debian.org] on our hosts. ([commit](https://salsa.debian.org/qa/jenkins.debian.net/commit/6f915a55))
+    * Use [APT pinning](https://wiki.debian.org/AptPreferences) instead of [using `apt-get install -t`](https://salsa.debian.org/qa/jenkins.debian.net/commit/67fb58ea).
+    * [Error out when `psql` fails](https://salsa.debian.org/qa/jenkins.debian.net/commit/3cb06dd7).
+    * Move the "automatic artifacts" notifications [back to the `#debian-devel-changes` IRC channel](https://salsa.debian.org/qa/jenkins.debian.net/commit/0b123b98).
+    * Perform [SQL `UPSERT` operation](https://wiki.postgresql.org/wiki/UPSERT) instead of [manually checking for presence before UPDATE/INSERT](https://salsa.debian.org/qa/jenkins.debian.net/commit/34570a21).
+    * [Reschedule stale builds, instead of dropping them](https://salsa.debian.org/qa/jenkins.debian.net/commit/99cdf49a).
+    * [Rename the `404` status to `E404` to make a future change easier](https://salsa.debian.org/qa/jenkins.debian.net/commit/ca3416d7).
+

 Misc.
 -----

--- a/bin/generate-draft
+++ b/bin/generate-draft
@@ -43,7 +43,6 @@ def main(*args):
    return 0


-
 def log(msg, *args, **kwargs):
    print("I: " + msg.format(*args, **kwargs), file=sys.stderr)

@@ -114,7 +113,6 @@ def get_data(week, max_age=3600):
    return data


-
 def get_author(week_start, week_end):
    return os.environ.get('DEBFULLNAME', 'FIXME')

@@ -159,7 +157,8 @@ def bugs(week_start, week_end, extra="true"):
        AND
            {extra}
        AND
-            CAST(arrival AS DATE) BETWEEN to_timestamp(@{week_start}) AND to_timestamp(@{week_end})
+            CAST(arrival AS DATE) BETWEEN
+            to_timestamp(@{week_start}) AND to_timestamp(@{week_end})
    """.format(**{
        'fields': ', '.join(fields),
        'extra': extra,
@@ -199,7 +198,8 @@ def get_uploads(week_start, week_end):
        WHERE
            source IN ({sources})
        AND
-            CAST(date AS date) BETWEEN to_timestamp(@{week_start}) AND to_timestamp(@{week_end})
+            CAST(date AS date) BETWEEN
+            to_timestamp(@{week_start}) AND to_timestamp(@{week_end})
        ORDER BY
            date
    """.format(**{
@@ -250,7 +250,8 @@ def get_packages_yml(week_start, week_end):

 def open_packages_yml(date):
    return subprocess.Popen(
-        "git show $(git rev-list -n1 --until @{0} origin/master):packages.yml".format(date),
+        "git show $(git rev-list -n1 --until @{0} "
+        "origin/master):packages.yml".format(date),
        shell=True,
        cwd=sibling_repo_gitdir('reproducible-notes'),
        stdout=subprocess.PIPE).stdout
@@ -290,6 +291,7 @@ def commits(week_start, week_end, project, path='.'):
        '--all',
        '--',
        path,
+        ':(exclude,glob)_blog/posts/*.md',
    ), cwd=git_dir).decode('utf-8')

    result = collections.defaultdict(list)
No results found