Commit 034491ea authored by Paul Spooren's avatar Paul Spooren Committed by Holger Levsen

reproducible openwrt: add snapshot build script

The script downloads all buildinfo and checksums from the OpenWrt
download server and tries to rebuild them. Once done the upstream
checksums are used to verify the newly created files.

A website based on templates-mustache/openwrt/ is rendered.

Use locally installed diffoscope.

Store created and origin packages in database.
Signed-off-by: Paul Spooren's avatarPaul Spooren <mail@aparcar.org>
parent 764d6ce9
......@@ -21,7 +21,7 @@ def parse_packages(package_list_fp):
packages = []
for line in package_list_fp:
if line == '\n':
if line == '\n' or line == '':
parser = email.parser.Parser()
package = parser.parsestr(linebuffer)
packages.append(package)
......
This diff is collapsed.
</div>
<hr id="footer_separator" />
<p style="font-size:0.9em;">
<div id="page_footer">
This page was built by the jenkins job <a href="https://jenkins.debian.net/job/reproducible_openwrt-target-ar71xx/"> reproducible_openwrt-target-ar71xx</a>
which is configured via this <a href="https://salsa.debian.org/qa/jenkins.debian.net">git repo</a>.
There is more information
<a href="/userContent/about.html">about jenkins.debian.net</a>
and about <a href="https://wiki.debian.org/ReproducibleBuilds"> reproducible builds of Debian</a>
available elsewhere.
<br /> Please send technical feedback about jenkins to <a href="mailto:qa-jenkins-dev@lists.alioth.debian.org">the Debian jenkins development list</a>,
or as a <a href="https://www.debian.org/Bugs/Reporting">bug report against the <tt>jenkins.debian.org</tt> package</a>.
Feedback about specific job results should go to their respective lists and/or the BTS.
<br />
The code of <a href="https://salsa.debian.org/qa/jenkins.debian.net">jenkins.debian.net</a>
is mostly GPL-2 licensed. The weather icons are public domain and were taken from the
<a href="http://tango.freedesktop.org/Tango_Icon_Library" target=_blank> Tango Icon Library</a>.
Copyright 2014-2019 <a href="mailto:holger@layer-acht.org">Holger Levsen</a>
and <a href="https://jenkins.debian.net/userContent/thanks.html">many others</a>.
</div>
</p>
</div>
</body>
</html>
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width">
<title>Reproducible OpenWrt ?</title>
<link rel='stylesheet' id='kamikaze-style-css' href='cascade.css?ver=4.0' type='text/css' media='all'>
</head>
<style type="text/css">
html,
body {
margin: 0;
padding: 0;
height: 100%;
}
body {
color: #333;
padding-top: 2em;
font-family: Helvetica, Arial, sans-serif;
width: 90%;
min-width: 700px;
max-width: 1100px;
margin: auto;
font-size: 120%;
background-color: #ddd;
}
h1 {
font-size: 120%;
line-height: 1em;
}
h2 {
font-size: 100%;
line-height: 1em;
}
table {
width: 100%;
box-shadow: 0 0 0.5em #999;
margin: 0;
border: none !important;
margin-bottom: 2em;
border-collapse: collapse;
border-spacing: 0;
}
th {
background: #000;
background: -webkit-linear-gradient(top, #444, #000);
background: -moz-linear-gradient(top, #444, #000);
background: -ms-linear-gradient(top, #444, #000);
background: -o-linear-gradient(top, #444, #000);
background: linear-gradient(top, #444, #000);
font-size: 14px;
line-height: 24px;
border: none;
text-align: left;
color: #fff;
}
tr {
background: rgba(255, 255, 255, 0.8);
}
tr:hover {
background: rgba(255, 255, 255, 0.6);
}
p,
th,
td {
font-size: 14px;
}
th,
td {
height: 20px;
vertical-align: middle;
white-space: nowrap;
padding: 0.2em 0.5em;
border: 1px solid #ccc;
}
a:link,
a:visited {
color: #337ab7;
font-weight: bold;
text-decoration: none;
}
a:hover,
a:active,
a:focus {
color: #23527c;
text-decoration: underline;
}
.s {
text-align: right;
width: 15%;
}
.d {
text-align: center;
width: 15%;
}
.sh {
font-family: monospace;
}
</style>
<body>
<div id="content">
<pre>
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt {{ version }}, {{ commit_string }}
-----------------------------------------------------
</pre>
</div>
<div id="main-content">
<h1>OpenWrt - <em>reproducible</em> wireless freedom!</h1>
<p>
<em>Reproducible builds</em> enable anyone to reproduce bit by bit
identical binary packages from a given source, so that anyone can verify
that a given binary derived from the source it was said to be derived.
There is more information about
<a href="https://wiki.debian.org/ReproducibleBuilds">reproducible builds on the Debian wiki</a>
and on
<a href="https://reproducible-builds.org">https://reproducible-builds.org</a>.
These pages explain in more depth why this is
useful, what common issues exist and which
workarounds and solutions are known.
</p>
<p>
<em>Reproducible OpenWrt</em> is an effort to apply this to OpenWrt. Thus each OpenWrt target is build twice, with a few variations added and then the resulting images and packages from the two builds are compared using <a href="https://tracker.debian.org/diffoscope">diffoscope</a>. OpenWrt generates many different types of raw <code>.bin</code> files, and diffoscope does not know how to parse these. Thus the resulting diffoscope output is not nearly as clear as it could be - hopefully this limitation will be overcome eventually, but in the meanwhile the input components (uImage kernel file, rootfs.tar.gz, and/or rootfs squashfs) can be inspected. Also please note that the toolchain is not varied at all as the rebuild happens on exactly the same system. More variations are expected to be seen in the wild.</p>
<p>
There is a weekly run <a href="https://jenkins.debian.net/view/reproducible/job/reproducible_openwrt/">jenkins
job</a> to test the <code>master</code> branch of <a href="https://github.com/openwrt/openwrt.git">OpenWrt.git</a>. The
jenkins job is running <a href="https://salsa.debian.org/qa/jenkins.debian.net/tree/master/bin/reproducible_openwrt.sh">reproducible_openwrt.sh</a>
in a Debian environment and this script is solely responsible for creating
this page. Feel invited to join <code>#reproducible-builds</code> (on
irc.oftc.net) to request job runs whenever sensible. Patches and other <a href="mailto:reproducible-builds@lists.alioth.debian.org">feedback</a> are
very much appreciated - if you want to help, please start by looking at the
<a href="https://jenkins.debian.net/userContent/todo.html#_reproducible_openwrt">ToDo
list for OpenWrt</a>, you might find something easy to contribute. <br />Thanks to <a href="https://www.profitbricks.co.uk">Profitbricks</a> for
donating the virtual machines this is running on!</p>
<p>
<ul>
{{ #targets }}
<li><a href="{{ version }}/{{ name}}">{{ version }} - {{ name }}</a></li>
{{ /targets }}
</ul>
</p>
<table>
<tr>
<th>git commit log</th>
</tr>
<tr>
<td>
<pre>{{ git_log_oneline }}</pre>
</td>
</tr>
</table>
<p>
{{ images_repro }} ({{ images_repro_percent }}%) out of {{ images_total }}
built images and {{ packages_repro }} ({{ packages_repro_percent }}%) out of
{{ packages_total }} built packages were reproducible in our test setup.
</p>
<p>
These tests were last run on {{ today }} for version using diffoscope {{
diffoscope_version }}.
</p>
<p style="clear:both;">
</p>
<table>
<tr>
<th>Images for <code>{{ target }}</code></th>
</tr>
{{ #images }}
<tr>
{{ #repro }}
<td><img src="/userContent/reproducible/static/weather-clear.png" alt="reproducible icon" /> {{ name }} ({{ sha256sum }}, {{ size }}K) is reproducible.</td>
{{ /repro }}
{{ ^repro }}
<td><a href="/{{ version }}/{{ target }}/{{ name }}.html"><img src="/userContent/reproducible/static/weather-showers-scattered.png" alt="unreproducible icon"> {{ name }}</a> ({{ size }}K) is unreproducible.</td>
{{ /repro }}
</tr>
{{ /images }}
</table>
<table>
<tr>
<th>Unreproducible and otherwise broken packages</th>
</tr>
{{ #packages }}
{{ ^repro }}
<tr>
<td><a href="/{{ version }}/{{ target }}/{{ name }}.html"><img src="/userContent/reproducible/static/weather-showers-scattered.png" alt="unreproducible icon"> {{ name }}</a> ({{ size }}K) is unreproducible.</td>
</tr>
{{ /repro }}
{{ /packages }}
</table>
<table>
<tr>
<th>Reproducible packages</th>
</tr>
{{ #packages }}
{{ #repro }}
<tr>
<td><img src="/userContent/reproducible/static/weather-clear.png" alt="reproducible icon" /> {{ name }} ({{ sha256sum }}, {{ size }}K) is reproducible.</td>
</tr>
{{ /repro }}
{{ /packages }}
</table>
<table>
<tr>
<th>Missing files after rebuild</th>
</tr>
{{ #missing}}
<tr>
<td>{{ name }}</td>
</tr>
{{ /missing }}
</table>
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment