Skip to content
......@@ -199,7 +199,7 @@ usage: reproducible_setup_notify.py [-h] [-o] [-p PACKAGES [PACKAGES ...]]
* Job configuration is at the usual location for 'jenkins.debian.net': there's a 'job-cfg/reproducible.yaml' defining all the jobs and lots of scripts in 'bin/reproducible_*.(sh|py)', plus a few config files like for 'sudo' or 'apache2'.
* Finally, there are also jobs testing the link:http://www.coreboot.org/[coreboot], link:https://lede-project.org/[LEDE], link:https://openwrt.org/[OpenWrt], link:http://www.netbsd.org/[NetBSD] and https://www.freebsd.org/[FreeBSD] projects. The results of the tests can be seen respectively at https://tests.reproducible-builds.org/coreboot/, https://tests.reproducible-builds.org/openwrt/, https://tests.reproducible-builds.org/netbsd/ and https://tests.reproducible-builds.org/freebsd/.
* Finally, there are also jobs testing the link:http://www.coreboot.org/[coreboot], link:https://openwrt.org/[OpenWrt], link:http://www.netbsd.org/[NetBSD] and https://www.freebsd.org/[FreeBSD] projects. The results of the tests can be seen respectively at https://tests.reproducible-builds.org/coreboot/, https://tests.reproducible-builds.org/openwrt/, https://tests.reproducible-builds.org/netbsd/ and https://tests.reproducible-builds.org/freebsd/.
=== torbrowser-launcher jobs
......
......@@ -7,8 +7,8 @@
** 15 cores and 48 GB memory for profitbricks-build1-amd64.debian.net used for building amd64 Debian packages for tests.reproducible-builds.org (t.r-b.o)
** 10 cores and 36 GB memory for profitbricks-build2-i386.debian.net used for building i386
Debian packages for t.r-b.o
** 12 cores and 24 GB memory for profitbricks-build3-amd64.debian.net used for building OpenWrt, LEDE, coreboot, Fedora, Archlinux and NetBSD for t.r-b.o
** 11 cores and 24 GB memory for profitbricks-build4-amd64.debian.net used for building OpenWrt, LEDE, coreboot, Fedora, Archlinux and NetBSD for t.r-b.o, running in the future
** 12 cores and 24 GB memory for profitbricks-build3-amd64.debian.net used for building OpenWrt, coreboot, Fedora, Archlinux and NetBSD for t.r-b.o
** 11 cores and 24 GB memory for profitbricks-build4-amd64.debian.net used for building OpenWrt, coreboot, Fedora, Archlinux and NetBSD for t.r-b.o, running in the future
** 16 cores and 48 GB memory for profitbricks-build5-amd64.debian.net used for building amd64 Debian packages for t.r-b.o, running in the future
** 9 cores and 36 GB memory for profitbricks-build6-i386.debian.net used for building i386 Debian packages for t.r-b.o, running in the future
** 15 cores and 48 GB memory for profitbricks-build11-amd64.debian.net used for building amd64 Debian packages for tests.reproducible-builds.org (t.r-b.o)
......
......@@ -216,23 +216,19 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
* explain status in plain english
* use disorderfs for 2nd build
==== reproducible LEDE
==== reproducible OpenWrt
* add credit for logo/artwork
* explain status in plain english
* build path variation
* use disorderfs for 2nd build
* incorporate popular third-party ("external feeds") packages?
* html: build variations are wrong
* html: git commit output includes garbage
* html: css: add some space on the left side
* html: css: make it more look like https://lede-project.org/
==== reproducible OpenWrt
* add credit for logo/artwork
* build path variation
* incorporate popular third-party ("external feeds") packages?
* explain status in plain english
* use disorderfs for 2nd build
==== reproducible NetBSD
* explain status in plain english
......
......@@ -92,7 +92,7 @@ elif [[ "$*" =~ ^rsync\ --server\ --sender\ .*\ \.\ /var/lib/jenkins/jobs/.*/wor
elif [[ "$*" =~ ^rsync\ --server\ .*\ \.\ /srv/d-i/isos/ ]] ; then
exec rsync --server "$3" . "$5" ; croak "Exec failed";
elif [[ "$*" =~ ^rsync\ --server\ .*\ \.\ /srv/workspace/chroots/.* ]] ; then
# LEDE is using this to share files between master node1 node2.
# OpenWrt is using this to share files between master node1 node2.
exec rsync --server "$3" . "$5" ; croak "Exec failed";
elif [[ "$*" =~ ^rsync\ --server\ .*\ \.\ /srv/reproducible-results/.* ]] ; then
# allow to push files to /srv/reproducible-results/
......@@ -125,8 +125,6 @@ elif [[ "$*" =~ ^reproducible_coreboot ]] ; then
exec /srv/jenkins/bin/reproducible_coreboot.sh ; croak "Exec failed";
elif [[ "$*" =~ ^reproducible_openwrt ]] ; then
shift ; exec /srv/jenkins/bin/reproducible_openwrt.sh $@ ; croak "Exec failed";
elif [[ "$*" =~ ^reproducible_lede ]] ; then
shift ; exec /srv/jenkins/bin/reproducible_lede.sh $@ ; croak "Exec failed";
elif [[ "$*" =~ ^reproducible_netbsd ]] ; then
exec /srv/jenkins/bin/reproducible_netbsd.sh ; croak "Exec failed";
elif [[ "$*" =~ ^reproducible_freebsd ]] ; then
......
......@@ -319,8 +319,7 @@ write_page_header() {
write_page " <li><a href=\"/openwrt/\">OpenWrt</a></li>, "
write_page " <li><a href=\"/netbsd/\">NetBSD</a></li>, "
write_page " <li><a href=\"/freebsd/\">FreeBSD</a></li>, "
write_page " <li><a href=\"/archlinux/\">Arch Linux</a></li> "
write_page " and <li><a href=\"/lede/\">LEDE</a></li>, "
write_page " and <li><a href=\"/archlinux/\">Arch Linux</a></li> "
write_page " though not as thoroughly as Debian (yet?) - and testing of "
write_page " <li><a href=\"/rpms/fedora-23.html\">Fedora</a></li> "
write_page " has just begun, and there are plans to test "
......@@ -343,13 +342,9 @@ write_page_intro() {
write_page " <p><em>Reproducible Coreboot</em> is an effort to apply this to coreboot. Thus each coreboot.rom is build twice (without payloads), with a few variations added and then those two ROMs are compared using <a href=\"https://tracker.debian.org/diffoscope\">diffoscope</a>. Please note that the toolchain is not varied at all as the rebuild happens on exactly the same system. More variations are expected to be seen in the wild.</p>"
local PROJECTNAME="$1"
local PROJECTURL="https://review.coreboot.org/p/coreboot.git"
elif [ "$1" = "OpenWrt" ] || [ "$1" = "LEDE" ]; then
elif [ "$1" = "OpenWrt" ] ; then
local PROJECTNAME="$1"
if [ "$PROJECTNAME" = "OpenWrt" ] ; then
local PROJECTURL="https://github.com/openwrt/openwrt.git"
else
local PROJECTURL="https://git.lede-project.org/?p=source.git;a=summary"
fi
write_page " <p><em>Reproducible $PROJECTNAME</em> is an effort to apply this to $PROJECTNAME. Thus each $PROJECTNAME target is build twice, with a few variations added and then the resulting images and packages from the two builds are compared using <a href=\"https://tracker.debian.org/diffoscope\">diffoscope</a>. $PROJECTNAME generates many different types of raw <code>.bin</code> files, and diffoscope does not know how to parse these. Thus the resulting diffoscope output is not nearly as clear as it could be - hopefully this limitation will be overcome eventually, but in the meanwhile the input components (uImage kernel file, rootfs.tar.gz, and/or rootfs squashfs) can be inspected. Also please note that the toolchain is not varied at all as the rebuild happens on exactly the same system. More variations are expected to be seen in the wild.</p>"
elif [ "$1" = "NetBSD" ] ; then
write_page " <p><em>Reproducible NetBSD</em> is an effort to apply this to NetBSD. Thus each NetBSD target is build twice, with a few variations added and then the resulting files from the two builds are compared using <a href=\"https://tracker.debian.org/diffoscope\">diffoscope</a>. Please note that the toolchain is not varied at all as the rebuild happens on exactly the same system. More variations are expected to be seen in the wild.</p>"
......@@ -444,7 +439,7 @@ write_variation_table() {
write_page "</td><td>i-capture-the-hostname</td></tr>"
write_page "<tr><td>domainname</td><td>$(hostname -d)</td><td>i-capture-the-domainname</td></tr>"
else
if [ "$1" = "LEDE" ] || [ "$1" != "Arch Linux" ] || [ "$1" != "OpenWrt" ] ; then
if [ "$1" != "Arch Linux" ] || [ "$1" != "OpenWrt" ] ; then
write_page "<tr><td>hostname</td><td> profitbricks-build3-amd64 or profitbricks-build4-amd64</td><td>the other one</td></tr>"
else
write_page "<tr><td>hostname</td><td colspan=\"2\"> is not yet varied between rebuilds of $1.</td></tr>"
......
......@@ -223,7 +223,7 @@ build_job_health_page() {
FILTER[4]="(json|le_scheduler|meta|le_nodes|rsync|notes)"
FILTER[5]="archlinux"
FILTER[6]="coreboot"
FILTER[7]="(openwrt|lede)"
FILTER[7]="(openwrt)"
FILTER[8]="(le_netbsd|le_freebsd)"
FILTER[9]="fdroid"
FILTER[10]="fedora"
......
#!/bin/bash
# Copyright 2014-2017 Holger Levsen <holger@layer-acht.org>
# © 2015 Reiner Herrmann <reiner@reiner-h.de>
# 2016-2018 Alexander Couzens <lynxis@fe80.eu>
# released under the GPLv=2
OPENWRT_GIT_REPO=https://git.openwrt.org/openwrt/staging/lynxis.git
OPENWRT_GIT_BRANCH=master
DEBUG=false
LEDE_CONFIG=
LEDE_TARGET=
. /srv/jenkins/bin/common-functions.sh
common_init "$@"
# common code defining db access
. /srv/jenkins/bin/reproducible_common.sh
# common code defining functions for OpenWrt/LEDE
. /srv/jenkins/bin/reproducible_lede_common.sh
set -e
echo "$0 got called with '$*'"
# this script is called from positions
# * it's called from the reproducible_wrapper when running on the master
# * it's called from reproducible_opewnrt_common when doing remote builds
case $1 in
node)
shift
case $1 in
openwrt_build |\
openwrt_download |\
openwrt_get_banner |\
node_create_tmpdirs |\
node_debug |\
node_save_logs |\
node_cleanup_tmpdirs) ;; # this is the allowed list
*)
echo "Unsupported remote node function $*"
exit 1
;;
esac
"$@"
trap - INT TERM EXIT
exit 0
;;
master)
# master code following
LEDE_TARGET=$2
LEDE_CONFIG=$3
;;
*)
echo "Unsupported mode $1. Arguments are $*"
exit 1
;;
esac
#
# main
#
DATE=$(date -u +'%Y-%m-%d')
START=$(date +'%s')
TMPBUILDDIR=$(mktemp --tmpdir=/srv/workspace/chroots/ -d -t "rbuild-lede-build-${DATE}-XXXXXXXX") # used to build on tmpfs
RESULTSDIR=$(mktemp --tmpdir=/srv/reproducible-results -d -t rbuild-lede-results-XXXXXXXX) # accessable in schroots, used to compare results
BANNER_HTML=$(mktemp "--tmpdir=$RESULTSDIR")
trap master_cleanup_tmpdirs INT TERM EXIT
cd "$TMPBUILDDIR"
create_results_dirs lede
build_two_times lede "$LEDE_TARGET" "$LEDE_CONFIG"
# for now we only build one architecture until it's at most reproducible
#build_two_times x86_64 "CONFIG_TARGET_x86=y\nCONFIG_TARGET_x86_64=y\n"
#build_two_times ramips_rt288x_RTN15 "CONFIG_TARGET_ramips=y\nCONFIG_TARGET_ramips_rt288x=y\nCONFIG_TARGET_ramips_rt288x_RTN15=y\n"
#
# create html about toolchain used
#
echo "============================================================================="
echo "$(date -u) - Creating Documentation HTML"
echo "============================================================================="
# created & copied by build_two_times()
TOOLCHAIN_HTML=$RESULTSDIR/toolchain.html
# clean up builddir to save space on tmpfs
rm -rf "$TMPBUILDDIR/lede"
# run diffoscope on the results
# (this needs refactoring rather badly)
TIMEOUT="30m"
DIFFOSCOPE="$(schroot --directory /tmp -c "chroot:jenkins-reproducible-${DBDSUITE}-diffoscope" diffoscope -- --version 2>&1)"
echo "============================================================================="
echo "$(date -u) - Running $DIFFOSCOPE on LEDE images and packages."
echo "============================================================================="
DBD_HTML=$(mktemp "--tmpdir=$RESULTSDIR")
DBD_GOOD_PKGS_HTML=$(mktemp "--tmpdir=$RESULTSDIR")
DBD_BAD_PKGS_HTML=$(mktemp "--tmpdir=$RESULTSDIR")
# run diffoscope on the images
GOOD_IMAGES=0
ALL_IMAGES=0
SIZE=""
cd "$RESULTSDIR/b1/targets"
tree .
# call_diffoscope requires TMPDIR
TMPDIR=$RESULTSDIR
# iterate over all images (merge b1 and b2 images into one list)
# call diffoscope on the images
for target in * ; do
cd "$target"
for subtarget in * ; do
cd "$subtarget"
# search images in both paths to find non-existing ones
IMGS1=$(find -- * -type f -name "*.bin" -o -name "*.squashfs" | sort -u )
pushd "$RESULTSDIR/b2/targets/$target/$subtarget"
IMGS2=$(find -- * -type f -name "*.bin" -o -name "*.squashfs" | sort -u )
popd
echo " <table><tr><th>Images for <code>$target/$subtarget</code></th></tr>" >> "$DBD_HTML"
for image in $(printf "%s\n%s" "$IMGS1" "$IMGS2" | sort -u ) ; do
let ALL_IMAGES+=1
if [ ! -f "$RESULTSDIR/b1/targets/$target/$subtarget/$image" ] || [ ! -f "$RESULTSDIR/b2/targets/$target/$subtarget/$image" ] ; then
echo " <tr><td><img src=\"/userContent/static/weather-storm.png\" alt=\"ftbfs icon\" /> $image (${SIZE}) failed to build.</td></tr>" >> "$DBD_HTML"
rm -f "$BASE/lede/dbd/targets/$target/$subtarget/$image.html" # cleanup from previous (unreproducible) tests - if needed
continue
fi
if [ "$(sha256sum "$RESULTSDIR/b1/targets/$target/$subtarget/$image" "$RESULTSDIR/b2/targets/$target/$subtarget/$image" \
| cut -f 1 -d ' ' | uniq -c | wc -l)" != "1" ] ; then
call_diffoscope "targets/$target/$subtarget" "$image"
else
echo "$(date -u) - targets/$target/$subtarget/$image is reproducible, yip!"
fi
get_filesize "$image"
if [ -f "$RESULTSDIR/targets/$target/$subtarget/$image.html" ] ; then
mkdir -p "$BASE/lede/dbd/targets/$target/$subtarget"
mv "$RESULTSDIR/targets/$target/$subtarget/$image.html" "$BASE/lede/dbd/targets/$target/$subtarget/$image.html"
echo " <tr><td><a href=\"dbd/targets/$target/$subtarget/$image.html\"><img src=\"/userContent/static/weather-showers-scattered.png\" alt=\"unreproducible icon\" /> $image</a> (${SIZE}) is unreproducible.</td></tr>" >> "$DBD_HTML"
else
SHASUM=$(sha256sum "$image" |cut -d " " -f1)
echo " <tr><td><img src=\"/userContent/static/weather-clear.png\" alt=\"reproducible icon\" /> $image ($SHASUM, $SIZE) is reproducible.</td></tr>" >> "$DBD_HTML"
let GOOD_IMAGES+=1
rm -f "$BASE/lede/dbd/targets/$target/$subtarget/$image.html" # cleanup from previous (unreproducible) tests - if needed
fi
done
cd ..
echo " </table>" >> "$DBD_HTML"
done
cd ..
done
GOOD_PERCENT_IMAGES=$(echo "scale=1 ; ($GOOD_IMAGES*100/$ALL_IMAGES)" | bc | grep -qs . || echo 0.00)
# run diffoscope on the packages
GOOD_PACKAGES=0
ALL_PACKAGES=0
cd "$RESULTSDIR/b1"
for i in * ; do
cd "$i"
# search packages in both paths to find non-existing ones
PKGS1=$(find -- * -type f -name "*.ipk" | sort -u )
pushd "$RESULTSDIR/b2/$i"
PKGS2=$(find -- * -type f -name "*.ipk" | sort -u )
popd
for j in $(printf "%s\n%s" "$PKGS1" "$PKGS2" | sort -u ) ; do
let ALL_PACKAGES+=1
if [ ! -f "$RESULTSDIR/b1/$i/$j" ] || [ ! -f "$RESULTSDIR/b2/$i/$j" ] ; then
echo " <tr><td><img src=\"/userContent/static/weather-storm.png\" alt=\"ftbfs icon\" /> $j (${SIZE}) failed to build.</td></tr>" >> "$DBD_BAD_PKGS_HTML"
rm -f "$BASE/lede/dbd/$i/$j.html" # cleanup from previous (unreproducible) tests - if needed
continue
fi
if [ "$(sha256sum "$RESULTSDIR/b1/$i/$j" "$RESULTSDIR/b2/$i/$j" | cut -f 1 -d ' ' | uniq -c | wc -l)" != "1" ] ; then
call_diffoscope "$i" "$j"
else
echo "$(date -u) - $i/$j is reproducible, yip!"
fi
get_filesize "$j"
if [ -f "$RESULTSDIR/$i/$j.html" ] ; then
mkdir -p "$BASE/lede/dbd/$i/$(dirname "$j")"
mv "$RESULTSDIR/$i/$j.html" "$BASE/lede/dbd/$i/$j.html"
echo " <tr><td><a href=\"dbd/$i/$j.html\"><img src=\"/userContent/static/weather-showers-scattered.png\" alt=\"unreproducible icon\" /> $j</a> ($SIZE) is unreproducible.</td></tr>" >> "$DBD_BAD_PKGS_HTML"
else
SHASUM=$(sha256sum "$j" |cut -d " " -f1)
echo " <tr><td><img src=\"/userContent/static/weather-clear.png\" alt=\"reproducible icon\" /> $j ($SHASUM, $SIZE) is reproducible.</td></tr>" >> "$DBD_GOOD_PKGS_HTML"
let GOOD_PACKAGES+=1
rm -f "$BASE/lede/dbd/$i/$j.html" # cleanup from previous (unreproducible) tests - if needed
fi
done
cd ..
done
echo " <table><tr><th>Unreproducible and otherwise broken packages</th></tr>" >> "$DBD_HTML"
cat "$DBD_BAD_PKGS_HTML" >> "$DBD_HTML"
echo " </table>" >> "$DBD_HTML"
echo " <table><tr><th>Reproducible packages</th></tr>" >> "$DBD_HTML"
cat "$DBD_GOOD_PKGS_HTML" >> "$DBD_HTML"
echo " </table>" >> "$DBD_HTML"
GOOD_PERCENT_PACKAGES=$(echo "scale=1 ; ($GOOD_PACKAGES*100/$ALL_PACKAGES)" | bc | grep -qs . || echo 0.00)
# are we there yet?
if [ "$GOOD_PERCENT_IMAGES" = "100.0" ] || [ "$GOOD_PERCENT_PACKAGES" = "100.0" ]; then
MAGIC_SIGN="!"
else
MAGIC_SIGN="?"
fi
#
# finally create the webpage
#
cd "$RESULTSDIR" ; mkdir lede
PAGE=lede/lede_$LEDE_TARGET.html
cat > "$PAGE" <<- EOF
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width">
<title>Reproducible LEDE ?</title>
<link rel="stylesheet" href="foundation.css"/>
<link rel="stylesheet" href="font-awesome.css"/>
<link rel="stylesheet" href="coderay.css"/>
<link rel="stylesheet" href="asciidoctor.css"/>
<link rel="stylesheet" href="lede.css"/>
</head>
<body>
<div id="content">
<pre>
EOF
cat "$BANNER_HTML" >> "$PAGE"
write_page " </pre>"
write_page " </div><div id=\"main-content\">"
write_page " <h1>LEDE - <em>reproducible</em> wireless freedom$MAGIC_SIGN</h1>"
write_page_intro LEDE
write_page " <p>$GOOD_IMAGES ($GOOD_PERCENT_IMAGES%) out of $ALL_IMAGES built images and $GOOD_PACKAGES ($GOOD_PERCENT_PACKAGES%) out of $ALL_PACKAGES built packages were reproducible in our test setup."
write_page " These tests were last run on $DATE for version ${OPENWRT_VERSION} using ${DIFFOSCOPE}.</p>"
write_variation_table LEDE
cat "$DBD_HTML" >> "$PAGE"
cat "$TOOLCHAIN_HTML" >> "$PAGE"
write_page " </div>"
write_page_footer LEDE
publish_page
rm -f "$DBD_HTML" "$DBD_GOOD_PKGS_HTML" "$DBD_BAD_PKGS_HTML" "$TOOLCHAIN_HTML" "$BANNER_HTML"
# the end
calculate_build_duration
print_out_duration
for CHANNEL in reproducible-builds lede-dev ; do
irc_message $CHANNEL "$REPRODUCIBLE_URL/$PAGE has been updated. ($GOOD_PERCENT_IMAGES% images and $GOOD_PERCENT_PACKAGES% packages reproducible in our current test framework.)"
done
echo "============================================================================="
# remove everything, we don't need it anymore...
master_cleanup_tmpdirs success
trap - INT TERM EXIT
#!/bin/bash
# Copyright 2014-2017 Holger Levsen <holger@layer-acht.org>
# © 2015 Reiner Herrmann <reiner@reiner-h.de>
# 2016-2017 Alexander Couzens <lynxis@fe80.eu>
# released under the GPLv=2
# configuration
GENERIC_NODE1=profitbricks-build3-amd64.debian.net
GENERIC_NODE2=profitbricks-build4-amd64.debian.net
# run on jenkins master
node_debug() {
ls -al "$1" || true
ls -al "$1/" || true
ls -al "$1/download" || true
}
# only called direct on a remote build node
node_cleanup_tmpdirs() {
export TMPBUILDDIR=$1
cd
# (very simple) check we are deleting the right stuff
if [ "${TMPBUILDDIR:0:23}" != "/srv/workspace/chroots/" ] || [ ${#TMPBUILDDIR} -le 23 ] ; then
echo "Something very strange with \$TMPBUILDDIR=$TMPBUILDDIR exiting instead of doing cleanup."
exit 1
fi
rm -rf "$TMPBUILDDIR"
}
node_create_tmpdirs() {
export TMPBUILDDIR=$1
# (very simple) check what we are creating
if [ "${TMPBUILDDIR:0:23}" != "/srv/workspace/chroots/" ] || [ ${#TMPBUILDDIR} -le 23 ] ; then
echo "Something very strange with \$TMPBUILDDIR=$TMPBUILDDIR exiting instead of doing create."
exit 1
fi
mkdir -p "$TMPBUILDDIR/download"
}
# called as trap handler and also to cleanup after a success build
master_cleanup_tmpdirs() {
# we will save the logs in case we got called as trap handler
# in a success build the logs are saved on a different function
if [ "$1" != "success" ] ; then
# job failed
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node node_save_logs "$TMPBUILDDIR" || true
ssh "$GENERIC_NODE2" "reproducible_$TYPE" node node_save_logs "$TMPBUILDDIR" || true
# save failure logs
mkdir -p "$WORKSPACE/results/"
rsync -av "$GENERIC_NODE1:$RESULTSDIR/build_logs.tar.xz" "$WORKSPACE/results/build_logs_b1.tar.xz" || true
rsync -av "$GENERIC_NODE2:$RESULTSDIR/build_logs.tar.xz" "$WORKSPACE/results/build_logs_b2.tar.xz" || true
fi
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node node_cleanup_tmpdirs "$TMPBUILDDIR" || true
ssh "$GENERIC_NODE2" "reproducible_$TYPE" node node_cleanup_tmpdirs "$TMPBUILDDIR" || true
cd
# (very simple) check we are deleting the right stuff
if [ "${RESULTSDIR:0:26}" != "/srv/reproducible-results/" ] || [ ${#RESULTSDIR} -le 26 ] || \
[ "${TMPBUILDDIR:0:23}" != "/srv/workspace/chroots/" ] || [ ${#TMPBUILDDIR} -le 23 ] ; then
echo "Something very strange with \$RESULTSDIR=$RESULTSDIR or \$TMPBUILDDIR=$TMPBUILDDIR, exiting instead of doing cleanup."
exit 1
fi
rm -rf "$RESULTSDIR"
rm -rf "$TMPBUILDDIR"
if [ -f "$BANNER_HTML" ] ; then
rm -f "$BANNER_HTML"
fi
}
create_results_dirs() {
local project=$1
mkdir -p "$BASE/$project/dbd"
}
node_document_environment() {
local tmpdir=$1
local toolchain_html=$tmpdir/toolchain.html
cd "$tmpdir/build/source"
cat /dev/null > "$toolchain_html"
echo " <table><tr><th>git commit built</th></tr><tr><td><code>" >> "$toolchain_html"
git log -1 >> "$toolchain_html"
echo " </code></td></tr></table>" >> "$toolchain_html"
echo "<table><tr><th>Target toolchains built</th></tr>" >> "$toolchain_html"
for i in $(ls -1d staging_dir/toolchain*|cut -d "-" -f2-|xargs echo) ; do
echo " <tr><td><code>$i</code></td></tr>" >> "$toolchain_html"
done
echo "</table>" >> "$toolchain_html"
echo "<table><tr><th>Contents of <code>build_dir/host/</code></th></tr>" >> "$toolchain_html"
for i in $(ls -1 build_dir/host/) ; do
echo " <tr><td>$i</td></tr>" >> "$toolchain_html"
done
echo "</table>" >> "$toolchain_html"
echo "<table><tr><th>Downloaded software</th></tr>" >> "$toolchain_html"
for i in $(ls -1 dl/) ; do
echo " <tr><td>$i</td></tr>" >> "$toolchain_html"
done
echo "</table>" >> "$toolchain_html"
echo "<table><tr><th>Debian $(cat /etc/debian_version) package on $(dpkg --print-architecture)</th><th>installed version</th></tr>" >> "$toolchain_html"
for i in gcc binutils bzip2 flex python perl make findutils grep diffutils unzip gawk util-linux zlib1g-dev libc6-dev git subversion ; do
echo " <tr><td>$i</td><td>" >> "$toolchain_html"
dpkg -s $i|grep '^Version'|cut -d " " -f2 >> "$toolchain_html"
echo " </td></tr>" >> "$toolchain_html"
done
echo "</table>" >> "$toolchain_html"
cd -
}
# node_save_logs can be called over ssh OR called within openwrt_build
# it's always only run on a remote host.
node_save_logs() {
local tmpdir=$1
if [ "${tmpdir:0:23}" != "/srv/workspace/chroots/" ] || [ ${#tmpdir} -le 23 ] ; then
echo "Something very strange with \$TMPDIR=$tmpdir exiting instead of doing node_save_logs."
exit 1
fi
if [ ! -d "$tmpdir/build/source/logs" ] ; then
# we create an empty tar.xz instead of failing
touch "$tmpdir/build_logs.tar.xz"
else
tar cJf "$tmpdir/build_logs.tar.xz" -C "$tmpdir/build/source" ./logs
fi
node_document_environment "$tmpdir"
}
# RUN - is b1 or b2. b1 for first run, b2 for second
# save the images and packages under $TMPDIR/$RUN
# run on the master
save_lede_results() {
RUN=$1
# first save all images and target specific packages
pushd bin/targets
for target in * ; do
pushd "$target" || continue
for subtarget in * ; do
pushd "$subtarget" || continue
# save firmware images
mkdir -p "$TMPDIR/$RUN/targets/$target/$subtarget/"
for image in $(find * -name "*.bin" -o -name "*.squashfs") ; do
cp -p "$image" "$TMPDIR/$RUN/targets/$target/$subtarget/"
done
# save subtarget specific packages
if [ -d packages ] ; then
pushd packages
for package in $(find * -name "*.ipk") ; do
mkdir -p $TMPDIR/$RUN/packages/$target/$subtarget/$(dirname $package) || ( echo $TMPDIR/$RUN/packages/$target/$subtarget/$(dirname $package) ; continue )
cp -p $package $TMPDIR/$RUN/packages/$target/$subtarget/$(dirname $package)/
done
popd
fi
popd
done
popd
done
popd
# save all generic packages
# arch is like mips_34kc_dsp
pushd bin/packages/
for arch in * ; do
pushd "$arch" || continue
for feed in * ; do
pushd "$feed" || continue
for package in $(find * -name "*.ipk") ; do
mkdir -p "$TMPDIR/$RUN/packages/$arch/$feed/$(dirname "$package")"
cp -p "$package" "$TMPDIR/$RUN/packages/$arch/$feed/$(dirname "$package")/"
done
popd
done
popd
done
popd
}
# RUN - is b1 or b2. b1 for first run, b2 for second
# save the images and packages under $TMPDIR/$RUN
save_openwrt_results() {
RUN=$1
cd bin
for i in * ; do
cd "$i"
# save images
mkdir -p "$TMPDIR/$RUN/$i"
for j in $(find * -name "*.bin" -o -name "*.squashfs") ; do
cp -p "$j" "$TMPDIR/$RUN/$i/"
done
# save packages
cd packages
for j in $(find * -name "*.ipk") ; do
mkdir -p "$TMPDIR/$RUN/$i/$(dirname "$j")"
cp -p "$j" "$TMPDIR/$RUN/$i/$(dirname "$j")/"
done
cd ../..
done
cd ..
}
# apply variations change the environment for
# the subsequent run
# RUN - b1 or b2. b1 for first run, b2 for the second
openwrt_apply_variations() {
RUN=$1
if [ "$RUN" = "b1" ] ; then
export TZ="/usr/share/zoneinfo/Etc/GMT+12"
export MAKE=make
else
export TZ="/usr/share/zoneinfo/Etc/GMT-14"
export LANG="fr_CH.UTF-8"
export LC_ALL="fr_CH.UTF-8"
export PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path"
export CAPTURE_ENVIRONMENT="I capture the environment"
# needs to fix openwrt/lede ;)
# umask 0002
# use allmost all cores for second build
export NEW_NUM_CPU=$(echo "$NUM_CPU-1" | bc)
export MAKE=make
fi
}
openwrt_config() {
CONFIG=$1
printf "$CONFIG\n" | grep '^[^ ]' > .config
printf "CONFIG_ALL=y\n" >> .config
printf "CONFIG_AUTOREMOVE=y\n" >> .config
printf "CONFIG_BUILDBOT=y\n" >> .config
printf "CONFIG_CLEAN_IPKG=y\n" >> .config
printf "CONFIG_TARGET_ROOTFS_TARGZ=y\n" >> .config
printf 'CONFIG_KERNEL_BUILD_USER="lede"\n' >> .config
printf 'CONFIG_KERNEL_BUILD_DOMAIN="buildhost"\n' >> .config
make defconfig
}
openwrt_build_toolchain() {
echo "============================================================================="
echo "$(date -u) - Building the toolchain."
echo "============================================================================="
OPTIONS="-j $NUM_CPU IGNORE_ERRORS=ym BUILD_LOG=1"
ionice -c 3 make $OPTIONS tools/install
ionice -c 3 make $OPTIONS toolchain/install
}
# TYPE - openwrt or lede
# RUN - b1 or b2. b1 means first run, b2 second
# TARGET - a target including subtarget. E.g. ar71xx_generic
openwrt_compile() {
local TYPE=$1
local RUN=$2
local TARGET=$3
OPTIONS="-j $NUM_CPU IGNORE_ERRORS=ym BUILD_LOG=1"
# make $RUN more human readable
[ "$RUN" = "b1" ] && RUN="first"
[ "$RUN" = "b2" ] && RUN="second"
echo "============================================================================="
echo "$(date -u) - Building $TYPE ${OPENWRT_VERSION} ($TARGET) - $RUN build run."
echo "============================================================================="
ionice -c 3 $MAKE $OPTIONS
}
openwrt_create_signing_keys() {
echo "============================================================================="
cat <<- EOF
# LEDE signs the release with a signing key, but generate the signing key if not
# present. To have a reproducible release we need to take care of signing keys.
# LEDE will also put the key-build.pub into the resulting image (pkg: base-files)!
# At the end of the build it will use the key-build to sign the Packages repo list.
# Use a workaround this problem:
# key-build.pub contains the pubkey of LEDE buildbot
# key-build contains our build key
# Meaning only signed files will be different but not the images.
# Packages.sig is unreproducible.
# here is our random signing key
# chosen by fair dice roll.
# guaranteed to be random.
# private key
EOF
echo -e 'untrusted comment: Local build key\nRWRCSwAAAAB12EzgExgKPrR4LMduadFAw1Z8teYQAbg/EgKaN9SUNrgteVb81/bjFcvfnKF7jS1WU8cDdT2VjWE4Cp4cxoxJNrZoBnlXI+ISUeHMbUaFmOzzBR7B9u/LhX3KAmLsrPc=' | tee key-build
echo "\n# public key"
echo -e 'untrusted comment: Local build key\nRWQ/EgKaN9SUNja2aAZ5VyPiElHhzG1GhZjs8wUewfbvy4V9ygJi7Kz3' | tee key-build.pub
echo "# override the pubkey with 'LEDE usign key for unattended build jobs' to have the same base-files pkg and images"
echo -e 'untrusted comment: LEDE usign key for unattended build jobs\nRWS1BD5w+adc3j2Hqg9+b66CvLR7NlHbsj7wjNVj0XGt/othDgIAOJS+' | tee key-build.pub
echo "============================================================================="
}
# called by openwrt_two_times
# ssh $GENERIC_NODE1 reproducible_$TYPE node openwrt_download $TYPE $TARGET $CONFIG $TMPDIR
openwrt_download() {
local TARGET=$1
local CONFIG=$2
local TMPBUILDDIR=$3
local tries=5
cd "$TMPBUILDDIR/download"
# checkout the repo
echo "================================================================================"
echo "$(date -u) - Cloning git repository from $OPENWRT_GIT_REPO $OPENWRT_GIT_BRANCH. "
echo "================================================================================"
git clone -b "$OPENWRT_GIT_BRANCH" "$OPENWRT_GIT_REPO" source
cd source
echo "================================================================================"
echo "$(date -u) - received git version $(git log -1 --pretty=oneline)"
echo "================================================================================"
# otherwise LEDE will generate new release keys every build
openwrt_create_signing_keys
# update feeds
./scripts/feeds update
./scripts/feeds install -a
# configure openwrt because otherwise it wont download everything
openwrt_config "$CONFIG"
while ! make tools/tar/compile download -j "$NUM_CPU" IGNORE_ERRORS=ym BUILD_LOG=1 ; do
tries=$((tries - 1))
if [ $tries -eq 0 ] ; then
echo "================================================================================"
echo "$(date -u) - Failed to download sources"
echo "================================================================================"
exit 1
fi
done
}
openwrt_get_banner() {
local TMPDIR=$1
local TYPE=$2
cd "$TMPDIR/build/source"
echo "===bannerbegin==="
find staging_dir/ -name banner | grep etc/banner|head -1| xargs cat /dev/null
echo "===bannerend==="
}
# openwrt_build is run on a remote host
# TYPE - openwrt or lede
# RUN - b1 or b2. b1 means first run, b2 second
# TARGET - a target including subtarget. E.g. ar71xx_generic
# CONFIG - a simple basic .config as string. Use \n to seperate lines
# TMPPATH - is a unique path generated with mktmp
# lede has a different output directory than openwrt
openwrt_build() {
local TYPE=$1
local RUN=$2
local TARGET=$3
local CONFIG=$4
export TMPDIR=$5
export TMPBUILDDIR=$TMPDIR/build/
mv "$TMPDIR/download" "$TMPBUILDDIR"
# openwrt/lede is checkouted under /download
cd "$TMPBUILDDIR/source"
# set tz, date, core, ..
openwrt_apply_variations "$RUN"
openwrt_build_toolchain
# build images and packages
openwrt_compile "$TYPE" "$RUN" "$TARGET"
# save the results
[ "$TYPE" = "lede" ] && save_lede_results "$RUN"
[ "$TYPE" = "openwrt" ] && save_openwrt_results "$RUN"
# copy logs
node_save_logs "$TMPDIR"
}
# build openwrt/lede on two different hosts
# TARGET a target including subtarget. E.g. ar71xx_generic
# CONFIG - a simple basic .config as string. Use \n to seperate lines
# TYPE - openwrt or lede
# lede has a different output directory than openwrt
build_two_times() {
TYPE=$1
TARGET=$2
CONFIG=$3
# create openwrt
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node node_create_tmpdirs "$TMPBUILDDIR"
ssh "$GENERIC_NODE2" "reproducible_$TYPE" node node_create_tmpdirs "$TMPBUILDDIR"
mkdir -p "$TMPBUILDDIR/download/"
# create results directory saved by jenkins as artifacts
mkdir -p "$WORKSPACE/results/"
# download and prepare openwrt on node b1
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node openwrt_download "$TARGET" "$CONFIG" "$TMPBUILDDIR"
echo "== master"
ls -la "$TMPBUILDDIR/download/" || true
echo "== node1"
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node node_debug "$TMPBUILDDIR"
echo "== node2"
ssh "$GENERIC_NODE2" "reproducible_$TYPE" node node_debug "$TMPBUILDDIR"
rsync -a "$GENERIC_NODE1:$TMPBUILDDIR/download/" "$TMPBUILDDIR/download/"
rsync -a "$TMPBUILDDIR/download/" "$GENERIC_NODE2:$TMPBUILDDIR/download/"
## first run
RUN=b1
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node openwrt_build "$TYPE" "$RUN" "$TARGET" "$CONFIG" "$TMPBUILDDIR"
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node openwrt_get_banner "$TMPBUILDDIR" "$TYPE" > "$BANNER_HTML"
# cut away everything before begin and after the end…
# (thats noise generated by the way we run this via reproducible_common.sh)
cat "$BANNER_HTML" | sed '/===bannerend===/,$d' | tac | sed '/===bannerbegin===/,$d' | tac > "$BANNER_HTML.out"
mv "$BANNER_HTML".out "$BANNER_HTML"
# rsync back logs and images
rsync -av "$GENERIC_NODE1:$TMPBUILDDIR/$RUN/" "$RESULTSDIR/$RUN/"
rsync -av "$GENERIC_NODE1:$TMPBUILDDIR/build_logs.tar.xz" "$WORKSPACE/results/build_logs_b1.tar.xz"
rsync -av "$GENERIC_NODE1:$TMPBUILDDIR/toolchain.html" "$RESULTSDIR/toolchain.html"
ssh "$GENERIC_NODE1" "reproducible_$TYPE" node node_cleanup_tmpdirs "$TMPBUILDDIR"
## second run
RUN=b2
ssh "$GENERIC_NODE2" "reproducible_$TYPE" node openwrt_build "$TYPE" "$RUN" "$TARGET" "$CONFIG" "$TMPBUILDDIR"
# rsync back logs and images
rsync -av "$GENERIC_NODE2:$TMPBUILDDIR/$RUN/" "$RESULTSDIR/$RUN/"
rsync -av "$GENERIC_NODE2:$TMPBUILDDIR/build_logs.tar.xz" "$WORKSPACE/results/build_logs_b2.tar.xz"
ssh "$GENERIC_NODE2" "reproducible_$TYPE" node node_cleanup_tmpdirs "$TMPBUILDDIR"
}
This diff is collapsed.
......@@ -120,10 +120,10 @@ RewriteRule ^/(.*) /coreboot/coreboot.html [R,L]
RewriteCond %{REQUEST_URI} ^/openwrt(/|)$
RewriteRule ^/(.*) /openwrt/openwrt.html [R,L]
# redirect /lede/ to lede/lede.html
# redirect /lede/ to openwrt/openwrt.html
# note: the missing slash in the RewriteRule is wanted to avoid a double slash
RewriteCond %{REQUEST_URI} ^/lede(/|)$
RewriteRule ^/(.*) /lede/lede.html [R,L]
RewriteRule ^/(.*) /openwrt/openwrt.html [R,L]
# redirect /netbsd/ to netbsd/netbsd.html
# note: the missing slash in the RewriteRule is wanted to avoid a double slash
......
......@@ -584,28 +584,24 @@
- '{name}_{otherproject_artifacts}':
otherproject_artifacts:
- 'lede-target-ar71xx':
- 'openwrt-target-ar71xx':
my_timed: '42 21 * * 4'
my_lede_config: 'CONFIG_TARGET_ar71xx=y'
my_lede_target: 'ar71xx'
- 'lede-target-ramips':
my_openwrt_config: 'CONFIG_TARGET_ar71xx=y'
my_openwrt_target: 'ar71xx'
- 'openwrt-target-ramips':
my_timed: '42 23 * * 4'
my_lede_config: 'CONFIG_TARGET_ramips=y'
my_lede_target: 'ramips'
- 'lede-target-x86':
my_openwrt_config: 'CONFIG_TARGET_ramips=y'
my_openwrt_target: 'ramips'
- 'openwrt-target-x86':
my_timed: '42 1 * * 5'
my_lede_config: 'CONFIG_TARGET_x86=y'
my_lede_target: 'x86'
- 'openwrt':
my_description: 'Rebuild OpenWrt (some targets, some packages) twice (with variations) and investigate regarding reproducibility.'
my_timed: '42 23 * * 2'
my_shell: '/srv/jenkins/bin/reproducible_openwrt.sh master'
my_description: 'Rebuild LEDE (one target, some packages) twice (with variations) and investigate regarding reproducibility.'
my_shell: '/srv/jenkins/bin/reproducible_lede.sh master {my_lede_target} {my_lede_config}'
my_openwrt_config: 'CONFIG_TARGET_x86=y'
my_openwrt_target: 'x86'
my_description: 'Rebuild OpenWrt (one target, some packages) twice (with variations) and investigate regarding reproducibility.'
my_shell: '/srv/jenkins/bin/reproducible_openwrt.sh master {my_openwrt_target} {my_openwrt_config}'
my_trigger: ''
my_hname: ''
my_recipients: 'jenkins+reproducible-builds qa-jenkins-scm@lists.alioth.debian.org lynxis@fe80.eu'
my_blocker: '^reproducible_(lede|openwrt).*'
my_blocker: '^reproducible_openwrt.*'
- '{name}_{otherproject_triggering}':
otherproject_triggering:
......
......@@ -12,7 +12,6 @@
<a href="https://tests.reproducible-builds.org/debian/">Debian</a> /
<a href="https://tests.reproducible-builds.org/fedora/">Fedora</a> /
<a href="https://tests.reproducible-builds.org/freebsd/">FreeBSD</a> /
<a href="https://tests.reproducible-builds.org/lede/">LEDE</a> /
<a href="https://tests.reproducible-builds.org/netbsd/">NetBSD</a> /
<a href="https://tests.reproducible-builds.org/openwrt/">OpenWrt</a>
</li>
......
......@@ -303,7 +303,7 @@ if [ -f /etc/debian_version ] ; then
codethink-sled9*|codethink-sled11*|codethink-sled13*|codethink-sled15*) DEBS="$DEBS ntpdate" ;;
*) ;;
esac
# needed to run coreboot/openwrt/lede/netbsd/fedora jobs
# needed to run coreboot/openwrt/netbsd/fedora jobs
case $HOSTNAME in
profitbricks-build3-amd64|profitbricks-build4-amd64) DEBS="$DEBS
bison
......
This diff is collapsed.
/* Stylesheet for CodeRay to match GitHub theme | MIT License | http://foundation.zurb.com */
pre.CodeRay {background-color:#f7f7f8; border: solid; border-color: #dfdfdf; border-width: 1px;}
.CodeRay code{border-style: none; padding: 0}
.CodeRay .line-numbers{border-right:1px solid #d8d8d8;padding:0 0.5em 0 .25em}
.CodeRay span.line-numbers{display:inline-block;margin-right:.5em;color:rgba(0,0,0,.3)}
.CodeRay .line-numbers strong{font-weight: normal}
table.CodeRay{border-collapse:separate;border-spacing:0;margin-bottom:0;border:0;background:none}
table.CodeRay td{vertical-align: top}
table.CodeRay td.line-numbers{text-align:right}
table.CodeRay td.line-numbers>pre{padding:0;color:rgba(0,0,0,.3)}
table.CodeRay td.code{padding:0 0 0 .5em}
table.CodeRay td.code>pre{padding:0}
.CodeRay .debug{color:#fff !important;background:#000080 !important}
.CodeRay .annotation{color:#007}
.CodeRay .attribute-name{color:#000080}
.CodeRay .attribute-value{color:#700}
.CodeRay .binary{color:#509}
.CodeRay .comment{color:#998;font-style:italic}
.CodeRay .char{color:#04d}
.CodeRay .char .content{color:#04d}
.CodeRay .char .delimiter{color:#039}
.CodeRay .class{color:#458;font-weight:bold}
.CodeRay .complex{color:#a08}
.CodeRay .constant,.CodeRay .predefined-constant{color:#008080}
.CodeRay .color{color:#099}
.CodeRay .class-variable{color:#369}
.CodeRay .decorator{color:#b0b}
.CodeRay .definition{color:#099}
.CodeRay .delimiter{color:#000}
.CodeRay .doc{color:#970}
.CodeRay .doctype{color:#34b}
.CodeRay .doc-string{color:#d42}
.CodeRay .escape{color:#666}
.CodeRay .entity{color:#800}
.CodeRay .error{color:#808}
.CodeRay .exception{color:inherit}
.CodeRay .filename{color:#099}
.CodeRay .function{color:#900;font-weight:bold}
.CodeRay .global-variable{color:#008080}
.CodeRay .hex{color:#058}
.CodeRay .integer,.CodeRay .float{color:#099}
.CodeRay .include{color:#555}
.CodeRay .inline{color:#00}
.CodeRay .inline .inline{background:#ccc}
.CodeRay .inline .inline .inline{background:#bbb}
.CodeRay .inline .inline-delimiter{color:#d14}
.CodeRay .inline-delimiter{color:#d14}
.CodeRay .important{color:#555;font-weight:bold}
.CodeRay .interpreted{color:#b2b}
.CodeRay .instance-variable{color:#008080}
.CodeRay .label{color:#970}
.CodeRay .local-variable{color:#963}
.CodeRay .octal{color:#40e}
.CodeRay .predefined{color:#369}
.CodeRay .preprocessor{color:#579}
.CodeRay .pseudo-class{color:#555}
.CodeRay .directive{font-weight:bold}
.CodeRay .type{font-weight:bold}
.CodeRay .predefined-type{color:inherit}
.CodeRay .reserved,.CodeRay .keyword {color:#000;font-weight:bold}
.CodeRay .key{color:#808}
.CodeRay .key .delimiter{color:#606}
.CodeRay .key .char{color:#80f}
.CodeRay .value{color:#088}
.CodeRay .regexp .delimiter{color:#808}
.CodeRay .regexp .content{color:#808}
.CodeRay .regexp .modifier{color:#808}
.CodeRay .regexp .char{color:#d14}
.CodeRay .regexp .function{color:#404;font-weight:bold}
.CodeRay .string{color:#d20}
.CodeRay .string .string .string{background:#ffd0d0}
.CodeRay .string .content{color:#d14}
.CodeRay .string .char{color:#d14}
.CodeRay .string .delimiter{color:#d14}
.CodeRay .shell{color:#d14}
.CodeRay .shell .delimiter{color:#d14}
.CodeRay .symbol{color:#990073}
.CodeRay .symbol .content{color:#a60}
.CodeRay .symbol .delimiter{color:#630}
.CodeRay .tag{color:#008080}
.CodeRay .tag-special{color:#d70}
.CodeRay .variable{color:#036}
.CodeRay .insert{background:#afa}
.CodeRay .delete{background:#faa}
.CodeRay .change{color:#aaf;background:#007}
.CodeRay .head{color:#f8f;background:#505}
.CodeRay .insert .insert{color:#080}
.CodeRay .delete .delete{color:#800}
.CodeRay .change .change{color:#66f}
.CodeRay .head .head{color:#f4f}
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
a {
text-decoration: none;
}
p {
line-height: 1.8;
}
.top-bar .toggle-topbar span,
.fa-ul a {
font-family:"Open Sans","DejaVu Sans",sans-serif;
}
.top-bar-section {
max-width: 100rem !important;
}
@media only screen and (min-width: 40.063em) {
.top-bar {
height: auto;
min-height: 2.8125rem;
}
}
.top-bar .top-bar-title {
padding-left: 10px;
}
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width">
<title>Reproducible LEDE ?</title>
<link rel="stylesheet" href="foundation.css"/>
<link rel="stylesheet" href="font-awesome.css"/>
<link rel="stylesheet" href="coderay.css"/>
<link rel="stylesheet" href="asciidoctor.css"/>
<link rel="stylesheet" href="lede.css"/>
</head>
<body>
<div id="content">
<pre>
</pre>
</div><div id="main-content">
<h1>LEDE - <em>reproducible</em> wireless freedom!</h1>
</div>
<p>
<ul>
<li><a href="lede_ar71xx.html">ar71xx</a></li>
<li><a href="lede_ramips.html">ramips</a></li>
<li><a href="lede_x86.html">x86</a></li>
</ul>
</p>
</div>
</body></html>