Commit 34a9e2bb authored by Holger Levsen's avatar Holger Levsen
Browse files

reproducible F-Droid: move jobs from profitbricks7 to osuosl168 



Signed-off-by: default avatarHolger Levsen <holger@layer-acht.org>
parent 01ade2fe

THANKS.head

+5 −6
Original line number Diff line number Diff line
@@ -7,8 +7,8 @@ 
 ** 15 cores and 48 GB memory for profitbricks-build1-amd64.debian.net used for building amd64 Debian packages for tests.reproducible-builds.org (t.r-b.o)

 ** 10 cores and 36 GB memory for profitbricks-build2-i386.debian.net used for building i386

Debian packages for t.r-b.o

 ** 12 cores and 24 GB memory for profitbricks-build3-amd64.debian.net used for building OpenWrt, coreboot, Fedora, Archlinux and NetBSD for t.r-b.o

 ** 11 cores and 24 GB memory for profitbricks-build4-amd64.debian.net used for building OpenWrt, coreboot, Fedora, Archlinux and NetBSD for t.r-b.o, running in the future

 ** 12 cores and 24 GB memory for profitbricks-build3-amd64.debian.net used for building OpenWrt, coreboot, Fedora and NetBSD for t.r-b.o

 ** 11 cores and 24 GB memory for profitbricks-build4-amd64.debian.net used for building OpenWrt, coreboot, Fedora and NetBSD for t.r-b.o, running in the future

 ** 16 cores and 48 GB memory for profitbricks-build5-amd64.debian.net used for building amd64 Debian packages for t.r-b.o, running in the future

 ** 9 cores and 36 GB memory for profitbricks-build6-i386.debian.net used for building i386 Debian packages for t.r-b.o, running in the future

 ** 15 cores and 48 GB memory for profitbricks-build11-amd64.debian.net used for building amd64 Debian packages for tests.reproducible-builds.org (t.r-b.o)
@@ -16,7 +16,6 @@ Debian packages for t.r-b.o 
Debian packages for t.r-b.o

 ** 16 cores and 48 GB memory for profitbricks-build15-amd64.debian.net used for building amd64 Debian packages for t.r-b.o, running in the future

 ** 9 cores and 36 GB memory for profitbricks-build16-i386.debian.net sed for building i386 Debian packages for t.r-b.o, running in the future

 ** 2 cores and 8 GB memory for profitbricks-build7-amd64.debian.net used for building F-Droid for t.r-b.o

 ** 5 cores and 10 GB memory for profitbricks-build9-amd64.debian.net used for rebootstrap jobs

 ** 4 cores and 12 GB memory for profitbricks-build10-amd64.debian.net used for chroot-installation jobs

 ** 9 cores and 19 GB memory for freebsd-jenkins.debian.net (also running on Profitbricks virtual hardware), used for building FreeBSD for t.r-b.o
@@ -40,9 +39,9 @@ Debian packages for t.r-b.o 
 * link:https://letsencrypt.org[Let's encrypt] provides free of charge SSL certificates for jenkins.debian.net, reproducible.debian.net and tests.reproducible-builds.org.

 * In December 2018 were given access to eight nodes which were donated by Facebook to the GCC Compile Farm project and are now hosted by OSUOSL:

  ** 32 cores with 144 GB memory for osuosl167-amd64.debian.net

  ** 32 cores with 144 GB memory for osuosl168-amd64.debian.net

  ** 32 cores with 144 GB memory for osuosl169-amd64.debian.net

  ** 32 cores with 144 GB memory for osuosl170-amd64.debian.net

  ** 32 cores with 144 GB memory for osuosl168-amd64.debian.net used for building F-Droid for t.r-b.o

  ** 32 cores with 144 GB memory for osuosl169-amd64.debian.net used for building Arch Linux for t.r-b.o

  ** 32 cores with 144 GB memory for osuosl170-amd64.debian.net used for building Arch Linux for t.r-b.o

  ** 32 cores with 144 GB memory for osuosl171-amd64.debian.net

  ** 32 cores with 144 GB memory for osuosl172-amd64.debian.net

  ** 32 cores with 144 GB memory for osuosl173-amd64.debian.net

TODO

+7 −14
Original line number Diff line number Diff line
@@ -55,7 +55,6 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian 
* teach bin/chroot-*.sh and bin/d-i_build.sh how to nicely deal with network problems… (as both reproducible_build.sh and schroot-create.sh do)

* use static IP for the nodes (h01ger)

* use vmdebootstrap where applicable

* Tango Icons are gone: #824477 - update footer once this bug is fixed

* this is outdated with salsa: add to all git post-receive hooks: `curl -s "https://jenkins.debian.net/git/notifyCommit?url=https://salsa.debian.org/installer-team/$(basename $PWD .git)"` which will trigger jenkins to pull (check) that git repo…

* salsa stuff:

** triggers for d-i repos are missing
@@ -63,27 +62,21 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian 
=== new nodes at OSUOSL



* use those nodes, preliminary plan:

** osuosl167: squid (only, atm)

** osuosl168: fdroid

** osuosl169: archlinux

** osuosl170: archlinux (future)

** done: osuosl167: squid (only, atm)

** done: osuosl168: fdroid

*** shutdown pb7

*** inform hans

** done: osuosl169: archlinux

** done: osuosl170: archlinux (future)

** osuosl171: openwrt, coreboot, netbsd, opensuse

** osuosl172: openwrt, coreboot, netbsd, opensuse (future)

** osuosl173: debian

** osuosl174: debian (future)

* tmpfs, setup /srv/workspace as needed

* make nodes run in the future

* add to README 

* probably add links to OSUOSL in THANKS and definitly explain there usage

* add link to OSUOSL in THANKS and explain usage

* drop 'setup in progress' from r_html_nodes_info

* document how to contact OSUOSL

* blog post when done



=== ToDo for improving disk space



* make live-build jobs work again or remove them

* make sure the live-build jobs clean up /srv/live-build/results/*iso once they are done. thats 8gb wasted.



==== proper backup



* postponed til we run on .debian.org

bin/common-functions.sh

+1 −4
Original line number Diff line number Diff line 
#!/bin/bash

# vim: set noexpandtab:



# Copyright 2014-2018 Holger Levsen <holger@layer-acht.org>

# Copyright 2014-2019 Holger Levsen <holger@layer-acht.org>

#         © 2018      Mattia Rizzolo <mattia@mapreri.org>

# released under the GPLv=2
@@ -147,9 +147,6 @@ else 
		profitbricks-build4*|profitbricks-build5*|profitbricks-build6*|profitbricks-build10*|profitbricks-build15*|profitbricks-build16*)

			# pb datacenter in frankfurt uses pb10 as proxy:

			export http_proxy="http://85.184.249.68:3128" ;;

		profitbricks-build7*)

			# let pb7 use its own proxy for the fdroid stuff:

			export http_proxy="http://127.0.0.1:3128" ;;

		osuosl*)

			# all nodes at OSUOSL use osuosl167 as proxy:

			export http_proxy="http://10.6.5.46:3128" ;;

bin/reproducible_html_nodes_info.sh

+3 −6
Original line number Diff line number Diff line
@@ -19,9 +19,9 @@ explain_nodes() { 
		jenkins)	;;

		profitbricks3)	write_page "<br /><small>(r-b OpenWrt, NetBSD, Coreboot builds)</small>" ;;

		profitbricks4)	write_page "<br /><small>(r-b OpenWrt, NetBSD, Coreboot builds)</small>" ;;

		profitbricks7)	write_page "<br /><small>(r-b F-Droid builds)</small>" ;;

		profitbricks9)	write_page "<br /><small>(jenkins.d.n rebootstrap jobs)</small>" ;;

		profitbricks10)	write_page "<br /><small>(jenkins.d.n chroot-installation jobs)</small>" ;;

		osuosl168)	write_page "<br /><small>(r-b F-Droid builds)</small>" ;;

		osuosl169)	write_page "<br /><small>(r-b Archlinux builds)</small>" ;;

		osuosl170)	write_page "<br /><small>(r-b Archlinux builds)</small>" ;;

		osuosl*)	write_page "<br /><small>(TBD - setup in progress)</small>" ;;
@@ -66,10 +66,9 @@ build_nodes_health_page() { 
				# pb4	r-b archlinux

				REPRODUCIBLE_NODES="$REPRODUCIBLE_NODES profitbricks-build3-amd64.debian.net profitbricks-build4-amd64.debian.net"

			elif [ "$NODE" = "profitbricks-build6-i386.debian.net" ] ; then

				# pb7 	r-b fdroid build host

				# pb9 	rebootstrap jobs

				# pb10	chroot jobs

				REPRODUCIBLE_NODES="$REPRODUCIBLE_NODES profitbricks-build7-amd64.debian.net profitbricks-build9-amd64.debian.net profitbricks-build10-amd64.debian.net"

				REPRODUCIBLE_NODES="$REPRODUCIBLE_NODES profitbricks-build9-amd64.debian.net profitbricks-build10-amd64.debian.net"

			fi

		done

		for NODE in $REPRODUCIBLE_NODES ; do
@@ -115,9 +114,9 @@ build_nodes_health_page() { 
				jenkins)	write_page "<td></td>" ;;

				profitbricks3)	write_page "<td></td>" ;;

				profitbricks4)	write_page "<td></td>" ;;

				profitbricks7)	write_page "<td></td>" ;;

				profitbricks9)	write_page "<td></td>" ;;

				profitbricks10)	write_page "<td></td>" ;;

				osuosl*)	write_page "<td></td>" ;;

				*)		write_page "<td>"

						SHORTNAME=$(echo $NODE | cut -d '.' -f1)

						for WORKER in $(grep "${ARCH}_" /srv/jenkins/bin/reproducible_build_service.sh | grep -v \# |grep $SHORTNAME | cut -d ')' -f1) ; do
@@ -133,7 +132,6 @@ build_nodes_health_page() { 
					jenkins)	write_page "<td></td>" ;;

					profitbricks3)	write_page "<td></td>" ;;

					profitbricks4)	write_page "<td></td>" ;;

					profitbricks7)	write_page "<td></td>" ;;

					profitbricks9)	write_page "<td></td>" ;;

					profitbricks10)	write_page "<td></td>" ;;

					osuosl*)	write_page "<td></td>" ;;
@@ -215,7 +213,6 @@ build_graph_overview_pages() { 
							jenkins)	write_page "<td></td>" ; continue ;;

							profitbricks3)	write_page "<td></td>" ; continue ;;

							profitbricks4)	write_page "<td></td>" ; continue ;;

							profitbricks7)	write_page "<td></td>" ; continue ;;

							profitbricks9)	write_page "<td></td>" ; continue ;;

							profitbricks10)	write_page "<td></td>" ; continue ;;

							osuosl*)	write_page "<td></td>" ; continue ;;

deploy_jdn

+0 −2
Original line number Diff line number Diff line
@@ -51,7 +51,6 @@ profitbricks-build1-amd64.debian.net 
profitbricks-build3-amd64.debian.net

profitbricks-build4-amd64.debian.net

profitbricks-build5-amd64.debian.net

profitbricks-build7-amd64.debian.net

profitbricks-build9-amd64.debian.net

profitbricks-build10-amd64.debian.net

profitbricks-build11-amd64.debian.net
@@ -144,7 +143,6 @@ elif [ "$1" = "jenkins" ] ; then 
			4|pb4)	 HOSTS+=(profitbricks-build4-amd64.debian.net) ;;

			5|pb5)	 HOSTS+=(profitbricks-build5-amd64.debian.net) ;;

			6|pb6)	 HOSTS+=(profitbricks-build6-i386.debian.net) ;;

			7|pb7)	 HOSTS+=(profitbricks-build7-amd64.debian.net) ;;

			9|pb9)	 HOSTS+=(profitbricks-build9-amd64.debian.net) ;;

			10|pb10) HOSTS+=(profitbricks-build10-amd64.debian.net) ;;

			11|pb11) HOSTS+=(profitbricks-build11-amd64.debian.net) ;;