README 9.02 KB
Newer Older
1
2
3
4
5
About jenkins.debian.net
========================
:Author:           Holger Levsen
:Authorinitials:   holger
:EMail:            holger@layer-acht.org
6
:Status:           working, in progress
7
8
9
10
11
12
:lang:             en
:Doctype:          article
:Licence:	   GPLv2

== About jenkins.debian.net

13
https://jenkins.debian.net is a tool for automated quality monitoring of Debian. It is *work in progress* despite being in existence since October 15th 2012.
Holger Levsen's avatar
Holger Levsen committed
14

15
Get the source by running `git clone git://git.debian.org/git/qa/jenkins.debian.net.git`. It's all in there, no (relevant) manual setup has been done besides what's in this git repository. (The irrelevant bits are some very simple configuration files containing passwords.)
Holger Levsen's avatar
Holger Levsen committed
16

17
The (virtualized) hardware is sponsored since 2012 by http://www.profitbricks.com - currently it's using 15 cores and 50 GB memory, thanks a lot!
Holger Levsen's avatar
Holger Levsen committed
18

Holger Levsen's avatar
Holger Levsen committed
19
Some stats are available using link:https://jenkins.debian.net/munin/jenkins-month.html[munin-plugins for jenkins].
20

21
22
If you notice some jobs has problems and you want to find out why, read <<debug,debug certain jobs>> to learn how to do this locally.

23
24
25
26
== Notifications

There are two types of notifications being used: email and IRC. At the end of each builds console log it says to where notifcations has been send. An adress of the form 'jenkins-foo' means an IRC notification has been send to the #foo IRC channel.

27
28
== Jobs being run

29
There are over 400 jobs being run currently. If you can think of ways to improve the usefulness of certain jobs, please do give feedback!
30

31
=== g-i-installation jobs
Holger Levsen's avatar
Holger Levsen committed
32

33
34
Installation tests with g-i, the graphical version of d-i, the debian-installer.

Holger Levsen's avatar
update    
Holger Levsen committed
35
36
* 'g-i-installation_debian_sid_daily-rescue'
** boot of rescue system with daily build sid image
37

38
* 'g-i-installation_debian_sid_daily-lxde' and '-xfce' and '-kfreebsd' and '-hurd'
39
** sid installation of Xfce/LXDE desktop with daily build sid image
Holger Levsen's avatar
update    
Holger Levsen committed
40

Holger Levsen's avatar
Holger Levsen committed
41
42
43
44
45
* 'g-i-installation_debian_jessie_lxde','-xfce','-kde' and '-gnome' and '-kfreebsd'
** jessie installation of Xfce/LXDE/KDE desktop and kfreebsd install with weekly build jessie image

* 'g-i-installation_debian_wheezy_lxde','-xfce','-kde' and '-gnome' and '-kfreebsd'
** wheezy installation of Xfce/LXDE/KDE desktop and kfreebsd install with wheezy release image
46

Holger Levsen's avatar
update    
Holger Levsen committed
47
* 'g-i-installation_debian_squeeze_xfce'
Holger Levsen's avatar
Holger Levsen committed
48
** squeeze installation of Xfce desktop with squeeze image
Holger Levsen's avatar
update    
Holger Levsen committed
49

Holger Levsen's avatar
Holger Levsen committed
50
* Debian Edu installation tests of specific profiles and suites:
51
** 'g-i-installation_debian-edu_squeeze-test_standalone'
52
53
54
** 'g-i-installation_debian-edu_squeeze-test_workstation'
** 'g-i-installation_debian-edu_squeeze-test_combi-server'
** 'g-i-installation_debian-edu_wheezy-test_standalone'
55
** 'g-i-installation_debian-edu_wheezy-test_workstation'
56
** 'g-i-installation_debian-edu_wheezy-test_combi-server'
57
58
59

=== debian-installer jobs

Holger Levsen's avatar
Holger Levsen committed
60
* 'd_i_build_$source_package'
Holger Levsen's avatar
Holger Levsen committed
61
62
** there is one job for each git repo referred to in http://anonscm.debian.org/viewvc/d-i/trunk/.mrconfig?view=co
** each job pdebuilds the master branch of its git repo on every git push in a sid environment. (If the architecture(s) specified in debian/control are not amd64,all or any the build exits cleanly.)
63
** while these jobs are triggered on commits, the SCM is only polled every 6min to see if there are new commits.
Holger Levsen's avatar
Holger Levsen committed
64

Holger Levsen's avatar
Holger Levsen committed
65
* 'd_i_manual'
66
** builds the full installation-guide package with pdebuild in sid on every commit to svn://anonscm.debian.org/svn/d-i/ matching suitable patterns.
67
** while this job is triggered on commits, the SCM is only polled every 15min to see if there are new commits.
Holger Levsen's avatar
Holger Levsen committed
68

Holger Levsen's avatar
Holger Levsen committed
69
* 'd_i_manual_$language_html'
Holger Levsen's avatar
Holger Levsen committed
70
** builds a language (on wheezy) on every commit of svn/trunk/manual/$LANG with `make languages=$LANG architectures=amd64 formats=html`.
71
** while these jobs are triggered on commits, the SCM is only polled every 15min to see if there are new commits.
Holger Levsen's avatar
Holger Levsen committed
72
** on successful build, 'd_i_manual_$lang_pdf' is triggered.
Holger Levsen's avatar
Holger Levsen committed
73

Holger Levsen's avatar
Holger Levsen committed
74
* 'd_i_parse_build_logs' - parses logs from http://d-i.debian.org/daily-images/build-logs.html daily, to give them a bit more exposure.
75
** this job is run daily.
Holger Levsen's avatar
Holger Levsen committed
76

77
=== chroot-installation jobs
78

79
80
Installation tests inside chroot environments.

81
* 'chroot-installation_housekeeping_$distro':
82
** make sure chroots have been cleaned up properly
83
84
85
** runs daily at 05:00 UTC and triggers the $distro specific bootstrap job on success
** wheezy is only triggered on the 4th day and 18th of each month (as it was released on the 4th)
** squeeze only on the 25th of each month
86
87
* $distro-bootstrap jobs:
** just `debootstrap $distro` (install a base Debian distribution $distro)
88
** there is one job for *sid*, one for *wheezy* and one for *jessie*: 'chroot-installation_sid_bootstrap', 'chroot-installation_wheezy_bootstrap' and 'chroot-installation_jessie_bootstrap'
89
* plus there is 'chroot-installation_squeeze_bootstrap_upgrade_to_wheezy', which bootstraps *squeeze* and upgrades that to *wheezy*
90
91
** on successful run of the bootstrap job, six $distro-install(+upgrade) jobs are triggered.

92
93
* FIXME: describe upgrade jobs here too

94
95
* $distro-install jobs (and $distro-install+upgrade jobs):
** `debootstrap $distro`, install a *$set_of_packages* (and upgrade to *$2nd_distro*)
Holger Levsen's avatar
Holger Levsen committed
96
** these $set_of_packages exist: 'gnome', 'kde', 'kde-full', 'lxde', 'xfc', 'full_desktop' (all five desktops plus `vlc evince iceweasel chromium cups build-essential devscripts wine texlive-full asciidoc vim emacs` and (`libreoffice virt-manager mplayer2` (jessie/sid) or `openoffice virtualbox mplayer` (squeeze/wheezy)) and 'develop'
97
*** install is done with `apt-get install`, except for 'develop' where `apt-get build-dep` is used to install the build dependencies of these packages.
Holger Levsen's avatar
Holger Levsen committed
98

99
=== lintian jobs
100

101
102
103
104
105
106
107
108
109
* FIXME: describe lintian jobs

=== haskell jobs

* FIXME: describe haskell jobs

=== rebootstrap jobs

* FIXME: describe rebootstrap jobs
110

111
112
=== reproducible jobs

113
114
115
116
117
* Several jobs are being to test packages whether they can be build reproducible or not. The result is the website http://reproducible.debian.net which is actually just a collection of html files being served from this host.

* There are three builder job, alpha+beta+gamma which are each started every 2min., and there is a scheduler jobs scheduling builds for them. Several other jobs exist to build the html pages.

* The reproducible build jobs use their own pbuilder base.tgz, which has the reproducible apt repo added. And there another sid schroot is being used, where debbindiff (which has lots of depends installed) is run from
118

119
* Manual rescheduling of packages is easy, if you have console access. (Hint: you don't but someone on #debian-reproducible will have.) This is how to:
120
121

----
122
jenkins@jenkins:~$ /srv/jenkins/bin/reproducible_schedule_on_demand.sh package1 package2 abcde
123
----
124

125
126
127
128
129
130
* blacklisting packages can be done similarily, optionally list reaons in 'notes.git/packages.yml'

----
jenkins@jenkins:~$ /srv/jenkins/bin/reproducible_blacklist.sh package1
----

131
132
133
=== edu-packages jobs

* These jobs just build the master branch of the GIT repository of six Debian Edu packages in a sid environment on every GIT push.
134

135
136
137
138
139
=== dvswitch jobs

* FIXME: describe dvswitch jobs


140
141
=== webcheck jobs

142
* three jobs for running webcheck against www.debian.org, www.debian.org/News and www.debian.org/security
143
144
145
* several webcheck jobs for running again various debconf.org websites

These jobs are only run on the first day of each month.
146

Holger Levsen's avatar
Holger Levsen committed
147
148
149
150
=== self jobs

These are jobs for making sure jenkins.debian.net is running smoothly.

151
152
153
154
155
[[debug]]
== Debugging certain jobs

*FIXME*: To debug certain jobs, a jenkins setup is actually not needed. Make this easy and describe here.

156
*This is work in progress too, but mostly just needs documentation!* In principle the shell commands from the various jobs should run on any Debian system just fine. Please use a test system though, as all your data might be eaten.
157

158
* Build this source as a debian package and install the jenkins.d.n-debug package and all it's recommends.
159
160
161
162
163
164
165
166
167
168

=== Debugging g-i-installation jobs

as user run:

----
mkdir workspace ; cd workspace # test-script should check if they are in a directory called workspace and if not, mkdir it and cd into it.
/srv/jenkins/bin/cd_tester.sh 2 squeeze-test-debian-edu-standalone 20 http://ftp.skolelinux.org/cd-squeeze-test-amd64-i386-netinst/debian-edu-amd64-i386-NETINST-1.iso

----
Holger Levsen's avatar
Holger Levsen committed
169
FIXME: explain how to how to debug any job. Point this out at the end of any log.
170
171


172
TODO: svn/git using test-scripts need to learn checkout out - currently this is always done by jenkins.
173

Holger Levsen's avatar
Holger Levsen committed
174
175
176
177
=== Feedback

I love to get feedback on this! Please send an email to debian-qa@lists.debian.org or just join #debian-qa on irc.debian.org to report bugs or give other feedback there! 

178
179
180
181
==== Self signed SSL certificate

Yes, I know, there's a self signed certificate being used. But that's better than plain http anyway and if you provide a better certificate I will very likely happily take it.

Holger Levsen's avatar
Holger Levsen committed
182
183
=== Setup

Holger Levsen's avatar
Holger Levsen committed
184
See link:https://jenkins.debian.net/userContent/setup.html[INSTALL].
Holger Levsen's avatar
Holger Levsen committed
185
186
187

=== ToDo

Holger Levsen's avatar
Holger Levsen committed
188
There is still a lot of work left, check the current link:https://jenkins.debian.net/userContent/todo.html[ToDo list].
Holger Levsen's avatar
Holger Levsen committed
189
190


191
== License
192

193
* GPLv2, see link:http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/LICENSE[LICENSE].
194

Holger Levsen's avatar
Holger Levsen committed
195
// vim: set filetype=asciidoc: