README 7.81 KB
Newer Older
1
2
3
4
5
About jenkins.debian.net
========================
:Author:           Holger Levsen
:Authorinitials:   holger
:EMail:            holger@layer-acht.org
6
:Status:           working, in progress
7
8
9
10
11
12
:lang:             en
:Doctype:          article
:Licence:	   GPLv2

== About jenkins.debian.net

13
https://jenkins.debian.net is a tool for automated quality monitoring of Debian. It is *work in progress* despite being in existence since October 15th 2012.
Holger Levsen's avatar
Holger Levsen committed
14

15
Get the source by running `git clone git://git.debian.org/git/qa/jenkins.debian.net.git`. It's all in there, no (relevant) manual setup has been done besides what's in this git repository. (The irrelevant bits are some very simple configuration files containing passwords.)
Holger Levsen's avatar
Holger Levsen committed
16

17
The (virtualized) hardware is sponsored since 2012 by http://www.profitbricks.com - currently it's using 15 cores and 50 GB memory, thanks a lot!
Holger Levsen's avatar
Holger Levsen committed
18

Holger Levsen's avatar
Holger Levsen committed
19
Some stats are available using link:https://jenkins.debian.net/munin/jenkins-month.html[munin-plugins for jenkins].
20

21
22
If you notice some jobs has problems and you want to find out why, read <<debug,debug certain jobs>> to learn how to do this locally.

23
24
== Jobs being run

25
If you can think of ways to improve the usefulness of certain jobs, please do give feedback! Remember, this is still pretty much work in progress. 
26

27
=== g-i-installation jobs
Holger Levsen's avatar
Holger Levsen committed
28

29
30
Installation tests with g-i, the graphical version of d-i, the debian-installer.

Holger Levsen's avatar
update    
Holger Levsen committed
31
32
* 'g-i-installation_debian_sid_daily-rescue'
** boot of rescue system with daily build sid image
33

34
* 'g-i-installation_debian_sid_daily-lxde' and '-xfce' and '-kfreebsd' and '-hurd'
35
** sid installation of Xfce/LXDE desktop with daily build sid image
Holger Levsen's avatar
update    
Holger Levsen committed
36

Holger Levsen's avatar
Holger Levsen committed
37
38
39
40
41
* 'g-i-installation_debian_jessie_lxde','-xfce','-kde' and '-gnome' and '-kfreebsd'
** jessie installation of Xfce/LXDE/KDE desktop and kfreebsd install with weekly build jessie image

* 'g-i-installation_debian_wheezy_lxde','-xfce','-kde' and '-gnome' and '-kfreebsd'
** wheezy installation of Xfce/LXDE/KDE desktop and kfreebsd install with wheezy release image
42

Holger Levsen's avatar
update    
Holger Levsen committed
43
* 'g-i-installation_debian_squeeze_xfce'
Holger Levsen's avatar
Holger Levsen committed
44
** squeeze installation of Xfce desktop with squeeze image
Holger Levsen's avatar
update    
Holger Levsen committed
45

Holger Levsen's avatar
Holger Levsen committed
46
* Debian Edu installation tests of specific profiles and suites:
47
** 'g-i-installation_debian-edu_squeeze-test_standalone'
48
49
50
** 'g-i-installation_debian-edu_squeeze-test_workstation'
** 'g-i-installation_debian-edu_squeeze-test_combi-server'
** 'g-i-installation_debian-edu_wheezy-test_standalone'
51
** 'g-i-installation_debian-edu_wheezy-test_workstation'
52
** 'g-i-installation_debian-edu_wheezy-test_combi-server'
53
54
55

=== debian-installer jobs

Holger Levsen's avatar
Holger Levsen committed
56
* 'd_i_build_$source_package'
Holger Levsen's avatar
Holger Levsen committed
57
58
** there is one job for each git repo referred to in http://anonscm.debian.org/viewvc/d-i/trunk/.mrconfig?view=co
** each job pdebuilds the master branch of its git repo on every git push in a sid environment. (If the architecture(s) specified in debian/control are not amd64,all or any the build exits cleanly.)
59
** while these jobs are triggered on commits, the SCM is only polled every 6min to see if there are new commits.
Holger Levsen's avatar
Holger Levsen committed
60

Holger Levsen's avatar
Holger Levsen committed
61
* 'd_i_manual'
62
** builds the full installation-guide package with pdebuild in sid on every commit to svn://anonscm.debian.org/svn/d-i/ matching suitable patterns.
63
** while this job is triggered on commits, the SCM is only polled every 15min to see if there are new commits.
Holger Levsen's avatar
Holger Levsen committed
64

Holger Levsen's avatar
Holger Levsen committed
65
* 'd_i_manual_$language_html'
Holger Levsen's avatar
Holger Levsen committed
66
** builds a language (on wheezy) on every commit of svn/trunk/manual/$LANG with `make languages=$LANG architectures=amd64 formats=html`.
67
** while these jobs are triggered on commits, the SCM is only polled every 15min to see if there are new commits.
Holger Levsen's avatar
Holger Levsen committed
68
** on successful build, 'd_i_manual_$lang_pdf' is triggered.
Holger Levsen's avatar
Holger Levsen committed
69

Holger Levsen's avatar
Holger Levsen committed
70
* 'd_i_parse_build_logs' - parses logs from http://d-i.debian.org/daily-images/build-logs.html daily, to give them a bit more exposure.
71
** this job is run daily.
Holger Levsen's avatar
Holger Levsen committed
72

73
=== chroot-installation jobs
74

75
76
Installation tests inside chroot environments.

77
* 'chroot-installation_housekeeping_$distro':
78
** make sure chroots have been cleaned up properly
79
80
81
** runs daily at 05:00 UTC and triggers the $distro specific bootstrap job on success
** wheezy is only triggered on the 4th day and 18th of each month (as it was released on the 4th)
** squeeze only on the 25th of each month
82
83
* $distro-bootstrap jobs:
** just `debootstrap $distro` (install a base Debian distribution $distro)
84
** there is one job for *sid*, one for *wheezy* and one for *jessie*: 'chroot-installation_sid_bootstrap', 'chroot-installation_wheezy_bootstrap' and 'chroot-installation_jessie_bootstrap'
85
* plus there is 'chroot-installation_squeeze_bootstrap_upgrade_to_wheezy', which bootstraps *squeeze* and upgrades that to *wheezy*
86
87
** on successful run of the bootstrap job, six $distro-install(+upgrade) jobs are triggered.

88
89
* FIXME: describe upgrade jobs here too

90
91
* $distro-install jobs (and $distro-install+upgrade jobs):
** `debootstrap $distro`, install a *$set_of_packages* (and upgrade to *$2nd_distro*)
Holger Levsen's avatar
Holger Levsen committed
92
** these $set_of_packages exist: 'gnome', 'kde', 'kde-full', 'lxde', 'xfc', 'full_desktop' (all five desktops plus `vlc evince iceweasel chromium cups build-essential devscripts wine texlive-full asciidoc vim emacs` and (`libreoffice virt-manager mplayer2` (jessie/sid) or `openoffice virtualbox mplayer` (squeeze/wheezy)) and 'develop'
93
*** install is done with `apt-get install`, except for 'develop' where `apt-get build-dep` is used to install the build dependencies of these packages.
Holger Levsen's avatar
Holger Levsen committed
94

95
=== lintian jobs
96

97
98
99
100
101
102
103
104
105
* FIXME: describe lintian jobs

=== haskell jobs

* FIXME: describe haskell jobs

=== rebootstrap jobs

* FIXME: describe rebootstrap jobs
106

107
108
109
110
=== reproducible jobs

* FIXME: describe reproducible jobs

111
* manual rescheduling is easy, if you have console access. (Hint: you don't but someone on #debian-reproducible will have.) This is how to:
112
113

----
114
jenkins@jenkins:~$ /srv/jenkins/bin/reproducible_schedule_on_demand.sh package1 package2 abcde
115
----
116

117
118
119
120
121
122
123
* blacklisting packages can be done similarily, optionally list reaons in 'notes.git/packages.yml'

----
jenkins@jenkins:~$ /srv/jenkins/bin/reproducible_blacklist.sh package1
----


124
125
126
127
128
=== dvswitch jobs

* FIXME: describe dvswitch jobs


129
130
=== webcheck jobs

131
* three jobs for running webcheck against www.debian.org, www.debian.org/News and www.debian.org/security
132
133
134
* several webcheck jobs for running again various debconf.org websites

These jobs are only run on the first day of each month.
135

Holger Levsen's avatar
Holger Levsen committed
136
137
138
139
=== self jobs

These are jobs for making sure jenkins.debian.net is running smoothly.

140
141
142
143
144
[[debug]]
== Debugging certain jobs

*FIXME*: To debug certain jobs, a jenkins setup is actually not needed. Make this easy and describe here.

145
*This is work in progress too, but mostly just needs documentation!* In principle the shell commands from the various jobs should run on any Debian system just fine. Please use a test system though, as all your data might be eaten.
146

147
* Build this source as a debian package and install the jenkins.d.n-debug package and all it's recommends.
148
149
150
151
152
153
154
155
156
157

=== Debugging g-i-installation jobs

as user run:

----
mkdir workspace ; cd workspace # test-script should check if they are in a directory called workspace and if not, mkdir it and cd into it.
/srv/jenkins/bin/cd_tester.sh 2 squeeze-test-debian-edu-standalone 20 http://ftp.skolelinux.org/cd-squeeze-test-amd64-i386-netinst/debian-edu-amd64-i386-NETINST-1.iso

----
Holger Levsen's avatar
Holger Levsen committed
158
FIXME: explain how to how to debug any job. Point this out at the end of any log.
159
160


161
TODO: svn/git using test-scripts need to learn checkout out - currently this is always done by jenkins.
162

Holger Levsen's avatar
Holger Levsen committed
163
164
165
166
167
168
=== Feedback

I love to get feedback on this! Please send an email to debian-qa@lists.debian.org or just join #debian-qa on irc.debian.org to report bugs or give other feedback there! 

=== Setup

Holger Levsen's avatar
Holger Levsen committed
169
See link:https://jenkins.debian.net/userContent/setup.html[INSTALL].
Holger Levsen's avatar
Holger Levsen committed
170
171
172

=== ToDo

Holger Levsen's avatar
Holger Levsen committed
173
There is still a lot of work left, check the current link:https://jenkins.debian.net/userContent/todo.html[ToDo list].
Holger Levsen's avatar
Holger Levsen committed
174
175


176
== License
177

178
* GPLv2, see link:http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/LICENSE[LICENSE].
179

Holger Levsen's avatar
Holger Levsen committed
180
// vim: set filetype=asciidoc: