TODO 7.63 KB
Newer Older
1 2
ToDo for jenkins.debian.net
===========================
3 4 5
:Author:           Holger Levsen
:Authorinitials:   holger
:EMail:            holger@layer-acht.org
6
:Status:           working, in progress
7 8 9 10
:lang:             en
:Doctype:          article
:Licence:	   GPLv2

11 12
== About jenkins.debian.net

13
See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian.net"].
14

Holger Levsen's avatar
Holger Levsen committed
15 16 17 18
== Fix user submitted bugs

* See https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=jenkins;users=qa.debian.org@packages.debian.org

19 20
== General ToDo

21
* move /srv/jenkins.debian.net-scm-sync.git somewhere public?
22
* keep git+svn urls, and use commit hooks. also better for test setups of this setup as well as overall stability.
23
* fully automate backup and backup /var/lib/jenkins/jobs /var/lib/munin /var/log /root/ /var/lib/jenkins/reproducible.db too
Holger Levsen's avatar
Holger Levsen committed
24
* put kgb-client.conf in git and sed passwords from filesystem into it...
Holger Levsen's avatar
Holger Levsen committed
25
* push jenkins munin plugins upstream
26

27 28 29 30
== Debian Packaging related

* /usr/sbin/jenkins.debian.net-setup needs to be written
* what update-j.d.n.sh does, needs to be put elsewhere...
31
* debian/copyright is incorrect about some licenses: a.) the profitbricks+debian+jenkins logos b.) the preseeding files
32

33 34
=== jenkins-job-builder related

35
* get rid of some reduncacy in job-cfg/*.yaml
Holger Levsen's avatar
Holger Levsen committed
36 37
* this seems to be helpful: http://en.wikipedia.org/wiki/YAML#References (pyyaml which jenkins-job-builder uses supports them)
* FIXME: sometimes true and some True used in yaml files
38 39
* support for gitweb in jobs 
** needs clean merge of my patches to jenkins-job-builder first...
Holger Levsen's avatar
Holger Levsen committed
40
* cleanup my patches (eg add documentation) and send pull requests on github:
41 42
** publisher:logparse
** publisher:htmlpublisher
43
** publisher:imagegallery
44 45
** svn:scm
** properties: sidebar
Holger Levsen's avatar
Holger Levsen committed
46
* d-i and chroot housekeeping jobs should be kept 365 days
47

Holger Levsen's avatar
Holger Levsen committed
48
== Improve existing tests
Holger Levsen's avatar
Holger Levsen committed
49

Holger Levsen's avatar
Holger Levsen committed
50
=== d-i_manual*
51

52
* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
53
* svn:trunk/manual is all thats needed, not whole svn:trunk
54
* inform translators (and debian-boot for the package)
Holger Levsen's avatar
Holger Levsen committed
55 56 57

=== d-i_build*

Holger Levsen's avatar
Holger Levsen committed
58
* run scripts/digress/ ?
Holger Levsen's avatar
Holger Levsen committed
59
* bubulle wrote: "Another interesting target would be d-i builds *including non uploaded packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup).
60
* build branches? (which?)
Holger Levsen's avatar
Holger Levsen committed
61

62
=== chroot-installation_*
Holger Levsen's avatar
Holger Levsen committed
63 64

* inform debian-devel@
65
* "Hash Sum Mismatch" problem with squid, see https://jenkins.debian.net/job/chroot-installation_squeeze_install_developer_upgrade_to_wheezy/13/console
66
** pere says this is supposed to be fixed in squid3
67
* chroot-installation: only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare)
Holger Levsen's avatar
Holger Levsen committed
68
*  warn about transitional packages installed (on non-upgrades only)
Holger Levsen's avatar
Holger Levsen committed
69

Holger Levsen's avatar
Holger Levsen committed
70 71 72
----
<pabs> h01ger: how about all of the blends?
<h01ger> pabs, if you could give me concrete (meta-)package names to install, that would be great+very helpful
Holger Levsen's avatar
Holger Levsen committed
73 74
<h01ger> for ^education* and ^debian-edu* i can find them myself
<pabs> h01ger: hmm, doesn't seem to be easy to find that list, but here are a few: ezgo-* med-* science-* junior-* gis-*
Holger Levsen's avatar
Holger Levsen committed
75 76
----

Holger Levsen's avatar
Holger Levsen committed
77 78
=== webchecks*

Holger Levsen's avatar
Holger Levsen committed
79
* inform debian-www@ ?
80

81
=== g-i-installation_*:
82

83
* babelbox.git has a nice langlist
Holger Levsen's avatar
Holger Levsen committed
84
* $name-preseed.base -> sed .cfg (LANG)
85 86
* for edu: pick LANG from predefined list at random - if last build was not successful or unstable fall back to English
** these jobs would not need to do an install, just booting them in rescue mode is probably enough
Holger Levsen's avatar
Holger Levsen committed
87
* test moar images: debian wheezy+sid cd and dvd images too and use lxde cd for installing lxde + etc
Holger Levsen's avatar
Holger Levsen committed
88 89 90
* use jigdo to download images - but no edu jigdo images.
* still havent got http://www.os-autoinst.org/ out of my mind

91 92
=== reproducible

Holger Levsen's avatar
Holger Levsen committed
93 94 95 96 97 98 99
* schedule packages by adding them to a table in the db, to make sure each package is only scheduled once.
** needs new table: sources_scheduled: name, schedule_date (older=will be scheduled sooner, so its possible to set schedule_date to 1970...
** schedule (once every hour) if there are less then 300 packages scheduled:
*** schedule up to 200 new ones
*** then schedule up to 100 new versions (but not more than 200 in total)
** if thats less than 200 and in total there are less then 300 scheduled:
*** schedule up to 100 same versions, older than a week
100
*** schedule the oldest FTBFS packages - once every other scheduler run, which is run 24x a day
Holger Levsen's avatar
Holger Levsen committed
101 102 103 104 105 106 107 108 109 110 111 112
** another job: reschedule packages supplied by params (first those initital ones but also on demand, via cli or webgui)
** dont reschedule packages which 404ed or are blacklisted

* then have other three runner job constantly picking up scheduled packages
** needs new table: runner_status, id, status (RUNNING, IDLE, STOPPED, START), timestamp
** if there are more than 30 packages scheduled, take 10, if more than 10, take 3, else 1. if 0, sleep 15m
** they write a timestamp to runner_status on each started package, so hanging builds can be detected
** have another job which kills them and marks them killed in db
** setup script shall cleanup workspaces from workers
*** killer job is run before setup
*** scheduler and builder jobs are started after

113
* bugs
114
** add note about notes on every page which has them...
115 116 117 118

* CSS/HTML
** use icons within index pages (refactor to get rid of that case statement, use array instead)

Holger Levsen's avatar
Holger Levsen committed
119
* watch: zephyr (debbindiff timeout?)
120
* watch: haskell-hsql-odbc (debbindiff error only in jenkins log, not in rbuild log
Holger Levsen's avatar
Holger Levsen committed
121
* watch: cxxtest: dbd failure should be in rbuild output!
122
* watch: android: no debbindiff output?
123
* watch: hurd-libfuse: should be not for us
124
* new view: issues (show all issues in one page and the packages which are affected.)
Holger Levsen's avatar
Holger Levsen committed
125 126 127
* generate .json for tracker.d.o
* reschedule all pkg tested before oct 06 00:00 UTC (we want to know if they generate .buildinfo files)
* reschedule all files with 0 length .rbuild.log files
128
* watch it build :)
129

Holger Levsen's avatar
Holger Levsen committed
130 131 132 133 134
=== Test Debian live

* daily build from sid
* test live images from http://live.debian.net/cdimage/release/current-next and .../current

Holger Levsen's avatar
Holger Levsen committed
135 136 137 138 139 140 141 142
== Add more tests

=== Test more Debian Edu related things

* manual - by language and full
* build a cd-image from all edu packages (even (and especially) UNRELEASED ones and do a g-i-installation test with it
** this probably better is done after the edu packages has been switched to git and when edu cd are (also?) build on pettersson.d.o
** also because it's probably better to do this with unmodified d-i first :-)
Holger Levsen's avatar
Holger Levsen committed
143
* test minimal and sugar profile, as well as gnome and lxde
Holger Levsen's avatar
Holger Levsen committed
144
* test dvd. test amd64 +i386 chroot setup.
145

146
=== Test tasks:
147

148
* test installations with (at least) more language tasks enabled, though tasksel doesn't work like this:
Holger Levsen's avatar
Holger Levsen committed
149

150
----
151 152 153 154
<h01ger> how do i install a task outside d-i?
<h01ger> what interesting tasks are there?
<jcristau> tasksel install <task>
<daemonkeeper> h01ger:  http://packages.debian.org/source/sid/tasksel
155
----
156

157
Help explaining how to test tasks (ie all the language tasks) very much welcome!
158

159 160
== Ideas...

161 162 163 164 165 166 167 168 169
=== Test co-installability

----
<jwilk> Install everything with priority >= optional? These packages are not supposed to conflict with each other. </wishful-thinking>
<h01ger> jwilk, there are tools to detect these package sets just based on debian/control information. no need to test this every day by installing them :)
<h01ger> running these tools daily OTOH...
<h01ger> http://coinst.irill.org/
----

170 171 172 173 174
=== Test them all

* build packages from all team repos on alioth with jenkins-debian-glue on team request (eg, via a .txt file in a git.repo) for specific branches (which shall also be automated, eg. to be able to only have squeeze+sid branches build, but not all other branches.)


Holger Levsen's avatar
Holger Levsen committed
175
// vim: set filetype=asciidoc: