Commit ca1925d9 authored by James Troup's avatar James Troup

Convert away from silly names

parent c06f4204
Assumptions
-----------
o Usernames do not contain ",". [julia]
o Version numbers do not contain "~" [logging, charisma, db_access.py, heidi, neve]
o Package names do not contain "~" [rene]
o Suites are case-independent in conf files, but forced lower case in use. [jenna]
o Components are case-sensitive. [jenna]
o Usernames do not contain ",". [dak import-users-from-passwd]
o Package names do not contain "~" [dak cruft-report]
o Suites are case-independent in conf files, but forced lower case in use. [dak make-suite-file-list]
o Components are case-sensitive. [dak make-suite-file-list]
o There's always source of some sort
o If you have a large archive, you have a lot of memory and don't mind
it being used. [jenna[, neve]]
it being used. [dak make-suite-file-list[, dak import-archive]]
[Very incomplete...]
Explanation of configuration files options for katie
====================================================
Explanation of configuration files options for dak
==================================================
DB
--
......@@ -14,7 +14,7 @@ Essential. List of database details, e.g.
| };
Name (required): The name of the PostgreSQL database which has been created
for katie.
for dak.
Host (required): The name of the host on which the database is located. If
the database is local, Host should be blank.
......@@ -33,7 +33,7 @@ Mandatory. List of directory locations, e.g.
| {
| Root "/org/ftp.debian.org/ftp/";
| Pool "/org/ftp.debian.org/ftp/pool/";
| Templates "/org/ftp.debian.org/katie/templates/";
| Templates "/org/ftp.debian.org/dak/templates/";
| PoolRoot "pool/";
| Override "/org/ftp.debian.org/scripts/override/";
| Lists "/org/ftp.debian.org/database/dists/";
......@@ -56,29 +56,30 @@ Mandatory. List of directory locations, e.g.
Root (required): Specifies the path of the root of the FTP archive.
Pool (required): This variable specifies the path of the pool directory.
Debian packages will be placed in the pool by kelly after they have been
accepted by jennifer.
Pool (required): This variable specifies the path of the pool
directory. Debian packages will be placed in the pool by 'dak
process-accepted' after they have been accepted by dak
process-unchecked.
Templates (required): katie sends various mails and uses templates from
Templates (required): dak sends various mails and uses templates from
this directory.
PoolRoot (required): This variable specifies the basename of the pool
directory.
Override (optional): This directory optionally stores override files (used
by denise).
by 'dak make-overrides').
Lists (optional): This directory holds file lists used by apt-ftparchive to
generate Packages and Sources files (used by jenna).
generate Packages and Sources files (used by 'dak make-suite-file-list').
Log (required): Log files are put in this directory.
Morgue (required): Removed files are moved there. The morgue has various
sub-directories, including (optionally) those defined by
Shania::MorgueSubDir and Rhona::MorgueSubDir.
Clean-Queues::MorgueSubDir and Clean-Suites::MorgueSubDir.
MorgueReject (required): if katie cannot move a rejected package to
MorgueReject (required): if dak cannot move a rejected package to
Dir::Queue::Reject, it will try to move it to the Dir::MorgueReject
directory located under Dir::Morgue.
......@@ -86,9 +87,9 @@ QueueBuild (optional): This variable is only relevant if any suites
are to be auto built, i.e. if Dinstall::QueueBuildSuites has any
values.
UrgencyLog (optional): If this directory is specified, kelly will store the
urgency value of each upload. This is mainly used for britney (the testing
script).
UrgencyLog (optional): If this directory is specified, 'dak
process-accepted' will store the urgency value of each upload. This
is mainly used for britney (the testing script).
Queue (required): This sub-tree defines important directories for the
incoming queue. The following variables have to be set: Accepted, Byhand
......@@ -130,28 +131,28 @@ Mandatory. List of all suites, e.g.
Announce (optional): controls where "Installed foo" mails are sent.
CodeName, Origin and Description (optional): This settings are used by ziyi
and put in the Release files.
CodeName, Origin and Description (optional): This settings are used by
'dak generate-releases' and put in the Release files.
OverrideCodeName (optional): used by Denise.
OverrideCodeName (optional): used by 'dak make-overrides'.
Priority (optional) determines which suite is used for the Maintainers file
as generated by charisma/da_mkmaintainers (highest wins).
as generated by 'dak make-maintainers' (highest wins).
CopyChanges (optional): if this variable is present it should be a path
into the archive (i.e. "Dir::RootDir"); any upload targeted for a suite
with this config option present will have the .changes file copied into
that path.
CopyKatie (optional): if this is present it should be an absolute path; any
CopyDotDak (optional): if this is present it should be an absolute path; any
upload targeted for a suite with this config option present will have the
.katie file copied into that path. This option is similar to CopyChanges
.dak file copied into that path. This option is similar to CopyChanges
and will most often be used with it; they're seperate because .changes
files are mirrored and .katie files aren't, so the paths will usually be
files are mirrored and .dak files aren't, so the paths will usually be
different.
There are more optional variables, such as VersionChecks. Please see
katie.conf for examples.
dak.conf for examples.
================================================================================
......@@ -210,7 +211,7 @@ Mandatory. List of dinstall options, e.g.:
| BugServer "bugs.debian.org";
| PackagesServer "packages.debian.org";
| TrackingServer "packages.qa.debian.org";
| LockFile "/org/ftp.debian.org/katie/lock";
| LockFile "/org/ftp.debian.org/dak/lock";
| Bcc "archive@ftp-master.debian.org";
| GroupOverrideFilename "override.group-maint";
| FutureTimeTravelGrace 28800; // 8 hours
......@@ -223,9 +224,10 @@ Mandatory. List of dinstall options, e.g.:
| };
PGPKeyring and GPGKeyring (required): filenames of the PGP and GnuPG
keyrings to be used by katie respectively.
keyrings to be used by dak respectively.
SigningKeyring (optional): this is the private keyring used by ziyi.
SigningKeyring (optional): this is the private keyring used by 'dak
generate-releases'.
SendmailCommand (required): command to call the MTA.
......@@ -235,16 +237,17 @@ as a script/daemon.
MyAdminAddress (required): used as a contact address in mails.
MyDistribution (required): this variable is used in emails sent out by
katie and others. It should indicate the name of the distribution.
dak and others. It should indicate the name of the distribution.
BugServer (required): is used by katie and melanie when closing bugs.
BugServer (required): is used by 'dak process-unchecked' and 'dak rm'
when closing bugs.
PackagesServer (requried): used by melanie when carbon-copying a bug close
mail to a package maintainer.
PackagesServer (requried): used by 'dak rm' when carbon-copying a bug
close mail to a package maintainer.
TrackingServer (optional): used by katie and melanie to send messages for
the maintainer also to an alias for people tracking a specific source
package.
TrackingServer (optional): used by 'dak process-unchecked' and 'dak
rm' to send messages for the maintainer also to an alias for people
tracking a specific source package.
LockFile (required): contains the filename of the lockfile used by dinstall
when in action mode (i.e. not using -n/--no-action).
......@@ -264,11 +267,11 @@ deciding whether or not to reject packages based on the file timestamp.
BXANotify (optional): a boolean (default: no); if true (Debian-specific)
BXA notification is sent. The template for the BXA notification is located
in Dir::Templates/lisa.bxa_notification and should be changed if this
in Dir::Templates/process-new.bxa_notification and should be changed if this
option is set.
OverrideDisparityCheck (optional): a boolean (default: no); if true,
jennifer compares an uploads section/priority with the overrides and whines
dak process-unchecked compares an uploads section/priority with the overrides and whines
at the maintainer if they differ.
CloseBugs (optional): a boolean (default: no); if true the automated bug
......@@ -285,7 +288,7 @@ __MAINTAINER_TO__ and __MAINTAINER_FROM__ variables in template mails.
Use with caution.
SkipTime (required): an integer value which is the number of seconds that a
file must be older than (via it's last modified timestamp) before jennifer
file must be older than (via it's last modified timestamp) before dak process-unchecked
will REJECT rather than SKIP the package.
================================================================================
......@@ -305,7 +308,7 @@ Mandatory. List of all archives, e.g.
| };
| };
OriginServer and PrimaryMirror (required): used melanie's bug closing mail
OriginServer and PrimaryMirror (required): used 'dak rm's bug closing mail
templates. The host name and it's OriginServer and Description are part of
the SQL database in the 'archive' table.
......@@ -428,7 +431,7 @@ Note that the archive value specified here must correspond to one defined
in Archive.
[Note: yes, this is horrible, it dates back to the original `import
the existent archive into the SQL Database' script (neve) and
the existent archive into the SQL Database' script ('dak import-archive') and
isn't otherwise used. It should be revisted at some stage.]
================================================================================
......@@ -455,7 +458,3 @@ This defines the valid and default urgency of an upload. If a package is
uploaded with an urgency not listed here, it will be rejected.
================================================================================
--
James Troup <james@nocrew.org>, Horsforth, Leeds
Wed, 26 Sep 2001 03:20:39 +0100
Notes
=====
o Please be careful: katie sends out lots of emails and if not
o Please be careful: dak sends out lots of emails and if not
configured properly will happily send them to lots of people who
probably didn't want those emails.
o Don't use katie.conf, apt.conf, cron.* etc. as starting points for
your own configuration files, they're the configuration files for
auric (aka ftp-master.debian.org) and are highly Debian specific.
Start from scratch and refer to the security.debian.org config files
(-security) as they're a better example for a private archive.
o Don't be put off by the names, see doc/README.names for a mapping of
name -> what the script does.
o Don't use the debian dak.conf, apt.conf, cron.* etc. as starting
points for your own configuration files, they're highly Debian
specific. Start from scratch and refer to the security.debian.org
config files (-security) as they're a better example for a private
archive.
What do all these scripts do?
=============================
......@@ -22,103 +19,103 @@ Generic and generally useful
o To process queue/:
* jennifer - processes queue/unchecked
* kelly - processes queue/accepted
* lisa - processes queue/new and queue/byhand
* dak process-unchecked - processes queue/unchecked
* dak process-accepted - move files from queue/accepted into the pool (and database)
* dak process-new - allows ftp administrator to processes queue/new and queue/byhand
o To generate indices files:
* jenna - generates file lists for apt-ftparchive and removes
obsolete packages from suites
* ziyi - generates Release
* dak make-suite-file-list - generates file lists for apt-ftparchive
and removes obsolete packages from
suites
* dak generate-releases - generates Release
o To clean things up:
* rhona - to remove obsolete files from the pool
* shania - to remove obsolete/stray files from the queue
* melanie - to remove package(s) from suite(s)
* alicia - to change individual override entries
* dak clean-suites - to remove obsolete files from the pool
* dak clean-queues - to remove obsolete/stray files from the queue
* dak rm - to remove package(s) from suite(s)
* dak override - to change individual override entries
o Information display:
* madison - shows information about package(s)
* helena - shows information about package(s) in queue/
* alicia - can show you individual override entries
* dak ls - shows information about package(s)
* dak queue-report - shows information about package(s) in queue/
* dak override - can show you individual override entries
Generic and useful, but only for those with existing archives
-------------------------------------------------------------
o catherine - migrates packages from legacy locations to the pool
o neve - initializes a projectb database from an exisiting archive
o dak poolize - migrates packages from legacy locations to the pool
o dak init-archive - initializes a projectb database from an exisiting archive
Generic but not overly useful (in normal use)
---------------------------------------------
o ashley - dumps info in .katie files
o julia - sync PostgreSQL users with system users
o rene - check for obsolete or duplicated packages
o rose - directory creation in the initial setup of an archive
o tea - various sanity checks of the database and archive
o natalie - manpiulates/list override entries
o heidi - removes/adds/lists package(s) from/to/for a suite
o saffron - produces various statistics
o rosamund - checks for users with no packages in the archive
o dak dot-dak-decode - dumps info in .dak files
o dak import-users-from-passwd - sync PostgreSQL users with system users
o dak cruft-report - check for obsolete or duplicated packages
o dak init-dirs - directory creation in the initial setup of an archive
o dak check-archive - various sanity checks of the database and archive
o dak control-overrides - manpiulates/list override entries
o dak control-suite - removes/adds/lists package(s) from/to/for a suite
o dak stats - produces various statistics
o dak find-null-maintainers - checks for users with no packages in the archive
Semi-generic
------------
To generate less-used indices files:
o charisma - generates Maintainers file
o denise - generates override.<foo> files
o dak make-maintainers - generates Maintainers file used by, e.g. debbugs
o dak make-overrides - generates override.<foo> files
Mostly Debian(.org) specific
----------------------------
o amber - wrapper for Debian security team
o halle - removes obsolete .changes files from proposed-updates
o jeri - basic dependency checking for proposed-updates
o emilie - syncs fingerprint and uid information with a debian.org LDAP DB
o lauren - manually reject packages from proposed-updates
o dak security-install - wrapper for Debian security team
o dak clean-proposed-updates - removes obsolete .changes files from proposed-updates
o dak check-proposed-updates - basic dependency checking for proposed-updates
o dak reject-proposed-updates - manually reject packages from proposed-updates
o dak import-ldap-fingerprints - syncs fingerprint and uid information with a debian.org LDAP DB
Very Incomplete or otherwise not generally useful
-------------------------------------------------
o alyson - currently only initializes a DB from a katie.conf config file
o andrea - looks for version descrepancies that shouldn't exist in many
archives
o cindy - override cruft checker that doesn't work well with New Incoming
o dak init-db - currently only initializes a DB from a dak.conf config file
o dak compare-suites - looks for version descrepancies that shouldn't exist in many
archives
o dak check-overrides - override cruft checker that doesn't work well with New Incoming
Scripts invoked by other scripts
--------------------------------
o fernanda - invoked by lisa to "check" NEW packages
o claire - invoked by catherine to determine packages still in legacy locations
o katie - common code used by lisa, jennifer, kelly and others
o dak examine-package - invoked by 'dak process-new' to "check" NEW packages
o dak symlink-dists - invoked by 'dak poolize' to determine packages still in legacy locations
How do I get started?
=====================
[Very incomplete - FIXME]
o Write your own katie.conf and apt.conf files. katie looks for those
config files in /etc/katie/. /etc/katie/katie.conf can define
alternative configuration files with Config::host::KatieConfig and
o Write your own dak.conf and apt.conf files. dak looks for those
config files in /etc/dak/. /etc/dak/dak.conf can define
alternative configuration files with Config::host::DakConfig and
Config::host::AptConfig (where "host" is the fully qualified domain
name of your machine).
o Create a PostgreSQL database on the host given in katie.conf's DB::Host
o Create a PostgreSQL database on the host given in dak.conf's DB::Host
with the name specified in DB::Name.
o Run rose: this will create all directories which are specified in
katie.conf and apt.conf.
o Run 'dak init-dirs': this will create all directories which are specified in
dak.conf and apt.conf.
o If you have an existing archive:
* Run neve
* Run 'dak init-archive'
otherwise:
* Create the table structure. init_pool.sql contains all SQL statements
which are needed for this. After changing all occurences of "projectb"
to the name of your database (as defined in DB::Name) you can run:
psql <DB::Name> < init_pool.sql
* Run alyson: it will populate your database with the values from
katie.conf and apt.conf.
* Run 'dak init-db': it will populate your database with the values from
dak.conf and apt.conf.
* Run 'psql <DB::Name> < add_constraints.sql'.
o Copy all templates from the "templates" directory to to the directory
specified in Dir::Templates, and adapt them to your distribution.
......
......@@ -95,7 +95,7 @@ What breaks:
================================================================================
[1] For versions of dependents meaning: binaries compiled from the
source of BYHAND or NEW uploads. Due to katie's fascist
source of BYHAND or NEW uploads. Due to dak's fascist
source-must-exist checking, these binaries must be held back until
the BYHAND/NEW uploads are processed.
......
......@@ -9,5 +9,5 @@ Standard options
-V/--version
Exceptions:
katie - backwardly compatible options with dinstall
heidi - ? hysterical raisins and testing compatibility
dak process-unchecked - backwardly compatible options with dinstall
dak control-suite - ? hysterical raisins and testing compatibility
......@@ -310,7 +310,7 @@ Canadians: This is a lighthouse. Your call.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
<neuro> i didn't slam today's dinstall too badly, did I?
<elmo> File "/org/ftp.debian.org/katie/kelly", line 608, in main
<elmo> File "/org/ftp.debian.org/dak/process_accepted.py", line 608, in main
<elmo> sys.stderr.write("Installed %d package %s, %s.\n" % (install_count, sets, utils.size_type(int(install_bytes))));
<elmo> OverflowError: float too large to convert
......
Rough Guide to doing Stable Point Releases in Debian
----------------------------------------------------
o use halle to get rid of obsolete .changes from p-u ['halle *.changes' from within p-u]
o [also check for obsolete .changes caused by melanie-ing from p-u]
o run 'dak clean-proposed-updates' to get rid of obsolete .changes
from p-u ['dak clean-proposed-updates *.changes' from within p-u]
o [also check for obsolete .changes caused by 'dak rm'-ing from p-u]
o Install, reject and remove packages as directed by the SRM using
kelly (installs), lauren (rejects) and melanie (removals)
'dak process-accepted' (installs), 'dak reject-proposed-updates'
(rejects) and 'dak rm' (removals)
NB: removing packages are not logged to the stable ChangeLog; you
need to do that byhand.
......@@ -15,15 +17,15 @@ o Close any applicable woody bugs
o Update version number in README and dists/README (ftp-master only)
o Update the 'Debian<n>.<n>r<n>' symlink in dists/
o Clean up dists/ChangeLog (add header, basically)
o Update version fields in katie.conf[-non-US]
o Update version fields in dak.conf[-non-US]
o Update fields in suite table in postgresql (see below)
o Comment out "Untouchable" in katie.conf[-non-US]
o Run 'jenna -s stable'
o Comment out "Untouchable" in dak.conf[-non-US]
o Run 'dak make-suite-file-list -s stable'
o Run apt-ftparchive generate apt.conf.stable[-non-US]
o Run 'ziyi stable' ** FIXME: requires apt.conf.stable stanza for stable in apt.conf
** FIXME: must be run as katie
o Uncomment "Untouchable" in katie.conf[-non-US]
o Run 'dak generate-releases stable' ** FIXME: requires apt.conf.stable stanza for stable in apt.conf
** FIXME: must be run as dak
o Uncomment "Untouchable" in dak.conf[-non-US]
Yes, this sucks and more of it should be automated.
......
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment