Commit d79acb19 authored by Raphaël Hertzog's avatar Raphaël Hertzog

docs: major overhaul of the documentation

Restructure the content in 3 parts:
- For end-users
- For administrators
- For developers/contributors

Add lots of missing content, in particular in the "end-users" part
with information about teams and package subscriptions, etc.

Fixes: #23
parent fbdff8c3
.. _about:
What is Distro Tracker?
=======================
Distro Tracker lets you follow the evolution of a Debian-based
distribution both with email updates and with a comprehensive web
interface. This information may be interesting to package maintainers,
contributors, advanced users, QA members, etc...
Most of the information tracked is about packages but it can be
displayed used in multiple contexts (per package, per maintenance team,
per maintainer, etc.).
Distro Tracker aims to be as extensible and as customizable as possible in
order to allow Debian derivatives to deploy their own instance of Distro
Tracker if they so choose.
.. _email_about:
Email Interface
---------------
The email interface forwards email messages regarding a package, to users
who are subscribed to that package.
Distro Tracker receives email messages for each package on a special address in the
form of ``dispatch+<package-name>@<distro-tracker-domain>``. If the local
part of the email is a valid package name, the message is a valid package
message. Messages to the package's address can be sent by either automated
tools or users themselves.
Each package email is first tagged with one of the existing keywords and then
forwarded only to the subscribers interested in that keyword. Users sending
package messages to the tracker can tag their own messages with a keyword by using a
local part of the address in the form of ``dispatch+<package-name>_<keyword>``.
A user can choose which mails they are interested in, by selecting to either
receive messages tagged with one of their "default" keywords or they can choose
a different set of keywords for each of their package subscriptions.
Each vendor can provide their own set of available keywords and a set of rules to
tag incoming messages with one of the keywords. If there is no such tagging
mechanism and no keyword in the local part of the email address, the message is
tagged with the "default" keyword.
The keyword of the forwarded message is included in a mail header
``X-Distro-Tracker-Keyword``.
There are three types of packages which are considered valid packages for the
email interface:
- source package
- pseudo package - a list of pseudo packages should be provided by a vendor specific
function :func:`distro_tracker.vendor.skeleton.rules.get_pseudo_package_list`
- subscription-only package - neither a pseudo package nor a source package, but
still allows the same email functionality as the other package types
.. _email_control_about:
Email Control Interface
+++++++++++++++++++++++
Users can control their subscriptions (packages subscribed to, keywords to
accept) by sending control commands enclosed in an email message to a robot. The
designated email address for its control interface is ``control@<distro-tracker-domain>``,
by default. This is customizable by vendors.
Control emails contain a list of commands, each in a separate line. Available
commands can be obtained by sending a control message with the ``help``
command (an email message with ``help`` on a single line).
If a message which reaches the control bot contains too many lines which are
invalid commands, processing is halted and only the commands found up until
that point are processed. If there were no valid commands found in the email,
no response is sent, otherwise an email message is constructed and sent in
response to indicate the status of each processed command.
Certain commands may require confirmation by the user, e.g. subscribing to
receive package messages. If a control message contains any such command,
apart from a response email, a separate "confirmation" email is also sent.
Only one confirmation email is sent regardless of how many commands from the
original control message require confirmation. It includes a confirmation
code, instructions on how to confirm the commands and any extra information
about each command which is to be confirmed.
.. _web_about:
Web Interface
-------------
Each source and pseudo package has its own Web dashboard displaying information
aggregated from various sources. This page is accessible at the URL
``/<package-name>``.
If a binary package name is entered, the user is redirected to the
corresponding source package's page.
A convenient search form is provided on the front page as well as on each
package's page which allows users to jump to another package page. It
supports auto completed suggestions for both source and pseudo packages.
The information currently provided for each package is divided into the
following panels:
- General package information panel
(:class:`GeneralInformationPanel <distro_tracker.core.panels.GeneralInformationPanel>`)
- Versions panel
(:class:`VersionsInformationPanel <distro_tracker.core.panels.VersionsInformationPanel>`)
- Binaries panel
(:class:`BinariesInformationPanel <distro_tracker.core.panels.BinariesInformationPanel>`)
- News panel
(:class:`NewsPanel <distro_tracker.core.panels.NewsPanel>`)
- Bugs panel
(:class:`BugsPanel <distro_tracker.core.panels.BugsPanel>`)
- Action needed panel
(:class:`ActionNeededPanel <distro_tracker.core.panels.ActionNeededPanel>`)
Vendors can easily customize and add new panels to the page. For more
information refer to the
:ref:`design overview documentation <panels_web_design>` regarding panels and
the individual documentation for each of the core panel classes for ways to
extend them.
.. _rss_about:
RSS news feed [coming soon]
+++++++++++++
.. _rest_about:
REST interface [coming soon]
++++++++++++++
.. _rdf_about:
RDF metadata [coming soon]
++++++++++++
Command-line Interface
----------------------
You may use some commands to start some tasks for instance. See the list of available commands with ::
$ ./manage.py --help
.. _mail-forwarding:
Forwarding Mails through the Package Tracker
===========================================
General Principle
-----------------
The email messages to forward are received through the
:samp:`dispatch@{distro-tracker-domain}` email address.
Distro Tracker is usually able to analyze the email messages from common
tools (debbugs, DAK, GitLab, etc.) and to figure out the name of the
package and the associated :ref:`keyword <keywords>`. If that is not the case,
there are two ways to pass the required package/keyword data:
* either by adding the ``X-Distro-Tracker-Package`` and
``X-Distro-Tracker-Keyword`` `headers <mail-headers>`_ in the mail sent to
the :samp:`dispatch@{distro-tracker-domain}` email address;
* or by sending the mails to :samp:`dispatch+{package}@{distro-tracker-domain}`
or :samp:`dispatch+{package}_{keyword}@{distro-tracker-domain}`.
Once the package and keyword are known, the message can be forwarded to
the package subscribers who have the corresponding keyword enabled.
.. warning::
When Distro-Tracker has been unable to find a keyword for the message,
it will assign the ``default`` keyword. To prevent random spam from
being forwarded through the package tracker, a message with the
``default`` keyword will only be forwarded if it contains the
``X-Distro-Tracker-Approved`` header with a non-empty value.
.. note::
Each :ref:`vendor <vendor-customization>` can provide their own set of
available keywords and a set of rules to tag incoming messages with
one of the keywords. If there is no such tagging mechanism and no
keyword in the local part of the email address, the message is tagged
with the ``default`` keyword.
.. _forwarding-vcs-commits:
Forwarding VCS Commits to the Package Tracker
---------------------------------------------
If you use a publicly accessible VCS repository for maintaining your
Debian package, you may want to forward the commit notifications to the
package tracker so that the subscribers (and possible co-maintainers) can
closely follow the package's evolution.
Debian's Distro Tracker knows how to parse email notifications generated by
GitLab's *Email on Pushes* integration or by Git's *git-multimail.py*
hook script. It assumes that the name of the git repository (with the
eventual ``.git`` suffix stripped) is the name of the source package and
it will automatically affect the `vcs` keyword to those messages.
Thanks to this, you just have to configure your git repository to send
the notifications to dispatch@tracker.debian.org.
.. note::
The git commit emails are recognized by their ``X-GitLab-Project`` or
``X-Git-Repo`` headers. Those fields are parsed as a path whose last
component is the name of the source package (with a possible ``.git``
suffix.
If you use another tool that doesn't generate any of the recognized headers
(or if your Git repositories are not named after the source package name),
you will have to send your VCS commit notification mails to
:samp:`dispatch+{sourcepackage}_vcs@{distro-tracker-domain}`.
.. _setup:
.. _initial-setup:
Setup
=====
Contents:
Initial Setup
=============
.. toctree::
:maxdepth: 2
setup
installation
webserver
mailbot
repositories
.. _vendor-customization:
Vendor Customization
--------------------
The package tracker has been developed to be usable outside of the Debian
project, for example in the context of a Debian derivative distribution.
To be able to tweak the behaviour of the package tracker for the specific
needs of each project, one can provide vendor-specific code in
:py:mod:`distro_tracker.vendor` and enable that code by selecting
the vendor in the ``DISTRO_TRACKER_VENDOR_RULES``.
Have a look at the :py:mod:`skeleton vendor
<distro_tracker.vendor.skeleton.rules>` to have an idea of the
things that you can tweak through this mechanism.
......@@ -2,7 +2,6 @@ distro_tracker
==============
.. toctree::
:maxdepth: 4
distro_tracker
functional_tests
.. Distro Tracker documentation master file, created by
sphinx-quickstart on Fri Jun 14 17:14:16 2013.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Distro Tracker's documentation!
Welcome to Distro Tracker's Documentation!
==========================================
Contents:
Distro Tracker is the name of the software that powers the
`Debian Package Tracker <https://tracker.debian.org>`_.
This software lets you follow the evolution of a Debian-based
distribution both with email updates and with a comprehensive web
interface.
Having all the information about packages conveniently available in a
single place is particularly interesting for package maintainers,
contributors, advanced users, etc.
Documentation for End-Users
---------------------------
.. toctree::
usage/follow-packages
usage/messages
usage/teams
usage/mailbot
Documentation for Administrators
--------------------------------
.. toctree::
admin/initial-setup
admin/email
admin/vendor
Documentation for Developers
----------------------------
.. toctree::
:maxdepth: 3
about
design
API Documentation <api/modules>
setup/index
contributing
devel/design
API Documentation <api/modules>
.. _follow-packages:
Following Packages
==================
A package tracker is mainly of interest to retrieve information about
packages. Depending on your needs, there are multiple ways to get those
information.
Visiting the Package's Page
---------------------------
Each source package has its own web dashboard displaying information
aggregated from various sources. This page is accessible at the URL
:samp:`https://{distro-tracker-domain}/pkg/{package-name}`.
.. note::
If a binary package name is entered, the user is redirected to the
corresponding source package's page.
A convenient search form is provided on the front page as well as on each
package's page which allows users to jump to another package page. It
supports auto-completion for both source and binary package names.
The information currently provided for each package is divided into the
following panels:
**general**
Contains general information extracted from the package currently
available in the development repository:
* name of the source package
* current version
* current maintainer and uploaders
* list of supported architectures
* standards version field
* URL(s) for the VCS repository
**versions**
For each repository tracked, lists the version of the source package
that it contains.
**versioned links**
Provides links to files which are specific to a given version.
The set of links can vary depending on the configuration but here
are the links that can be put in place:
* .dsc file (source package)
* debian/changelog (checklist icon)
* debian/copyright (balance of law icon)
* debian/rules (tools icon)
* debian/control file (package icon)
**binaries**
Lists the binary packages built by the source package and provides
related links.
**action needed**
Lists various issues affecting the source package. This information
is mainly of interest to package maintainers and contributors. The
higher priority issues are displayed first. More information can be
displayed by clicking on the chevron on the left of each entry.
**news**
Sorted list of news related to the package. Clicking on the link
shows the full content associated to the news. Most news are actually
email messages that one can get by subscribing to the package.
**bugs**
Statistics and links to bug reports.
**links**
Links to external sources of information that have something relevant
for this source package.
Other vendor-specific panels can appear. For example, the Debian Package Tracker has an
`ubuntu` panel with information related to the state of the package in the
Ubuntu derivative distribution.
.. _package-subscription:
Getting Updates by Email
------------------------
If you don't want to visit the web page regularly, you can subscribe to
the package to receive updates by email (see :ref:`email-messages` to
have an idea of what messages you can get).
There are two ways to subscribe to a package:
Subscribing on the Website
~~~~~~~~~~~~~~~~~~~~~~~~~~
On each package page
(:samp:`https://{distro-tracker-domain}/pkg/{package-name}`) you will find
a :guilabel:`Subscribe` button. If you are authenticated, it will
immediately subscribe you and show you an :guilabel:`Unsubscribe` button
that you can use to revert the operation. If you are not authenticated,
it will invite you to login first to be able to complete the operation.
If you have have multiple emails associated to your account, the
subscription process will ask you to select the email address that
will receive the notifications.
Subscribing by Email
~~~~~~~~~~~~~~~~~~~~
To subscribe to a package through email, you will have to send an email
to :samp:`control@{distro-tracker-domain}` containing the command
:samp:`subscribe {package-name}` either in the subject or in the body of
the mail. This will subscribe the email address that you used to send
the message. You can ask for the subscription of another email address
by using the command :samp:`subscribe {package-name} {email}`.
The mailbot will send back a confimation mail to the email address being
subscribed. The message will contain a confirmation command that the user
must send back to the mailbot. A simple reply is usually enough for
this as the mailbot is smart enough to detect the command even when it's
quoted in the reply.
Following Updates with an RSS Feed
----------------------------------
Each package provides a dedicated RSS feed available at the following URL:
:samp:`https://{distro-tracker-domain}/pkg/{package-name}/rss`
You can find a small `rss feed` icon at the top-right of the `news`
panel on the package's page, it is linked to the RSS feed.
The RSS feed collates the regular news (from the `news` panel) as well as
the items from the `action needed` panel.
.. _usage:
Documentation for end-users
===========================
Most users will be happy to just browse the website provided by Distro
Tracker and they will find there the information that they are looking
for. But more advanced users will want to create an account and be able
to:
* subscribe to packages and receive e-mail notifications for various
events (bug reports, package upload, etc.)
* subscribe to teams and receive e-mail notifications for all packages
monitored by the team
* customize the above subscriptions to select exactly the notifications
that they receive
All those operations can be done from the website once you are
authenticated but it can also be done through commands sent to
a mailbot (:samp:`control@{distro-tracker-domain}`).
.. _package-subscription:
Subscribing to a package
------------------------
On the website
~~~~~~~~~~~~~~
On each package page
(:samp:`https://{distro-tracker-domain}/pkg/{package-name}`) you will find
a :guilabel:`Subscribe` button. If you are authenticated, it will
immediately subscribe you and show you an :guilabel:`Unsubscribe` button
that you can use to revert the operation. If you are not authenticated,
it will invite you to login first to be able to complete the operation.
If you have have multiple emails associated to your account, the
subscription process will ask you to select the email address that
will receive the notifications.
With the mailbot
~~~~~~~~~~~~~~~~
To subscribe to a package through email, you will have to send an email
to :samp:`control@{distro-tracker-domain}` containing the command
:samp:`subscribe {package-name}` either in the subject or in the body of
the mail. This will subscribe the email address that you used to send
the message. You can ask for the subscription of another email address
by using the command :samp:`subscribe {package-name} {email}`.
The mailbot will send back a confimation mail to the email address being
subscribed. The message will contain a confirmation command that the user
must send back to the mailbot. A simple reply is usually enough for
this as the mailbot is smart enough to detect the command even when it's
quoted in the reply.
.. _team-subscription:
Subscribing to a team
---------------------
On the website
~~~~~~~~~~~~~~
On each team page
(:samp:`https://{distro-tracker-domain}/teams/{team-identifier}/`) you will find
a :guilabel:`Join` button. If you are authenticated, it will
immediately add you to the team and show you a :guilabel:`Leave` button
that you can use to revert the operation. If you are not authenticated,
it will invite you to login first to be able to complete the operation.
If you have have multiple emails associated to your account, the
subscription process will ask you to select the email address that
will receive the notifications.
With the mailbot
~~~~~~~~~~~~~~~~
To join a team through email, you will have to send an email
to :samp:`control@{distro-tracker-domain}` containing the command
:samp:`join-team {team-identifier}` either in the subject or in the body of
the mail. This will subscribe the email address that you used to send
the message. You can ask for the subscription of another email address
by using the command :samp:`join-team {team-identifier} {email}`.
The mailbot will send back a confimation mail to the email address being
subscribed. The message will contain a confirmation command that the user
must send back to the mailbot. A simple reply is usually enough for
this as the mailbot is smart enough to detect the command even when it's
quoted in the reply.
.. note::
The team identifier to use is the same identifier that is used in the URL
of the team on the website. This is the `slug` field in the team creation
form.
More details
------------
.. toctree::
email
web
.. _email-interaction:
Interacting with Distro-Tracker by Email
========================================
.. _mailbot:
How the Mailbot Works
---------------------
Users can control their subscriptions (packages subscribed to, keywords to
accept) by sending control commands enclosed in an email message to a robot. The
designated email address for its control interface is
:samp:`control@{distro-tracker-domain}` (aka control@tracker.debian.org
for Debian's instance).
Control emails contain a list of commands, each in a separate line. Available
commands can be obtained by sending a control message with the ``help``
command (an email message with ``help`` on a single line).
If a message which reaches the control bot contains too many lines which are
invalid commands, processing is halted and only the commands found up until
that point are processed. If there were no valid commands found in the email,
no response is sent, otherwise an email message is constructed and sent in
response to indicate the status of each processed command.
Certain commands may require confirmation by the user, e.g. subscribing to
receive package messages. If a control message contains any such command,
apart from a response email, a separate "confirmation" email is also sent.
Only one confirmation email is sent regardless of how many commands from the
original control message require confirmation. It includes a confirmation
code, instructions on how to confirm the commands and any extra information
about each command which is to be confirmed.
.. _email-commands:
Mailbot Commands Reference
--------------------------
:samp:`subscribe {sourcepackage} [{email}]`
Subscribes *email* to communications related to the source package
*sourcepackage*. Sender address is used if the second argument is not
present. If *sourcepackage* is not a valid source package, you will get a
warning. However if it is a valid binary package, the package tracker
will subscribe you to the corresponding source package.
:samp:`unsubscribe {sourcepackage} [{email}]`
Removes a previous subscription to the source package *sourcepackage*
using the specified email address or the sender address if the second
argument is left out.
:samp:`unsubscribeall [{email}]`
Removes all subscriptions of the specified email address or the sender
address if the second argument is left out.
:samp:`which [{email}]`
Lists all subscriptions for the sender or the email address optionally
specified.
:samp:`keyword [{email}]`
Tells you the :ref:`keywords <keywords>` that you are accepting.
:samp:`keyword {sourcepackage} [{email}]`
Same as the previous item but for the given source package, since you
may select a different set of keywords for each source package.
:samp:`keyword [{email}] +|-|= {list of keywords}`
Accept (``+``) or refuse (``-``) mails classified under the given
keyword(s). Define the list (``=``) of accepted keywords. This
changes the default set of keywords accepted by a user.
Examples:
* ``keyword + vcs derivatives``
* ``keyword user@example.net - bts bts-control``
* ``keyword = default contact upload-source``
:samp:`keywordall [{email}] +|-|= {list of keywords}`
Accept (``+``) or refuse (``-``) mails classified under the given
keyword(s). Define the list (``=``) of accepted keywords. This
changes the set of accepted keywords of all the currently active
subscriptions of a user.
:samp:`keyword {sourcepackage} [{email}] +|-|= {list of keywords}`
Same as previous item but overrides the keywords list for the
indicated source package.
:samp:`join-team {team-identifier} [{email}]`
Adds *email* (or sender address if not specified) to the team whose
identifier is *team-identifier*. If the team is not public or doesn't
exist, a warning is issued.
:samp:`leave-team {team-identifier} [{email}]`
Removes *email* (or sender address if not specified) from the team whose
identifier is *team-identifier*. If the user identified by the email
is not a member of the team, a warning is issued.
:samp:`list-team-packages {team-identifier}`
Lists all packages of the team whose identifier is *team-identifier*.
If the team is private, the result is only sent if the user is a
member of the team.
:samp:`which-teams [{email}]`
Lists all teams that have *email* (or the sender address if not
specified) as a member.
:samp:`quit | thanks | --`
Stops processing commands. All following lines are ignored by the bot.
.. _email-messages:
Email Messages
==============
When a user is subscribed to a package or to a team, it will receive
different types of emails related to the package (or to the team's packages).
.. _keywords:
Type of Emails
--------------
Each email sent through the package tracker is classified under one of the
**keywords** listed below. This classification enables the users to select the
mails that they want to receive.
The following keywords are enabled by default but users can opt-out
from each of them if they are not interested:
**bts**
All the bug reports and following discussions.
**bts-control**
Metadata and status changes on the bug reports. In the case of the Debian bug
tracker, notifications from the control bot at control@bugs.debian.org.
**upload-source**
Notification of a source package upload that got accepted in the
archive.
**archive**
Other notifications related to the package archive. In the case of
Debian, includes all other emails generated by
:abbr:`DAK (Debian Archive Kit)` like override disparity or package
removal notifications.
**build**
Build failure notifications.
**contact**
Mails sent to the package maintainer. In the case of Debian, a copy of
the mails sent to the :samp:`{package}@packages.debian.org` email aliases.
**summary**
Regular summary emails about the package's status. In the case of
Debian, this includes notification related to its testing migration.
Ideally it should also include notifications of new upstream versions,
and a notification if the package is orphaned (but this is not yet the
case).
**default**
Any other notification received by the package tracker that could not
be better classified.
The followed keyword are disabled by default but users can opt-in to
each of them if they wish to receive the corresponding messages:
**upload-binary**
Notification of a binary package upload that got accepted in the
archive. In the case of Debian, it means a dozen of emails for each
source package upload, every time that the package has been built
for another architecture.
**vcs**
VCS commit notifications, if the package has a VCS repository and the
maintainer has set up forwarding of commit notifications to the
package tracker.
**translation**
Notifications related to translations. In the case of Debian, it
includes translations of descriptions or debconf templates submitted
to the Debian Description Translation Project.
**derivatives**
Information about changes made to the package in derivative
distributions. In the case of Debian, this notably includes
information about Ubuntu source package uploads.
**derivatives-bugs**
Bugs reports and comments from derivative distributions. In the case
of Debian, this includes bug reports traffic from Ubuntu's Launchpad.
.. _mail-headers:
Mail Headers
------------
Once subscribed to a package, the user receives mails forwarded by the package