Commit 379ce51f authored by Felix Lechner's avatar Felix Lechner Committed by Chris Lamb

Fix instructions on how to run the test suite.

This does not bring the documentation in line with all recent changes,
but will hopefully avoid the worst vitriol from readers for its poor

Fixes the instructions on how to run the test suite.

Also marks the tutorials for writing tests and checks as out of date.

Gbp-Dch: ignore
parent e6312df8
......@@ -6,6 +6,8 @@ Lintian::Tutorial::TestSuite -- Quick intro to running the Lintian testsuite
Warning: This document may be out of date.
This guide will quickly introduce you to running the Lintian test
suite and some tricks. The Lintian test suite is fairly large and
accordingly it can take a substantial amount of time to run. To speed
......@@ -20,34 +22,21 @@ The Lintian test suite is an extensive collection of various test
cases. The test suite is divided into 4 "sub-suites". The majority
of tests are currently located in the "tests" sub-suite.
To run the full suite in all its glory, simply invoke:
$ debian/rules runtests
To run the full suite:
$ mkdir -p debian/test-out
$ t/bin/runtests -k --dump-logs t debian/test-out
$ rm -rf debian/test-out; t/bin/build-test-packages; t/bin/runtests
While writing a new tag (or check) you probably only want to run a
particular (subset of the) test(s). See L</Running a subset of the
tests> for the available options.
When run via I<debian/rules>, the test suite respects
"DEB_BUILD_OPTIONS=parallel=N". When using I<t/bin/runtests> directly, use
I<-jN> to choose the number of threads. Note that "N" denotes the
amount of "worker" threads. The test runner will generally have 2
threads more than that. Also each "worker" will run lintian, which
runs multiple unpacking jobs in parallel as well.
=head2 Running a subset of the tests
In case you are wondering about the 2 extra threads in the test
runner, the first of them is the "coordinator" thread (which will
generally be waiting when the workers are active) and the second one
is the "output" thread (which handles the fancy output).
First, you have to build the test packages with:
=head2 Running a subset of the tests
$ rm -rf debian/test-out; t/bin/build-test-packages;
The following options are available:
Then, the following options are available:
=over 4
......@@ -55,59 +44,31 @@ The following options are available:
To run a single test by its name, use:
$ debian/rules runtests onlyrun=$name
$ t/bin/runtests --dump-logs t debian/test-out $name
$ t/bin/runtests --onlyrun=test:$name
=item Running all tests for a check
To run all tests for a given check, use:
$ debian/rules runtests onlyrun=$check
$ t/bin/runtests --dump-logs -k t debian/test-out $check
$ t/bin/runtests --onlyrun=check:$name
$check must be the name of a check (it will test for
checks/$check.desc) or "legacy". This will run all tests that start
with "$check-".
Note: The "changes" sub-suite in the new test suite does not support
=item Running all tests in a sub-suite
To run all tests in a given sub-suite, use:
$ debian/rules runtests onlyrun=suite:$suites
$ t/bin/runtests --dump-logs -k t debian/test-out suite:$suites
$suites is a comma-separated list of names of sub-suites to run.
Note: this cannot be used to influence the order in which the sub-suites
are run.
=item Running all tests designed for a specific tag
To run all tests that have a "Test-For" or a "Test-Against" for a given
tag, use:
$ debian/rules runtests onlyrun=tag:$tag
$ t/bin/runtests --dump-logs -k t debian/test-out tag:$tag
$ t/bin/runtests --onlyrun=tag:$name
=head2 Running tests under coverage
This feature is currently untested.
It is possible to run most of the tests under L<Devel::Cover>. This is
done by passing I<--coverage> to I<t/bin/runtests>. Example:
......@@ -6,6 +6,8 @@ Lintian::Tutorial::WritingChecks -- Writing checks for Lintian
Warning: This tutorial may be outdated.
This guide will quickly guide you through the basics of writing a
Lintian check. Most of the work is in writing the two files:
......@@ -6,6 +6,8 @@ Lintian::Tutorial::WritingTests -- Short tutorial on writing tests
Warning: This document may be out of date.
This document attempts to be a short / quick tutorial to the Lintian
test suite from a test-writer's perspective. As such, it will only
cover the standard type of tests (from the "tests" suite).
......@@ -32,18 +34,25 @@ They are B<not> used for testing Lintian tags.
=item -
changes / debs / source
These tests all test for the presence of tags after building test
packages using skeletons. For most cases, we recommend
These suites are small test suites that test some particular tags for
Skeleton: upload-non-native
suites are small test suites that test some particular tags for
I<.changes>, I<.deb> or I<.dsc> files. Typically, you will find the
more exotic tags here, which require some special fiddling and cannot
be built by a "standard" dh7 + dpkg build.
=item -
This suite is the standard test suite for testing Lintian tags.
These tests look to match the literal output of Lintian. These tests
are useful as general false positives. They also catch Lintian messages
unrelated to tags.
......@@ -51,6 +60,8 @@ With this in mind, let us move on to the scope.
=head2 Scope of the tutorial
The "tests" suite alone is fairly complex on its own. To keep things
simple, the tutorial will limit itself to creating a "native"
package with no special requirements in the "tests" suite.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment