Commit 619dd56a authored by SVN-Git Migration's avatar SVN-Git Migration

Imported Upstream version 0.0.14

parent 6dab094b
......@@ -3,3 +3,6 @@ MANIFEST
test.xml
build
.testrepository
__pycache__
testrepository.egg-info
./doc/_build
[DEFAULT]
test_command=${PYTHON:-python} -m subunit.run $LISTOPT $IDOPTION testrepository.tests.test_suite
test_id_option=--load-list $IDFILE
test_list_option=--list
......@@ -9,18 +9,18 @@ are met:
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of Robert Collins nor the names of Subunit contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
3. Neither the name of Robert Collins nor the names of Testrepository
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY ROBERT COLLINS AND SUBUNIT CONTRIBUTORS ``AS IS''
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
THIS SOFTWARE IS PROVIDED BY ROBERT COLLINS AND TESTREPOSITORY CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
......@@ -28,6 +28,7 @@ A concordance of contributors is maintained here to provide an easy reference
for distributions such as Debian that wish to list all the copyright holders
in their metadata:
* Robert Collins <robertc@robertcollins.net>, 2009
* Hewlett-Packard Development Company, L.P., 2013
Code that has been incorporated into Testrepository from other projects will
......
include .bzrignore
include .testr.conf
include Apache-2.0
include BSD
include COPYING
......
......@@ -5,6 +5,268 @@ testrepository release notes
NEXT (In development)
+++++++++++++++++++++
0.0.14
++++++
IMPROVEMENTS
------------
* First cut at full Python 3 support. The 'works for me' release.
(Robert Collins)
0.0.13
++++++
IMPROVEMENTS
------------
* ``setup.py testr`` was not indicating test failures via it's return code.
(Monty Taylor)
0.0.12
++++++
IMPROVEMENTS
------------
* There is now a setuptools extension provided by ``testrespository`` making it
easy to invoke testr from setup.py driven workflows.
(Monty Taylor, Robert Collins)
INTERNALS
---------
* BSD license file incorrectly claimed copyright by subunit contributors.
(Monty Taylor)
* .testr.conf is now shipped in the source distribution to aid folk wanting to
validate that testrepository works correctly on their machine.
(Robert Collins)
0.0.11
++++++
IMPROVEMENTS
------------
* Fix another incompatability with Mac OS X - gdbm dbm modules don't support
get. (Robert Collins, #1094330)
0.0.10
++++++
IMPROVEMENTS
------------
* It's now possible to configure ``test_run_concurrency`` in ``.testr.conf``
to have concurrency defined by a callout. (Robert Collins)
* Testr supports running tests in arbitrary environments. See ``Remote or
isolated test environments`` in MANUAL.txt / ``testr help run``
(Robert Collins)
INTERNALS
---------
* TestCommand is now a fixture. This is used to ensure cached test instances
are disposed of - if using the object to run or list tests, you will need
to adjust your calls. (Robert Collins)
* ``TestCommand`` now offers, and ``TestListingFixture`` consumes a small
protocol for obtaining and releasing test execution instances.
(Robert Collins)
0.0.9
+++++
IMPROVEMENTS
------------
* On OSX the ``anydbm`` module by default returns an implementation that
doesn't support update(). Workaround that by falling back to a loop.
(Robert Collins, #1091500)
* ``testr --analyze-improvements`` now honours test regex filters and only
analyzes matching tests. (Robert Collins)
0.0.8
+++++
CHANGES
-------
* As a side effect of fixing bug #597060 additional arguments passed to testr
run or testr list are only passed to the underlying test runner if they are
preceeded by '--'. (Robert Collins, #597060)
* ``testr run --failing`` will no longer run any tests at all if there are
no failing tests. (Robert Collins, #904400)
IMPROVEMENTS
------------
* ``AbstractArgument`` now handles the case where additional arguments are
present that the argument type cannot parse, but enough have been parsed for
it to be valid. This allows optional arguments to be in the middle of a
grammar. (Robert Collins)
* ``cli.UI`` now passed '--' down to the argument layer for handling rather
than implicitly stripping it. (Robert Collins)
* ``DoubledashArgument`` added to allow fine grained control over the impact of
-- in command lines. (Robert Collins)
* New argument type ``ExistingPathArgument`` for use when commands want to take
the name of a file. (Robert Collins)
* ``testr`` will now show the version. (Robert Collins)
* ``testr last`` when just one test run has been run works again.
(Robert Collins)
* ``testr help command`` now shows the docstring for commands (Robert Collins)
* ``testr --help command`` or ``testr command --help`` now shows the options
for the command. (Robert Collins)
* ``testr run --analyze-isolation`` will search the current failing tests for
spurious failures caused by interactions with other tests.
(Robert Collins, #684069)
* ``testr run --until-failure`` will repeat a test run until interrupted by
ctrl-C or until a failure occurs. (Robert Collins, #680995)
* ``Repository.get_test_run`` now raises KeyError if asked for a missing or
nonexistant test run. (Robert Collins)
* Sphinx has been added to tie the documentation toghether (And it is available
on testrepository.readthedocs.org). (Robert Collins)
* ``StringArgument`` now rejects '--' - it should be handled by the use of a
``DoubledashArgument`` where one is expected. This is a bit awkward and does
not permit passing '--' down to a child process, so further work may be
needed - file a bug if this affects you. (Robert Collins)
* ``test failing --subunit`` now exits 0 unless there was a problem generating
the stream. This is consistent with the general processing model of subunit
generators. (Robert Collins)
* ``testr last`` now supports ``--subunit`` and when passed will output
the stored subunit stream. Note that the exit code is always 0 when this
is done (unless an exception occurs reading the stream) - subunit consumers
should parse the subunit to determine success/failure. (Robert Collins)
* ``testr load`` now supports passing filenames to subunit streams to load.
(Robert Collins, #620386)
* ``testr run`` will now fail a test run if the test process exits non-zero.
As a side effect of this change, if the test program closes its stdout but
does not exit, ``testr run`` will hang (waiting for the test program to
exit). (Robert Collins)
* ``testr run --load-list FILENAME`` will limit the tests run to the test ids
supplied in the list file FILENAME. This is useful for manually specifying
the tests to run, or running testr subordinate to testr (e.g. on remote
machines). (Robert Collins, partial fix for #597060)
* ``testr run foo`` now applies foo as a regex filter against the tests
found by doing a listing of the test runners tests. Likewise
``testr list-tests foo`` will apply foo as a filter against the found tests.
This makes it easy to limit the tests that will be requested for running by
the backend test process - simply pass one or more regex filters into testr
run. (Robert Collins, #597060)
* Test tags are now shown in failures. Of particular interest for folk debgging
cross-test interactions will be the worker-N tags which indicate which
backend test process executed a given test. (Robert Collins)
0.0.7
+++++
CHANGES
-------
* testrepository is now distributed via distribute rather than distutils,
allowing installation via pip into virtualenv environments. (Robert Collins)
IMPROVEMENTS
------------
* stream loading will now synthesise datestamps before demultiplexing rather
than on insertion into the repository. This fixes erroneously short times
being recorded on non timestamped streams. Additionally, moving the automatic
addition of timestamp material in front of the demuxer has removed the skew
that caused test times to be reported as longer than the stream could
indicate (by the amount of time the test runner took to start outputting
subunit). This time may be something we want to track later, but the prior
mechanism was inconsistent between the current run and reporting on prior
runs, which lead to a very confusing UI. Now it is consistent, but totally
ignores that overhead.
(Robert Collins, #1048126, #980950)
* ``testr run`` now accepts a --concurrency option, allowing command line
override of the number of workers spawned. This allows conccurency on
operating systems where autodetection is not yet implemented, or just
debugging problems with concurrent test suites. (Robert Collins, #957145)
* ''test_id_list_default'' would prevent ''test_list_option'' being used in
previous releases. For Python environments where the context to load tests
from is always needed this was not an issue (and thus not uncovered). However
given a test runner which wants either a global context or a list of specific
tests with no global context, there was no way to achieve that with this bug.
(Robert Collins, #1027042)
0.0.6
+++++
CHANGES
-------
* Now relies on subunit 0.0.8 or better and testtools 0.9.15 or better.
IMPROVEMENTS
------------
* Much better handling of unicode input from subunit streams. Specifically,
we won't crash when we can't figure something out. (Francesco Banconi,
Martin Packman, #955006)
* Parallel tests now record their worker thread number as tags in tests.
This makes identifying test ordering problems much easier.
(Benji York, #974622)
* Python2.7 changed the interface for DBM, this has been worked around.
(Robert Collins, #775214, #961103)
* Subunit 0.0.7 Changes its TestResultFilter implementation, requiring the
subclass in testrepository.filter to be come more robust.
(Robert Collins)
* A horrible thinko in the testrepository test suite came to light and has been
fixed. How the tests ever ran is a mystery. (Robert Collins, #881497)
* ''failing'', ''run'' and ''load'' now both take a ''--subunit'' option, which
displays output in raw subunit format. If ''--full-results'' is passed too,
then all subunit information is displayed. (Brad Crittenden, #949950)
* Setting ''filter_tags'' in ''.testr.conf'' will cause tests tagged with those
tags to be hidden unless the fail/error. This requires Subunit 0.0.8. If
an older version of subunit is configured, testr will return an error.
(Robert Collins, #914166)
* ``testr`` will drop into PDB from its command line UI if the environment
variable TESTR_PDB is set. (Robert Collins)
* Test partitioning now handles a corner case where multiple tests have a
reported duration of 0. Previously they could all accumulate into one
partition, now they split across partitions (the length of a partition is
used as a tie breaker if two partitions have the same duration).
(Robert Collins, #914359)
* The test 'test_outputs_results_to_stdout' was sensitive to changes in
testtools and has been made more generic. (Robert Collins)
0.0.5
+++++
......
Metadata-Version: 1.0
Metadata-Version: 1.1
Name: testrepository
Version: 0.0.5
Version: 0.0.14
Summary: A repository of test results.
Home-page: https://launchpad.net/testrepository
Author: Robert Collins
......@@ -37,6 +37,9 @@ Description: Test Repository
Quick Start
~~~~~~~~~~~
Create a config file::
$ touch .testr.conf
Create a repository::
$ testr init
......@@ -49,13 +52,24 @@ Description: Test Repository
$ testr failing
Delete a repository::
$ testr delete
$ rm -rf .testrepository
Documentation
~~~~~~~~~~~~~
More detailed documentation including design and implementation details, a
user manual, and guidelines for development of Test Repository itself can be
found in the doc/ directory.
found at https://testrepository.readthedocs.org/en/latest, or in the source
tree at doc/ (run make -C doc html).
Keywords: subunit unittest testrunner
Platform: UNKNOWN
Classifier: Development Status :: 6 - Mature
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
......@@ -29,6 +29,9 @@ Test Repository is under BSD / Apache 2.0 licences. See the file COPYING in the
Quick Start
~~~~~~~~~~~
Create a config file::
$ touch .testr.conf
Create a repository::
$ testr init
......@@ -41,11 +44,12 @@ Query the repository::
$ testr failing
Delete a repository::
$ testr delete
$ rm -rf .testrepository
Documentation
~~~~~~~~~~~~~
More detailed documentation including design and implementation details, a
user manual, and guidelines for development of Test Repository itself can be
found in the doc/ directory.
found at https://testrepository.readthedocs.org/en/latest, or in the source
tree at doc/ (run make -C doc html).
......@@ -47,5 +47,6 @@ reuse from other programs.
Threads/concurrency
~~~~~~~~~~~~~~~~~~~
In general using any public interface
In general using any public interface is fine, but keeping syncronisation
needs to a minimum for code readability.
......@@ -40,3 +40,15 @@ Running the tests
Generally just ``make`` is all that is needed to run all the tests. However
if dropping into pdb, it is currently more convenient to use
``python -m testtools.run testrepository.tests.test_suite``.
Diagnosing issues
-----------------
The cli UI will drop into pdb when an error is thrown if TESTR_PDB is set in
the environment. This can be very useful for diagnosing problems.
Releasing
---------
Update testrepository/__init__.py version numbers. Release to pypi. Pivot the
next milestone on LP to version, and make a new next milestone.
This diff is collapsed.
.. Test Repository documentation master file, created by
sphinx-quickstart on Mon Dec 3 23:24:00 2012.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Test Repository's documentation!
===========================================
Contents:
.. toctree::
:maxdepth: 2
MANUAL
DESIGN
DEVELOPERS
Indices and tables
==================
* :ref:`genindex`
* :ref:`search`
[egg_info]
tag_build =
tag_date = 0
tag_svn_revision = 0
#!/usr/bin/env python
#
# Copyright (c) 2009, 2010 Testrepository Contributors
# Copyright (c) 2009-2013 Testrepository Contributors
#
# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
# license at the users choice. A copy of both licenses are available in the
......@@ -13,7 +13,7 @@
# license you chose for the specific language governing permissions and
# limitations under that license.
from distutils.core import setup
from setuptools import setup
import email
import os
......@@ -41,7 +41,7 @@ def get_version_from_pkg_info():
def get_version():
"""Return the version of testtools that we are building."""
"""Return the version of testrepository that we are building."""
version = '.'.join(
str(component) for component in testrepository.__version__[0:3])
phase = testrepository.__version__[3]
......@@ -59,7 +59,7 @@ def get_version():
return version + '-r%s' % revno
description = file(os.path.join(os.path.dirname(__file__), 'README.txt'), 'rb').read()
description = open(os.path.join(os.path.dirname(__file__), 'README.txt'), 'rt').read()
setup(name='testrepository',
......@@ -68,6 +68,18 @@ setup(name='testrepository',
url='https://launchpad.net/testrepository',
description='A repository of test results.',
long_description=description,
keywords="subunit unittest testrunner",
classifiers = [
'Development Status :: 6 - Mature',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Topic :: Software Development :: Quality Assurance',
'Topic :: Software Development :: Testing',
],
scripts=['testr'],
version=get_version(),
packages=['testrepository',
......@@ -80,4 +92,23 @@ setup(name='testrepository',
'testrepository.tests.repository',
'testrepository.tests.ui',
'testrepository.ui',
])
],
install_requires=[
'fixtures',
'python-subunit >= 0.0.10',
'testtools >= 0.9.29',
],
extras_require = dict(
test=[
'bzr',
'pytz',
'testresources',
'testscenarios',
]
),
entry_points={
'distutils.commands': [
'testr = testrepository.setuptools_command:Testr',
],
},
)
Metadata-Version: 1.1
Name: testrepository
Version: 0.0.14
Summary: A repository of test results.
Home-page: https://launchpad.net/testrepository
Author: Robert Collins
Author-email: robertc@robertcollins.net
License: UNKNOWN
Description: Test Repository
+++++++++++++++
Overview
~~~~~~~~
This project provides a database of test results which can be used as part of
developer workflow to ensure/check things like:
* No commits without having had a test failure, test fixed cycle.
* No commits without new tests being added.
* What tests have failed since the last commit (to run just a subset).
* What tests are currently failing and need work.
Test results are inserted using subunit (and thus anything that can output
subunit or be converted into a subunit stream can be accepted).
A mailing list for discussion, usage and development is at
https://launchpad.net/~testrepository-dev - all are welcome to join. Some folk
hang out on #testrepository on irc.freenode.net.
CI for the project is at http://build.robertcollins.net/job/testrepository-default/.
Licensing
~~~~~~~~~
Test Repository is under BSD / Apache 2.0 licences. See the file COPYING in the source for details.
Quick Start
~~~~~~~~~~~
Create a config file::
$ touch .testr.conf
Create a repository::
$ testr init
Load a test run into the repository::
$ testr load < testrun
Query the repository::
$ testr stats
$ testr last
$ testr failing
Delete a repository::
$ rm -rf .testrepository
Documentation
~~~~~~~~~~~~~
More detailed documentation including design and implementation details, a
user manual, and guidelines for development of Test Repository itself can be
found at https://testrepository.readthedocs.org/en/latest, or in the source
tree at doc/ (run make -C doc html).
Keywords: subunit unittest testrunner
Platform: UNKNOWN
Classifier: Development Status :: 6 - Mature
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
.bzrignore
.testr.conf
Apache-2.0
BSD
COPYING
INSTALL.txt
MANIFEST.in
Makefile
NEWS
README.txt
setup.py
testr
doc/DESIGN.txt
doc/DEVELOPERS.txt
doc/MANUAL.txt
doc/index.txt
testrepository/__init__.py
testrepository/results.py
testrepository/setuptools_command.py
testrepository/testcommand.py
testrepository/testlist.py
testrepository/utils.py
testrepository.egg-info/PKG-INFO
testrepository.egg-info/SOURCES.txt
testrepository.egg-info/dependency_links.txt
testrepository.egg-info/entry_points.txt
testrepository.egg-info/requires.txt
testrepository.egg-info/top_level.txt
testrepository/arguments/__init__.py
testrepository/arguments/command.py
testrepository/arguments/doubledash.py
testrepository/arguments/path.py
testrepository/arguments/string.py
testrepository/commands/__init__.py
testrepository/commands/commands.py
testrepository/commands/failing.py
testrepository/commands/help.py
testrepository/commands/init.py
testrepository/commands/last.py
testrepository/commands/list_tests.py
testrepository/commands/load.py
testrepository/commands/quickstart.py
testrepository/commands/run.py
testrepository/commands/slowest.py
testrepository/commands/stats.py
testrepository/repository/__init__.py
testrepository/repository/file.py
testrepository/repository/memory.py
testrepository/repository/samba_buildfarm.py
testrepository/tests/__init__.py
testrepository/tests/monkeypatch.py
testrepository/tests/stubpackage.py
testrepository/tests/test_arguments.py
testrepository/tests/test_commands.py
testrepository/tests/test_matchers.py
testrepository/tests/test_monkeypatch.py
testrepository/tests/test_repository.py
testrepository/tests/test_results.py
testrepository/tests/test_setup.py
testrepository/tests/test_stubpackage.py
testrepository/tests/test_testcommand.py
testrepository/tests/test_testr.py
testrepository/tests/test_ui.py
testrepository/tests/arguments/__init__.py
testrepository/tests/arguments/test_command.py
testrepository/tests/arguments/test_doubledash.py
testrepository/tests/arguments/test_path.py
testrepository/tests/arguments/test_string.py
testrepository/tests/commands/__init__.py
testrepository/tests/commands/test_commands.py
testrepository/tests/commands/test_failing.py
testrepository/tests/commands/test_help.py
testrepository/tests/commands/test_init.py
testrepository/tests/commands/test_last.py
testrepository/tests/commands/test_list_tests.py
testrepository/tests/commands/test_load.py
testrepository/tests/commands/test_quickstart.py
testrepository/tests/commands/test_run.py
testrepository/tests/commands/test_slowest.py
testrepository/tests/commands/test_stats.py
testrepository/tests/repository/__init__.py
testrepository/tests/repository/test_file.py
testrepository/tests/ui/__init__.py
testrepository/tests/ui/test_cli.py
testrepository/tests/ui/test_decorator.py
testrepository/ui/__init__.py
testrepository/ui/cli.py
testrepository/ui/decorator.py
testrepository/ui/model.py
\ No newline at end of file
[distutils.commands]
testr = testrepository.setuptools_command:Testr
fixtures
python-subunit >= 0.0.10
testtools >= 0.9.29
[test]
bzr
pytz
testresources
testscenarios
\ No newline at end of file
......@@ -33,4 +33,4 @@ The tests package contains tests and test specific support code.
# established at this point, and setup.py will use a version of next-$(revno).
# If the releaselevel is 'final', then the tarball will be major.minor.micro.
# Otherwise it is major.minor.micro~$(revno).
__version__ = (0, 0, 5, 'final', 0)
__version__ = (0, 0, 14, 'final', 0)
......@@ -29,6 +29,10 @@ extending the testrepository.arguments __path__ to include a directory
containing their argument types - no __init__ is needed in that directory.)
"""
import sys
from testtools.compat import reraise
class AbstractArgument(object):
"""A argument that a command may need.
......@@ -83,13 +87,22 @@ class AbstractArgument(object):
"""
count = 0
result = []
error = None
while len(argv) > count and (
count < self.maximum_count or self.maximum_count is None):
self.maximum_count is None or count < self.maximum_count):
arg = argv[count]
count += 1
result.append(self._parse_one(arg))
try:
result.append(self._parse_one(arg))
except ValueError:
# argument rejected this element
error = sys.exc_info()
count -= 1