Commit 007617ad authored by Daniel Stender's avatar Daniel Stender

New upstream version 1.15.0

parent 30612be5
sudo: false
language: python
python:
- '3.5'
install: pip install -U tox setuptools_scm
env:
matrix:
- TESTENV=py26-pytest26
- TESTENV=py26-pytest27
- TESTENV=py26-pytest28
- TESTENV=py26-pytest29
- TESTENV=py27-pytest26
- TESTENV=py27-pytest27
- TESTENV=py27-pytest28
- TESTENV=py27-pytest29
- TESTENV=py34-pytest26
- TESTENV=py34-pytest27
- TESTENV=py34-pytest28
- TESTENV=py34-pytest29
- TESTENV=py35-pytest27
- TESTENV=py35-pytest28
- TESTENV=py35-pytest29
- TESTENV=py27-pytest28-pexpect
- TESTENV=py35-pytest28-pexpect
- TESTENV=flakes
- TESTENV=readme
script: tox --recreate -e $TESTENV
notifications:
irc:
channels:
- chat.freenode.net#pytest
on_success: change
on_failure: change
skip_join: true
email:
- pytest-commit@python.org
deploy:
provider: pypi
user: ronny
password:
secure: cxmSDho5d+PYKEM4ZCg8ms1P4lzhYkrw6fEOm2HtTcsuCyY6aZMSgImWAnEYbJHSkdzgcxlXK9UKJ9B0YenXmBCkAr7UjdnpNXNmkySr0sYzlH/sfqt/dDATCHFaRKxnkOSOVywaDYhT9n8YudbXI77pXwD12i/CeSSJDbHhsu0JYUfAcb+D6YjRYoA2SEGCnzSzg+gDDfwXZx4ZiODCGLVwieNp1klCg88YROUE1BaYYNuUOONvfXX8+TWowbCF6ChH1WL/bZ49OStEYQNuYxZQZr4yClIqu9VJbchrU8j860K9ott2kkGTgfB/dDrQB/XncBubyIX9ikzCQAmmBXWAI3eyvWLPDk2Jz7kW2l2RT7syct80tCq3JhvQ1qdwr5ap7siocTLgnBW0tF4tkHSTFN3510fkc43npnp6FThebESQpnI24vqpwJ9hI/kW5mYi014Og2E/cpCXnz2XO8iZPDbqAMQpDsqEQoyhfGNgPTGp4K30TxRtwZBI5hHhDKnnR16fXtRgt1gYPvz/peUQvvpOm4JzIzGXPzluuutpnCBy75v5+oiwT3YRrLL/Meims9FtDDXL3qQubAE/ezIOOpm0N5XXV8DxIom8EN71yq5ab1tqhM+tBX7owRjy4FR4If2Q8feBdmTuh26DIQt/y+qSG8VkB9Sw/JCjc7c=
on:
tags: true
distributions: sdist bdist_wheel
repo: pytest-dev/pytest-xdist
1.15.0
------
- new ``worker_id`` fixture, returns the id of the worker in a test or fixture.
Thanks Jared Hellman for the PR.
- display progress during collection only when in a terminal, similar to pytest #1397 issue.
Thanks Bruno Oliveira for the PR.
- fix internal error message when ``--maxfail`` is used (#62, #65).
Thanks Collin RM Stocks and Bryan A. Jones for reports and Bruno Oliveira for the PR.
1.14
----
- new hook: ``pytest_xdist_node_collection_finished(node, ids)``, called when
a worker has finished collection. Thanks Omer Katz for the request and
Bruno Oliveira for the PR.
- fix README display on pypi
- fix #22: xdist now works if the internal tmpdir plugin is disabled.
Thanks Bruno Oliveira for the PR.
- fix #32: xdist now works if looponfail or boxed are disabled.
Thanks Bruno Oliveira for the PR.
1.13.1
-------
- fix a regression -n 0 now disables xdist again
1.13
-------------------------
- extended the tox matrix with the supported py.test versions
- split up the plugin into 3 plugin's
to prepare the departure of boxed and looponfail.
looponfail will be a part of core
and forked boxed will be replaced
with a more reliable primitive based on xdist
- conforming with new pytest-2.8 behavior of returning non-zero when all
tests were skipped or deselected.
- new "--max-slave-restart" option that can be used to control maximum
number of times pytest-xdist can restart slaves due to crashes. Thanks to
Anatoly Bubenkov for the report and Bruno Oliveira for the PR.
- release as wheel
- "-n" option now can be set to "auto" for automatic detection of number
of cpus in the host system. Thanks Suloev Dmitry for the PR.
1.12
-------------------------
- fix issue594: properly report errors when the test collection
is random. Thanks Bruno Oliveira.
- some internal test suite adaptation (to become forward
compatible with the upcoming pytest-2.8)
1.11
-------------------------
- fix pytest/xdist issue485 (also depends on py-1.4.22):
attach stdout/stderr on --boxed processes that die.
- fix pytest/xdist issue503: make sure that a node has usually
two items to execute to avoid scoped fixtures to be torn down
pre-maturely (fixture teardown/setup is "nextitem" sensitive).
Thanks to Andreas Pelme for bug analysis and failing test.
- restart crashed nodes by internally refactoring setup handling
of nodes. Also includes better code documentation.
Many thanks to Floris Bruynooghe for the complete PR.
1.10
-------------------------
- add glob support for rsyncignores, add command line option to pass
additional rsyncignores. Thanks Anatoly Bubenkov.
- fix pytest issue382 - produce "pytest_runtest_logstart" event again
in master. Thanks Aron Curzon.
- fix pytest issue419 by sending/receiving indices into the test
collection instead of node ids (which are not neccessarily unique
for functions parametrized with duplicate values)
- send multiple "to test" indices in one network message to a slave
and improve heuristics for sending chunks where the chunksize
depends on the number of remaining tests rather than fixed numbers.
This reduces the number of master -> node messages (but not the
reverse direction)
1.9
-------------------------
- changed LICENSE to MIT
- fix duplicate reported test ids with --looponfailing
(thanks Jeremy Thurgood)
- fix pytest issue41: re-run tests on all file changes, not just
randomly select ones like .py/.c.
- fix pytest issue347: slaves running on top of Python3.2
will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency
bugs.
1.8
-------------------------
- fix pytest-issue93 - use the refined pytest-2.2.1 runtestprotocol
- fix pytest-issue93 - use the refined pytest-2.2.1 runtestprotocol
interface to perform eager teardowns for test items.
1.7
......@@ -85,4 +205,3 @@
- cleaned up termination handling
- make -x cause hard killing of test nodes to decrease wait time
until the traceback shows up on first failure
next release critical
-----------------------------------------------
tag: bug
miserably fails: --dist=each --tx popen --tx socket=...
rename / hooks
-----------------------------------------------
tag: bug
node -> slave
transition for hooks?
configure_node -> configure_slave
allow to remotely run xdist tests with xdist
-----------------------------------------------
tag: feature
allow to run xdist own tests using its own mechanism.
currently this doesn't work because the remote side
has no py.test plugin. How to configure/do
register "xdist.plugin" on the remote side?
see to avoid any "from _pytest" internal imports
-----------------------------------------------
tag: feature
currently tests and even xdist core code imports
names from the internal _pytest namespace.
See to avoid it.
The execnet package is released under the provisions of the Gnu Public
License (GPL), version 2 or later.
See http://www.fsf.org/licensing/licenses/ for more information.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
This package also contains some minor parts which
which are useable under the MIT license.
If you have questions and/or want to use parts of
the code under a different license than the GPL
please contact me.
holger krekel, January 2010, holger at merlinux eu
......@@ -2,5 +2,6 @@ include CHANGELOG
include LICENSE
include README.txt
include setup.py
include tox.ini
graft testing
prune .hg
prune .git
# Overview #
`xdist` works by spawning one or more **workers**, which are controlled
by the **master**. Each **worker** is responsible for performing
a full test collection and afterwards running tests as dictated by the **master**.
The execution flow is:
1. **master** spawns one or more **workers** at the beginning of
the test session. The communication between **master** and **worker** nodes makes use of
[execnet](http://codespeak.net/execnet/) and its [gateways](http://codespeak.net/execnet/basics.html#gateways-bootstrapping-python-interpreters).
The actual interpreters executing the code for the **workers** might
be remote or local.
1. Each **worker** itself is a mini pytest runner. **workers** at this
point perform a full test collection, sending back the collected
test-ids back to the **master** which does not
perform any collection itself.
1. The **master** receives the result of the collection from all nodes.
At this point the **master** performs some sanity check to ensure that
all **workers** collected the same tests (including order), bailing out otherwise.
If all is well, it converts the list of test-ids into a list of simple
indexes, where each index corresponds to the position of that test in the
original collection list. This works because all nodes have the same
collection list, and saves bandwidth because the **master** can now tell
one of the workers to just *execute test index 3* index of passing the
full test id.
1. If **dist-mode** is **each**: the **master** just sends the full list
of test indexes to each node at this moment.
1. If **dist-mode** is **load**: the **master** takes around 25% of the
tests and sends them one by one to each **worker** in a round robin
fashion. The rest of the tests will be distributed later as **workers**
finish tests (see below).
1. **workers** re-implement `pytest_runtestloop`: pytest's default implementation
basically loops over all collected items in the `session` object and executes
the `pytest_runtest_protocol` for each test item, but in xdist **workers** sit idly
waiting for **master** to send tests for execution. As tests are
received by **workers**, `pytest_runtest_protocol` is executed for each test.
Here it worth noting an implementation detail: **workers** always must keep at
least one test item on their queue due to how the `pytest_runtest_protocol(item, nextitem)`
hook is defined: in order to pass the `nextitem` to the hook, the worker must wait for more
instructions from master before executing that remaining test. If it receives more tests,
then it can safely call `pytest_runtest_protocol` because it knows what the `nextitem` parameter will be.
If it receives a "shutdown" signal, then it can execute the hook passing `nextitem` as `None`.
1. As tests are started and completed at the **workers**, the results are sent
back to the **master**, which then just forwards the results to
the appropriate pytest hooks: `pytest_runtest_logstart` and
`pytest_runtest_logreport`. This way other plugins (for example `junitxml`)
can work normally. The **master** (when in dist-mode **load**)
decides to send more tests to a node when a test completes, using
some heuristics such as test durations and how many tests each **worker**
still has to run.
1. When the **master** has no more pending tests it will
send a "shutdown" signal to all **workers**, which will then run their
remaining tests to completion and shut down. At this point the
**master** will sit waiting for **workers** to shut down, still
processing events such as `pytest_runtest_logreport`.
## FAQ ##
> Why does each worker do its own collection, as opposed to having
the master collect once and distribute from that collection to the workers?
If collection was performed by master then it would have to
serialize collected items to send them through the wire, as workers live in another process.
The problem is that test items are not easily (impossible?) to serialize, as they contain references to
the test functions, fixture managers, config objects, etc. Even if one manages to serialize it,
it seems it would be very hard to get it right and easy to break by any small change in pytest.
This diff is collapsed.
This diff is collapsed.
install:
- C:\Python35\python -m pip install tox setuptools_scm
build: false # Not a C# project, build stuff at the test step instead.
test_script:
- C:\Python35\python -m tox
If your testing involves C or C++ libraries you might have to deal
with crashing processes. The xdist-plugin provides the ``--boxed`` option
to run each test in a controlled subprocess. Here is a basic example::
# content of test_module.py
import pytest
import os
import time
# run test function 50 times with different argument
@pytest.mark.parametrize("arg", range(50))
def test_func(arg):
time.sleep(0.05) # each tests takes a while
if arg % 19 == 0:
os.kill(os.getpid(), 15)
If you run this with::
$ py.test --boxed
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 50 items
test_module.py f..................f..................f...........
================================= FAILURES =================================
_______________________________ test_func[0] _______________________________
/home/hpk/tmp/doc-exec-420/test_module.py:6: running the test CRASHED with signal 15
______________________________ test_func[19] _______________________________
/home/hpk/tmp/doc-exec-420/test_module.py:6: running the test CRASHED with signal 15
______________________________ test_func[38] _______________________________
/home/hpk/tmp/doc-exec-420/test_module.py:6: running the test CRASHED with signal 15
=================== 3 failed, 47 passed in 3.41 seconds ====================
You'll see that a couple of tests are reported as crashing, indicated
by lower-case ``f`` and the respective failure summary. You can also use
the xdist-provided parallelization feature to speed up your testing::
$ py.test --boxed -n3
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
gw0 I / gw1 I / gw2 I
gw0 [50] / gw1 [50] / gw2 [50]
scheduling tests via LoadScheduling
..f...............f..................f............
================================= FAILURES =================================
_______________________________ test_func[0] _______________________________
[gw0] linux2 -- Python 2.7.3 /home/hpk/venv/1/bin/python
/home/hpk/tmp/doc-exec-420/test_module.py:6: running the test CRASHED with signal 15
______________________________ test_func[19] _______________________________
[gw2] linux2 -- Python 2.7.3 /home/hpk/venv/1/bin/python
/home/hpk/tmp/doc-exec-420/test_module.py:6: running the test CRASHED with signal 15
______________________________ test_func[38] _______________________________
[gw2] linux2 -- Python 2.7.3 /home/hpk/venv/1/bin/python
/home/hpk/tmp/doc-exec-420/test_module.py:6: running the test CRASHED with signal 15
=================== 3 failed, 47 passed in 2.03 seconds ====================
This diff is collapsed.
.gitignore
.travis.yml
CHANGELOG
ISSUES.txt
LICENSE
MANIFEST.in
README.txt
OVERVIEW.md
README.rst
appveyor.yml
setup.cfg
setup.py
tox.ini
example/boxed.txt
pytest_xdist.egg-info/PKG-INFO
pytest_xdist.egg-info/SOURCES.txt
pytest_xdist.egg-info/dependency_links.txt
......@@ -12,13 +20,25 @@ pytest_xdist.egg-info/requires.txt
pytest_xdist.egg-info/top_level.txt
testing/acceptance_test.py
testing/conftest.py
testing/conftest.pyc
testing/test_boxed.py
testing/test_dsession.py
testing/test_looponfail.py
testing/test_newhooks.py
testing/test_plugin.py
testing/test_remote.py
testing/test_slavemanage.py
testing/__pycache__/acceptance_test.cpython-26-PYTEST.pyc
testing/__pycache__/test_boxed.cpython-26-PYTEST.pyc
testing/__pycache__/test_dsession.cpython-26-PYTEST.pyc
testing/__pycache__/test_looponfail.cpython-26-PYTEST.pyc
testing/__pycache__/test_newhooks.cpython-26-PYTEST.pyc
testing/__pycache__/test_plugin.cpython-26-PYTEST.pyc
testing/__pycache__/test_remote.cpython-26-PYTEST.pyc
testing/__pycache__/test_slavemanage.cpython-26-PYTEST.pyc
xdist/__init__.py
xdist/_version.py
xdist/boxed.py
xdist/dsession.py
xdist/looponfail.py
xdist/newhooks.py
......
[pytest11]
xdist = xdist.plugin
xdist.boxed = xdist.boxed
xdist.looponfail = xdist.looponfail
execnet>=1.0.8
pytest>=2.2.1
\ No newline at end of file
execnet>=1.1
pytest>=2.4.2
py>=1.4.22
[bdist_wheel]
universal = 1
[egg_info]
tag_svn_revision = 0
tag_build =
tag_date = 0
tag_svn_revision = 0
......@@ -2,29 +2,38 @@ from setuptools import setup
setup(
name="pytest-xdist",
version='1.8',
description='py.test xdist plugin for distributed testing and loop-on-failing modes',
long_description=open('README.txt').read(),
license='GPLv2 or later',
use_scm_version={'write_to': 'xdist/_version.py'},
description='py.test xdist plugin for distributed testing'
' and loop-on-failing modes',
long_description=open('README.rst').read(),
license='MIT',
author='holger krekel and contributors',
author_email='py-dev@codespeak.net,holger@merlinux.eu',
url='http://bitbucket.org/hpk42/pytest-xdist',
author_email='pytest-dev@python.org,holger@merlinux.eu',
url='https://github.com/pytest-dev/pytest-xdist',
platforms=['linux', 'osx', 'win32'],
packages = ['xdist'],
entry_points = {'pytest11': ['xdist = xdist.plugin'],},
packages=['xdist'],
entry_points={
'pytest11': [
'xdist = xdist.plugin',
'xdist.looponfail = xdist.looponfail',
'xdist.boxed = xdist.boxed',
],
},
zip_safe=False,
install_requires = ['execnet>=1.0.8', 'pytest>=2.2.1'],
install_requires=['execnet>=1.1', 'pytest>=2.4.2', 'py>=1.4.22'],
setup_requires=['setuptools_scm'],
classifiers=[
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: GNU General Public License (GPL)',
'Operating System :: POSIX',
'Operating System :: Microsoft :: Windows',
'Operating System :: MacOS :: MacOS X',
'Topic :: Software Development :: Testing',
'Topic :: Software Development :: Quality Assurance',
'Topic :: Utilities',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Development Status :: 5 - Production/Stable',
'Framework :: Pytest',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: POSIX',
'Operating System :: Microsoft :: Windows',
'Operating System :: MacOS :: MacOS X',
'Topic :: Software Development :: Testing',
'Topic :: Software Development :: Quality Assurance',
'Topic :: Utilities',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
],
)
This diff is collapsed.
import py
import pytest
import execnet
@pytest.fixture(scope="session", autouse=True)
def _ensure_imports():
# we import some modules because pytest-2.8's testdir fixture
# will unload all modules after each test and this cause
# (unknown) problems with execnet.Group()
execnet.Group
execnet.makegateway
pytest_plugins = "pytester"
#rsyncdirs = ['.', '../xdist', py.path.local(execnet.__file__).dirpath()]
# rsyncdirs = ['.', '../xdist', py.path.local(execnet.__file__).dirpath()]
@pytest.fixture(autouse=True)
def _divert_atexit(request, monkeypatch):
import atexit
l = []
def finish():
while l:
l.pop()()
monkeypatch.setattr(atexit, "register", l.append)
request.addfinalizer(finish)
def pytest_addoption(parser):
parser.addoption('--gx',
action="append", dest="gspecs", default=None,
help=("add a global test environment, XSpec-syntax. "))
action="append",
dest="gspecs",
help=("add a global test environment, XSpec-syntax. "))
def pytest_funcarg__specssh(request):
@pytest.fixture
def specssh(request):
return getspecssh(request.config)
def getgspecs(config):
return [execnet.XSpec(spec)
for spec in config.getvalueorskip("gspecs")]
@pytest.fixture
def testdir(testdir):
# pytest before 2.8 did not have a runpytest_subprocess
if not hasattr(testdir, "runpytest_subprocess"):
testdir.runpytest_subprocess = testdir.runpytest
return testdir
# configuration information for tests
def getgspecs(config):
return [execnet.XSpec(spec)
for spec in config.getvalueorskip("gspecs")]
return [execnet.XSpec(spec) for spec in config.getvalueorskip("gspecs")]
def getspecssh(config):
xspecs = getgspecs(config)
......@@ -30,10 +64,10 @@ def getspecssh(config):
return str(spec)
py.test.skip("need '--gx ssh=...'")
def getsocketspec(config):
xspecs = getgspecs(config)
for spec in xspecs:
if spec.socket:
return spec
py.test.skip("need '--gx socket=...'")
import py
import pytest
import os
@py.test.mark.skipif("not hasattr(os, 'fork')")
needsfork = pytest.mark.skipif(not hasattr(os, "fork"),
reason="os.fork required")
@needsfork
def test_functional_boxed(testdir):
p1 = testdir.makepyfile("""
import os
......@@ -13,16 +18,41 @@ def test_functional_boxed(testdir):
"*1 failed*"
])
@needsfork
@pytest.mark.parametrize("capmode", [
"no",
pytest.mark.xfail("sys", reason="capture cleanup needed"),
pytest.mark.xfail("fd", reason="capture cleanup needed")])
def test_functional_boxed_capturing(testdir, capmode):
p1 = testdir.makepyfile("""
import os
import sys
def test_function():
sys.stdout.write("hello\\n")
sys.stderr.write("world\\n")
os.kill(os.getpid(), 15)
""")
result = testdir.runpytest(p1, "--boxed", "--capture=%s" % capmode)
result.stdout.fnmatch_lines("""
*CRASHED*
*stdout*
hello
*stderr*
world
*1 failed*
""")
class TestOptionEffects:
def test_boxed_option_default(self, testdir):
tmpdir = testdir.tmpdir.ensure("subdir", dir=1)
config = testdir.parseconfig()
assert not config.option.boxed
py.test.importorskip("execnet")
pytest.importorskip("execnet")
config = testdir.parseconfig('-d', tmpdir)
assert not config.option.boxed
def test_is_not_boxed_by_default(self, testdir):
config = testdir.parseconfig(testdir.tmpdir)
assert not config.option.boxed
from xdist.dsession import DSession, LoadScheduling, EachScheduling
from _pytest import main as outcome
from xdist.dsession import (
DSession, LoadScheduling, EachScheduling, report_collection_diff,
)
import py
import pytest
import execnet
XSpec = execnet.XSpec
def run(item, node, excinfo=None):
runner = item.config.pluginmanager.getplugin("runner")
rep = runner.ItemTestReport(item=item,
excinfo=excinfo, when="call")
rep = runner.ItemTestReport(item=item, excinfo=excinfo, when="call")
rep.node = node
return rep
class MockGateway:
_count = 0
def __init__(self):
self.id = str(self._count)
self._count += 1
class MockNode:
def __init__(self):
self.sent = []
self.gateway = MockGateway()
self._shutdown = False
def send_runtest(self, nodeid):
self.sent.append(nodeid)
def send_runtest_some(self, indices):
self.sent.extend(indices)
def send_runtest_all(self):
self.sent.append("ALL")
def sendlist(self, items):
self.sent.extend(items)
def shutdown(self):
self._shutdown=True
self._shutdown = True
@property
def shutting_down(self):
return self._shutdown
def dumpqueue(queue):
while queue.qsize():
print(queue.get())
class TestEachScheduling:
def test_schedule_load_simple(self):
node1 = MockNode()
......@@ -51,9 +67,9 @@ class TestEachScheduling:
assert sched.tests_finished()
assert node1.sent == ['ALL']
assert node2.sent == ['ALL']
sched.remove_item(node1, collection[0])
sched.remove_item(node1, 0)
assert sched.tests_finished()
sched.remove_item(node2, collection[0])
sched.remove_item(node2, 0)
assert sched.tests_finished()