README.rst 9.08 KB
Newer Older
1 2 3


.. image:: http://img.shields.io/pypi/v/pytest-xdist.svg
4 5 6
    :alt: PyPI version
    :target: https://pypi.python.org/pypi/pytest-xdist

7 8 9
.. image:: https://img.shields.io/conda/vn/conda-forge/pytest-xdist.svg
    :target: https://anaconda.org/conda-forge/pytest-xdist

10 11
.. image:: https://img.shields.io/pypi/pyversions/pytest-xdist.svg
    :alt: Python versions
12 13 14
    :target: https://pypi.python.org/pypi/pytest-xdist

.. image:: https://travis-ci.org/pytest-dev/pytest-xdist.svg?branch=master
15
    :alt: Travis CI build status
16
    :target: https://travis-ci.org/pytest-dev/pytest-xdist
17 18

.. image:: https://ci.appveyor.com/api/projects/status/56eq1a1avd4sdd7e/branch/master?svg=true
19
    :alt: AppVeyor build status
20 21
    :target: https://ci.appveyor.com/project/pytestbot/pytest-xdist

22 23 24
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
    :target: https://github.com/ambv/black

25
xdist: pytest distributed testing plugin
26
========================================
27

28
The `pytest-xdist`_ plugin extends pytest with some unique
29 30 31 32 33 34 35 36
test execution modes:

* test run parallelization_: if you have multiple CPUs or hosts you can use
  those for a combined test run.  This allows to speed up
  development or to use special resources of `remote machines`_.


* ``--looponfail``: run your tests repeatedly in a subprocess.  After each run
37
  pytest waits until a file in your project changes and then re-runs
38 39 40 41 42 43
  the previously failing tests.  This is repeated until all tests pass
  after which again a full run is performed.

* `Multi-Platform`_ coverage: you can specify different Python interpreters
  or different platforms and run tests in parallel on all of them.

44
Before running tests remotely, ``pytest`` efficiently "rsyncs" your
45 46 47 48
program source code to the remote place.  All test results
are reported back and displayed to your local terminal.
You may specify different Python versions and interpreters.

49
If you would like to know how pytest-xdist works under the covers, checkout
50 51
`OVERVIEW <https://github.com/pytest-dev/pytest-xdist/blob/master/OVERVIEW.md>`_.

52 53

Installation
54
------------
55 56 57 58 59

Install the plugin with::

    pip install pytest-xdist

60
or use the package in develop/in-place mode with
61 62
a checkout of the `pytest-xdist repository`_ ::

63
    pip install --editable .
64 65 66 67

.. _parallelization:

Speed up test runs by sending tests to multiple CPUs
68
----------------------------------------------------
69 70 71

To send tests to multiple CPUs, type::

72
    pytest -n NUM
73 74

Especially for longer running tests or tests requiring
75
a lot of I/O this can lead to considerable speed ups. This option can
76 77 78
also be set to ``auto`` for automatic detection of the number of CPUs.

If a test crashes the interpreter, pytest-xdist will automatically restart
79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
that worker and report the failure as usual. You can use the
``--max-worker-restart`` option to limit the number of workers that can
be restarted, or disable restarting altogether using ``--max-worker-restart=0``.

By default, the ``-n`` option will send pending tests to any worker that is available, without
any guaranteed order, but you can control this with these options:

* ``--dist=loadscope``: tests will be grouped by **module** for *test functions* and
  by **class** for *test methods*, then each group will be sent to an available worker,
  guaranteeing that all tests in a group run in the same process. This can be useful if you have
  expensive module-level or class-level fixtures. Currently the groupings can't be customized,
  with grouping by class takes priority over grouping by module.
  This feature was added in version ``1.19``.

* ``--dist=loadfile``: tests will be grouped by file name, and then will be sent to an available
  worker, guaranteeing that all tests in a group run in the same worker. This feature was added
  in version ``1.21``.
96 97 98


Running tests in a Python subprocess
99
------------------------------------
100

101
To instantiate a python3.5 subprocess and send tests to it, you may type::
102

103
    pytest -d --tx popen//python=python3.5
104

105
This will start a subprocess which is run with the ``python3.5``
106 107 108 109
Python interpreter, found in your system binary lookup path.

If you prefix the --tx option value like this::

110
    --tx 3*popen//python=python3.5
111 112 113 114 115 116 117

then three subprocesses would be created and tests
will be load-balanced across these three processes.

.. _boxed:

Running tests in a boxed subprocess
118
-----------------------------------
119

120 121 122
This functionality has been moved to the
`pytest-forked <https://github.com/pytest-dev/pytest-forked>`_ plugin, but the ``--boxed`` option
is still kept for backward compatibility.
123 124 125 126

.. _`remote machines`:

Sending tests to remote SSH accounts
127
------------------------------------
128 129 130 131 132 133

Suppose you have a package ``mypkg`` which contains some
tests that you can successfully run locally. And you
have a ssh-reachable machine ``myhost``.  Then
you can ad-hoc distribute your tests by typing::

134
    pytest -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
135 136

This will synchronize your :code:`mypkg` package directory
137
to a remote ssh account and then locally collect tests
138 139 140 141 142 143 144
and send them to remote places for execution.

You can specify multiple :code:`--rsyncdir` directories
to be sent to the remote side.

.. note::

145
  For pytest to collect and send tests correctly
146 147 148
  you not only need to make sure all code and tests
  directories are rsynced, but that any test (sub) directory
  also has an :code:`__init__.py` file because internally
149
  pytest references tests as a fully qualified python
150 151 152 153 154 155 156 157 158 159 160 161
  module path.  **You will otherwise get strange errors**
  during setup of the remote side.


You can specify multiple :code:`--rsyncignore` glob patterns
to be ignored when file are sent to the remote side.
There are also internal ignores: :code:`.*, *.pyc, *.pyo, *~`
Those you cannot override using rsyncignore command-line or
ini-file option(s).


Sending tests to remote Socket Servers
162
--------------------------------------
163 164 165 166 167 168 169 170 171 172

Download the single-module `socketserver.py`_ Python program
and run it like this::

    python socketserver.py

It will tell you that it starts listening on the default
port.  You can now on your home machine specify this
new socket host with something like this::

173
    pytest -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
174 175 176 177 178 179 180


.. _`atonce`:
.. _`Multi-Platform`:


Running tests on many platforms at once
181
---------------------------------------
182 183 184

The basic command to run tests on multiple platforms is::

185
    pytest --dist=each --tx=spec1 --tx=spec2
186 187 188 189

If you specify a windows host, an OSX host and a Linux
environment this command will send each tests to all
platforms - and report back failures from all platforms
190
at once. The specifications strings use the `xspec syntax`_.
191 192 193 194 195 196 197 198

.. _`xspec syntax`: http://codespeak.net/execnet/basics.html#xspec

.. _`socketserver.py`: http://bitbucket.org/hpk42/execnet/raw/2af991418160/execnet/script/socketserver.py

.. _`execnet`: http://codespeak.net/execnet

Identifying the worker process during a test
199
--------------------------------------------
200

201
*New in version 1.15.*
202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222

If you need to determine the identity of a worker process in
a test or fixture, you may use the ``worker_id`` fixture to do so:

.. code-block:: python

    @pytest.fixture()
    def user_account(worker_id):
        """ use a different account in each xdist worker """
        return "account_%s" % worker_id

When ``xdist`` is disabled (running with ``-n0`` for example), then
``worker_id`` will return ``"master"``.

Additionally, worker processes have the following environment variables
defined:

* ``PYTEST_XDIST_WORKER``: the name of the worker, e.g., ``"gw2"``.
* ``PYTEST_XDIST_WORKER_COUNT``: the total number of workers in this session,
  e.g., ``"4"`` when ``-n 4`` is given in the command-line.

223 224
The information about the worker_id in a test is stored in the ``TestReport`` as
well, under the ``worker_id`` attribute.
225 226


227
Specifying test exec environments in an ini file
228
------------------------------------------------
229

230 231
You can use pytest's ini file configuration to avoid typing common options.
You can for example make running with three subprocesses your default like this:
232 233 234 235 236 237 238 239 240 241 242

.. code-block:: ini

    [pytest]
    addopts = -n3

You can also add default environments like this:

.. code-block:: ini

    [pytest]
243
    addopts = --tx ssh=myhost//python=python3.5 --tx ssh=myhost//python=python3.6
244 245 246

and then just type::

247
    pytest --dist=each
248 249 250

to run tests in each of the environments.

251

252
Specifying "rsync" dirs in an ini-file
253
--------------------------------------
254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269

In a ``tox.ini`` or ``setup.cfg`` file in your root project directory
you may specify directories to include or to exclude in synchronisation:

.. code-block:: ini

    [pytest]
    rsyncdirs = . mypkg helperpkg
    rsyncignore = .hg

These directory specifications are relative to the directory
where the configuration file was found.

.. _`pytest-xdist`: http://pypi.python.org/pypi/pytest-xdist
.. _`pytest-xdist repository`: https://github.com/pytest-dev/pytest-xdist
.. _`pytest`: http://pytest.org