From 3f545ee621c41ae05e0866c6af8a1ed16755f8d9 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Fri, 22 Dec 2017 16:22:28 -0200 Subject: [PATCH 1/5] Change section titles so they are more prominent in the README --- README.rst | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/README.rst b/README.rst index 9cf17d8..4ed43cb 100644 --- a/README.rst +++ b/README.rst @@ -55,15 +55,12 @@ Install the plugin with:: or use the package in develop/in-place mode with a checkout of the `pytest-xdist repository`_ :: - python setup.py develop - -Usage examples ---------------------- + pip install --editable . .. _parallelization: Speed up test runs by sending tests to multiple CPUs -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +---------------------------------------------------- To send tests to multiple CPUs, type:: @@ -80,7 +77,7 @@ be restarted, or disable restarting altogether using ``--max-slave-restart=0``. Running tests in a Python subprocess -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +------------------------------------ To instantiate a python2.5 sub process and send tests to it, you may type:: @@ -99,7 +96,7 @@ will be load-balanced across these three processes. .. _boxed: Running tests in a boxed subprocess -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +----------------------------------- If you have tests involving C or C++ libraries you might have to deal with tests crashing the process. For this case you may use the boxing @@ -120,7 +117,7 @@ create new boxed subprocesses for each test. .. _`remote machines`: Sending tests to remote SSH accounts -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +------------------------------------ Suppose you have a package ``mypkg`` which contains some tests that you can successfully run locally. And you @@ -155,7 +152,7 @@ ini-file option(s). Sending tests to remote Socket Servers -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +-------------------------------------- Download the single-module `socketserver.py`_ Python program and run it like this:: @@ -174,7 +171,7 @@ new socket host with something like this:: Running tests on many platforms at once -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +--------------------------------------- The basic command to run tests on multiple platforms is:: @@ -192,7 +189,7 @@ at once. The specifications strings use the `xspec syntax`_. .. _`execnet`: http://codespeak.net/execnet Identifying the worker process during a test -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +-------------------------------------------- *New in version 1.15.* @@ -221,9 +218,9 @@ well, under worker_id attribute. Specifying test exec environments in an ini file -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +------------------------------------------------ -pytest (since version 2.0) supports ini-style cofiguration. +pytest supports ini-style cofiguration. You can for example make running with three subprocesses your default like this: @@ -247,7 +244,7 @@ to run tests in each of the environments. Sending groups of related tests to the same worker -++++++++++++++++++++++++++++++++++++++++++++++++++ +-------------------------------------------------- *New in version 1.19.* @@ -322,7 +319,7 @@ Currently the groupings can't be customized, with grouping by class takes priority over grouping by module. Sending tests to the same worker based on their file -++++++++++++++++++++++++++++++++++++++++++++++++++++ +---------------------------------------------------- *New in version 1.21.* @@ -339,7 +336,7 @@ run in the same worker, as well as the tests in ``test_io.py``. Specifying "rsync" dirs in an ini-file -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +------------------------------------- In a ``tox.ini`` or ``setup.cfg`` file in your root project directory you may specify directories to include or to exclude in synchronisation: From b2c63a1ba5ad2dd752fdb13f30f9c1d8894b0e30 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Fri, 22 Dec 2017 16:42:35 -0200 Subject: [PATCH 2/5] Reworded some sections to be more newbie friendly --- README.rst | 118 +++++++++-------------------------------------------- 1 file changed, 20 insertions(+), 98 deletions(-) diff --git a/README.rst b/README.rst index 4ed43cb..575f422 100644 --- a/README.rst +++ b/README.rst @@ -21,7 +21,7 @@ :target: https://ci.appveyor.com/project/pytestbot/pytest-xdist xdist: pytest distributed testing plugin -========================================= +======================================== The `pytest-xdist`_ plugin extends py.test with some unique test execution modes: @@ -46,7 +46,7 @@ You may specify different Python versions and interpreters. Installation ------------------------ +------------ Install the plugin with:: @@ -67,7 +67,7 @@ To send tests to multiple CPUs, type:: py.test -n NUM Especially for longer running tests or tests requiring -a lot of IO this can lead to considerable speed ups. This option can +a lot of I/O this can lead to considerable speed ups. This option can also be set to ``auto`` for automatic detection of the number of CPUs. If a test crashes the interpreter, pytest-xdist will automatically restart @@ -75,6 +75,20 @@ that slave and report the failure as usual. You can use the ``--max-slave-restart`` option to limit the number of slaves that can be restarted, or disable restarting altogether using ``--max-slave-restart=0``. +By default, the ``-n`` option will send pending tests to any worker that is available, without +any guaranteed order, but you can control this with these options: + +* ``--dist=loadscope``: tests will be grouped by **module** for *test functions* and + by **class** for *test methods*, then each group will be sent to an available worker, + guaranteeing that all tests in a group run in the same process. This can be useful if you have + expensive module-level or class-level fixtures. Currently the groupings can't be customized, + with grouping by class takes priority over grouping by module. + This feature was added in version ``1.19``. + +* ``--dist=loadfile``: tests will be grouped by file name, and then will be sent to an available + worker, guaranteeing that all tests in a group run in the same worker. This feature was added + in version ``1.21``. + Running tests in a Python subprocess ------------------------------------ @@ -180,7 +194,7 @@ The basic command to run tests on multiple platforms is:: If you specify a windows host, an OSX host and a Linux environment this command will send each tests to all platforms - and report back failures from all platforms -at once. The specifications strings use the `xspec syntax`_. +at once. The specifications strings use the `xspec syntax`_. .. _`xspec syntax`: http://codespeak.net/execnet/basics.html#xspec @@ -213,8 +227,8 @@ defined: * ``PYTEST_XDIST_WORKER_COUNT``: the total number of workers in this session, e.g., ``"4"`` when ``-n 4`` is given in the command-line. -The information about the worker_id in a test is stored in the TestReport as -well, under worker_id attribute. +The information about the worker_id in a test is stored in the ``TestReport`` as +well, under the ``worker_id`` attribute. Specifying test exec environments in an ini file @@ -243,98 +257,6 @@ and then just type:: to run tests in each of the environments. -Sending groups of related tests to the same worker --------------------------------------------------- - -*New in version 1.19.* - -.. note:: - This is an **experimental** feature: the actual functionality will - likely stay the same, but the CLI might change slightly in future versions. - -You can send groups of related tests to the same worker by using the -``--dist=loadscope`` option. Tests will be grouped by **module** -for *test functions* and by **class** for *test methods*. - -For example, consider this two test files: - -.. code-block:: python - - # content of test_container.py - import pytest - - def test_container_startup(): - pass - - def test_container_logging(): - pass - - @pytest.mark.parametrize('methods', ['ssh', 'http']) - def test_container_communication(methods): - pass - - # content of test_io.py - class TestHDF: - - def test_listing(self): - pass - - def test_search(self): - pass - - - class TestXML: - - def test_listing(self): - pass - - def test_search(self): - pass - - -By executing ``pytest -v --dist=loadscope -n4`` you might get this output -(sorted by worker for readability):: - - ============================= test session starts ============================= - - gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] - scheduling tests via LoadScopeScheduling - - [gw0] PASSED test_container.py::test_container_communication[http] - [gw0] PASSED test_container.py::test_container_communication[ssh] - [gw0] PASSED test_container.py::test_container_logging - [gw0] PASSED test_container.py::test_container_startup - [gw1] PASSED test_io.py::TestHDF::test_listing - [gw1] PASSED test_io.py::TestHDF::test_search - [gw2] PASSED test_io.py::TestXML::test_listing - [gw2] PASSED test_io.py::TestXML::test_search - - ========================== 8 passed in 0.56 seconds =========================== - -As you can see, all test functions from ``test_container.py`` executed on -the same worker ``gw0``, while the test methods from classes ``TestHDF`` and -``TestXML`` executed in workers ``gw1`` and ``gw2`` respectively. - -Currently the groupings can't be customized, with grouping by class takes -priority over grouping by module. - -Sending tests to the same worker based on their file ----------------------------------------------------- - -*New in version 1.21.* - -.. note:: - This is an **experimental** feature: the actual functionality will - likely stay the same, but the CLI might change slightly in future versions. - -You can send tests to the same worker grouped by their filename by using the -``--dist=loadfile`` option, so tests of the same file are guaranteed to run -in the same worker. - -Using the example in the previous section, all tests from ``test_container.py`` will -run in the same worker, as well as the tests in ``test_io.py``. - - Specifying "rsync" dirs in an ini-file ------------------------------------- From 7c3a86a549a75c005006030727a3c19b82e84443 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Fri, 22 Dec 2017 16:49:14 -0200 Subject: [PATCH 3/5] Mention pytest-forked instead of the help for `--boxed` --- README.rst | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/README.rst b/README.rst index 575f422..12bfaf7 100644 --- a/README.rst +++ b/README.rst @@ -112,21 +112,9 @@ will be load-balanced across these three processes. Running tests in a boxed subprocess ----------------------------------- -If you have tests involving C or C++ libraries you might have to deal -with tests crashing the process. For this case you may use the boxing -options:: - - py.test --boxed - -which will run each test in a subprocess and will report if a test -crashed the process. You can also combine this option with -running multiple processes to speed up the test run and use your CPU cores:: - - py.test -n3 --boxed - -this would run 3 testing subprocesses in parallel which each -create new boxed subprocesses for each test. - +This functionality has been moved to the +`pytest-forked `_ plugin, but the ``--boxed`` option +is still kept for backward compatibility. .. _`remote machines`: From 6cd1eb2f43e17475ad8cbdc2628ebe2f13a29c60 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Fri, 22 Dec 2017 16:51:06 -0200 Subject: [PATCH 4/5] Use more modern python and reword part about pytest ini configuration --- README.rst | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/README.rst b/README.rst index 12bfaf7..0457c90 100644 --- a/README.rst +++ b/README.rst @@ -93,16 +93,16 @@ any guaranteed order, but you can control this with these options: Running tests in a Python subprocess ------------------------------------ -To instantiate a python2.5 sub process and send tests to it, you may type:: +To instantiate a python3.5 subprocess and send tests to it, you may type:: - py.test -d --tx popen//python=python2.5 + py.test -d --tx popen//python=python3.5 -This will start a subprocess which is run with the "python2.5" +This will start a subprocess which is run with the ``python3.5`` Python interpreter, found in your system binary lookup path. If you prefix the --tx option value like this:: - --tx 3*popen//python=python2.5 + --tx 3*popen//python=python3.5 then three subprocesses would be created and tests will be load-balanced across these three processes. @@ -222,9 +222,8 @@ well, under the ``worker_id`` attribute. Specifying test exec environments in an ini file ------------------------------------------------ -pytest supports ini-style cofiguration. -You can for example make running with three subprocesses -your default like this: +You can use pytest's ini file configuration to avoid typing common options. +You can for example make running with three subprocesses your default like this: .. code-block:: ini @@ -236,7 +235,7 @@ You can also add default environments like this: .. code-block:: ini [pytest] - addopts = --tx ssh=myhost//python=python2.5 --tx ssh=myhost//python=python3.6 + addopts = --tx ssh=myhost//python=python3.5 --tx ssh=myhost//python=python3.6 and then just type:: From e72eadb413b46ed41168363c1fcb1507ac568147 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Fri, 22 Dec 2017 17:30:47 -0200 Subject: [PATCH 5/5] Fix linting --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index 0457c90..92c7432 100644 --- a/README.rst +++ b/README.rst @@ -245,7 +245,7 @@ to run tests in each of the environments. Specifying "rsync" dirs in an ini-file -------------------------------------- +-------------------------------------- In a ``tox.ini`` or ``setup.cfg`` file in your root project directory you may specify directories to include or to exclude in synchronisation: