Prepare release 1.22.3

plugin: get cpu_count from sched_getaffinity

.gitignore

@@ -30,5 +30,6 @@ issue/
 pytestdebug.log
 .tox/
 .cache/
+.pytest_cache/
 .eggs/
 .idea/

.travis.yml

@@ -25,6 +25,11 @@ env:
 install: pip install tox setuptools_scm
 script: tox
 
+stages:
+- test
+- name: deploy
+  if: repo = pytest-dev/pytest-xdist AND tag IS present
+
 jobs:
   include:
     - stage: test
@@ -50,7 +55,7 @@ jobs:
       deploy:
         provider: pypi
         user: ronny
-        distributions: sdist bdist_wheel  
+        distributions: sdist bdist_wheel
         skip_upload_docs: true
         password:
           secure: cxmSDho5d+PYKEM4ZCg8ms1P4lzhYkrw6fEOm2HtTcsuCyY6aZMSgImWAnEYbJHSkdzgcxlXK9UKJ9B0YenXmBCkAr7UjdnpNXNmkySr0sYzlH/sfqt/dDATCHFaRKxnkOSOVywaDYhT9n8YudbXI77pXwD12i/CeSSJDbHhsu0JYUfAcb+D6YjRYoA2SEGCnzSzg+gDDfwXZx4ZiODCGLVwieNp1klCg88YROUE1BaYYNuUOONvfXX8+TWowbCF6ChH1WL/bZ49OStEYQNuYxZQZr4yClIqu9VJbchrU8j860K9ott2kkGTgfB/dDrQB/XncBubyIX9ikzCQAmmBXWAI3eyvWLPDk2Jz7kW2l2RT7syct80tCq3JhvQ1qdwr5ap7siocTLgnBW0tF4tkHSTFN3510fkc43npnp6FThebESQpnI24vqpwJ9hI/kW5mYi014Og2E/cpCXnz2XO8iZPDbqAMQpDsqEQoyhfGNgPTGp4K30TxRtwZBI5hHhDKnnR16fXtRgt1gYPvz/peUQvvpOm4JzIzGXPzluuutpnCBy75v5+oiwT3YRrLL/Meims9FtDDXL3qQubAE/ezIOOpm0N5XXV8DxIom8EN71yq5ab1tqhM+tBX7owRjy4FR4If2Q8feBdmTuh26DIQt/y+qSG8VkB9Sw/JCjc7c=

CHANGELOG.rst

@@ -1,3 +1,29 @@
+pytest-xdist 1.22.3 (2018-07-23)
+================================
+
+Bug Fixes
+---------
+
+- Fix issue of virtualized or containerized environments not reporting the number of CPUs correctly. (`#9 <https://github.com/pytest-dev/pytest-xdist/issues/9>`_)
+
+
+Trivial Changes
+---------------
+
+- Make all classes subclass from ``object`` and fix ``super()`` call in ``LoadFileScheduling``; (`#297 <https://github.com/pytest-dev/pytest-xdist/issues/297>`_)
+
+
+pytest-xdist 1.22.2 (2018-02-26)
+================================
+
+Bug Fixes
+---------
+
+- Add backward compatibility for ``slaveoutput`` attribute to
+  ``WorkerController`` instances. (`#285
+  <https://github.com/pytest-dev/pytest-xdist/issues/285>`_)
+
+
 pytest-xdist 1.22.1 (2018-02-19)
 ================================
 

HOWTORELEASE.rst

@@ -24,13 +24,13 @@ To publish a new release ``X.Y.Z``, the steps are as follows:
 
 #. Create a new branch named ``release-X.Y.Z`` from the latest ``master``.
 
-#. Install ``pytest-xdist`` and dev requirements in a virtualenv::
+#. Install ``tox`` in a virtualenv::
 
-    $ pip install -e . -U -r dev-requirements.txt
+    $ pip install tox
 
-#. Update ``CHANGELOG.rst`` file by running::
+#. Update the necessary files with::
 
-    $ towncrier --version X.Y.Z --yes
+    $ tox -e release -- X.Y.Z
 
 #. Commit and push the branch for review.
 

ISSUES.txt

@@ -19,7 +19,7 @@ tag: feature
 
 allow to run xdist own tests using its own mechanism.
 currently this doesn't work because the remote side
-has no py.test plugin.   How to configure/do
+has no pytest plugin.   How to configure/do
 register "xdist.plugin" on the remote side?
 
 see to avoid any "from _pytest" internal imports

README.rst

@@ -23,7 +23,7 @@
 xdist: pytest distributed testing plugin
 ========================================
 
-The `pytest-xdist`_ plugin extends py.test with some unique
+The `pytest-xdist`_ plugin extends pytest with some unique
 test execution modes:
 
 * test run parallelization_: if you have multiple CPUs or hosts you can use
@@ -32,14 +32,14 @@ test execution modes:
 
 
 * ``--looponfail``: run your tests repeatedly in a subprocess.  After each run
-  py.test waits until a file in your project changes and then re-runs
+  pytest waits until a file in your project changes and then re-runs
   the previously failing tests.  This is repeated until all tests pass
   after which again a full run is performed.
 
 * `Multi-Platform`_ coverage: you can specify different Python interpreters
   or different platforms and run tests in parallel on all of them.
 
-Before running tests remotely, ``py.test`` efficiently "rsyncs" your
+Before running tests remotely, ``pytest`` efficiently "rsyncs" your
 program source code to the remote place.  All test results
 are reported back and displayed to your local terminal.
 You may specify different Python versions and interpreters.
@@ -67,7 +67,7 @@ Speed up test runs by sending tests to multiple CPUs
 
 To send tests to multiple CPUs, type::
 
-    py.test -n NUM
+    pytest -n NUM
 
 Especially for longer running tests or tests requiring
 a lot of I/O this can lead to considerable speed ups. This option can
@@ -98,7 +98,7 @@ Running tests in a Python subprocess
 
 To instantiate a python3.5 subprocess and send tests to it, you may type::
 
-    py.test -d --tx popen//python=python3.5
+    pytest -d --tx popen//python=python3.5
 
 This will start a subprocess which is run with the ``python3.5``
 Python interpreter, found in your system binary lookup path.
@@ -129,10 +129,10 @@ tests that you can successfully run locally. And you
 have a ssh-reachable machine ``myhost``.  Then
 you can ad-hoc distribute your tests by typing::
 
-    py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
+    pytest -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
 
 This will synchronize your :code:`mypkg` package directory
-to an remote ssh account and then locally collect tests
+to a remote ssh account and then locally collect tests
 and send them to remote places for execution.
 
 You can specify multiple :code:`--rsyncdir` directories
@@ -140,11 +140,11 @@ to be sent to the remote side.
 
 .. note::
 
-  For py.test to collect and send tests correctly
+  For pytest to collect and send tests correctly
   you not only need to make sure all code and tests
   directories are rsynced, but that any test (sub) directory
   also has an :code:`__init__.py` file because internally
-  py.test references tests as a fully qualified python
+  pytest references tests as a fully qualified python
   module path.  **You will otherwise get strange errors**
   during setup of the remote side.
 
@@ -168,7 +168,7 @@ It will tell you that it starts listening on the default
 port.  You can now on your home machine specify this
 new socket host with something like this::
 
-    py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
+    pytest -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
 
 
 .. _`atonce`:
@@ -180,7 +180,7 @@ Running tests on many platforms at once
 
 The basic command to run tests on multiple platforms is::
 
-    py.test --dist=each --tx=spec1 --tx=spec2
+    pytest --dist=each --tx=spec1 --tx=spec2
 
 If you specify a windows host, an OSX host and a Linux
 environment this command will send each tests to all
@@ -242,7 +242,7 @@ You can also add default environments like this:
 
 and then just type::
 
-    py.test --dist=each
+    pytest --dist=each
 
 to run tests in each of the environments.
 

dev-requirements.txt

@@ -1 +0,0 @@
-towncrier

example/boxed.txt

@@ -25,7 +25,7 @@ to run each test in a controlled subprocess.  Here is a basic example::
 
 If you run this with::
 
-    $ py.test -n1
+    $ pytest -n1
     =========================== test session starts ============================
     platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
     plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
@@ -46,7 +46,7 @@ You'll see that a couple of tests are reported as crashing, indicated
 by lower-case ``f`` and the respective failure summary. You can also use
 the xdist-provided parallelization feature to speed up your testing::
 
-    $ py.test -n3
+    $ pytest -n3
     =========================== test session starts ============================
     platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
     plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov

example/loadscope/tox.ini

@@ -8,7 +8,7 @@ passenv = http_proxy https_proxy
 deps = -rrequirements.txt
 changedir = {envtmpdir}
 commands =
-    py.test -s -v \
+    pytest -s -v \
         --doctest-modules \
         --junitxml=tests.xml \
         --dist=loadscope \

setup.py

@@ -6,7 +6,7 @@ install_requires = ['execnet>=1.1', 'pytest>=3.0.0', 'pytest-forked']
 setup(
     name="pytest-xdist",
     use_scm_version={'write_to': 'xdist/_version.py'},
-    description='py.test xdist plugin for distributed testing'
+    description='pytest xdist plugin for distributed testing'
                 ' and loop-on-failing modes',
     long_description=open('README.rst').read(),
     license='MIT',

testing/acceptance_test.py

@@ -1,4 +1,6 @@
+import os
 import re
+import textwrap
 
 import py
 import pytest
@@ -54,8 +56,8 @@ class TestDistribution:
     def test_n1_skip(self, testdir):
         p1 = testdir.makepyfile("""
             def test_skip():
-                import py
-                py.test.skip("myreason")
+                import pytest
+                pytest.skip("myreason")
         """)
         result = testdir.runpytest(p1, "-n1")
         assert result.ret == 0
@@ -78,7 +80,7 @@ class TestDistribution:
     def test_manytests_to_one_popen(self, testdir):
         p1 = testdir.makepyfile(
             """
-                import py
+                import pytest
                 def test_fail0():
                     assert 0
                 def test_fail1():
@@ -86,7 +88,7 @@ class TestDistribution:
                 def test_ok():
                     pass
                 def test_skip():
-                    py.test.skip("hello")
+                    pytest.skip("hello")
             """, )
         result = testdir.runpytest(p1, "-v", '-d', '--tx=popen', '--tx=popen')
         result.stdout.fnmatch_lines([
@@ -124,7 +126,7 @@ class TestDistribution:
     def test_dist_ini_specified(self, testdir):
         p1 = testdir.makepyfile(
             """
-                import py
+                import pytest
                 def test_fail0():
                     assert 0
                 def test_fail1():
@@ -132,7 +134,7 @@ class TestDistribution:
                 def test_ok():
                     pass
                 def test_skip():
-                    py.test.skip("hello")
+                    pytest.skip("hello")
             """, )
         testdir.makeini("""
             [pytest]
@@ -145,13 +147,13 @@ class TestDistribution:
         ])
         assert result.ret == 1
 
-    @py.test.mark.xfail("sys.platform.startswith('java')", run=False)
+    @pytest.mark.xfail("sys.platform.startswith('java')", run=False)
     def test_dist_tests_with_crash(self, testdir):
-        if not hasattr(py.std.os, 'kill'):
-            py.test.skip("no os.kill")
+        if not hasattr(os, 'kill'):
+            pytest.skip("no os.kill")
 
         p1 = testdir.makepyfile("""
-                import py
+                import pytest
                 def test_fail0():
                     assert 0
                 def test_fail1():
@@ -159,7 +161,7 @@ class TestDistribution:
                 def test_ok():
                     pass
                 def test_skip():
-                    py.test.skip("hello")
+                    pytest.skip("hello")
                 def test_crash():
                     import time
                     import os
@@ -299,7 +301,7 @@ class TestDistEach:
         assert not result.ret
         result.stdout.fnmatch_lines(["*2 pass*"])
 
-    @py.test.mark.xfail(
+    @pytest.mark.xfail(
         run=False,
         reason="other python versions might not have py.test installed")
     def test_simple_diffoutput(self, testdir):
@@ -307,7 +309,7 @@ class TestDistEach:
         for name in ("python2.5", "python2.6"):
             interp = py.path.local.sysfind(name)
             if interp is None:
-                py.test.skip("%s not found" % name)
+                pytest.skip("%s not found" % name)
             interpreters.append(interp)
 
         testdir.makepyfile(
@@ -330,11 +332,11 @@ class TestDistEach:
 class TestTerminalReporting:
     def test_pass_skip_fail(self, testdir):
         testdir.makepyfile("""
-            import py
+            import pytest
             def test_ok():
                 pass
             def test_skip():
-                py.test.skip("xx")
+                pytest.skip("xx")
             def test_func():
                 assert 0
         """)
@@ -421,7 +423,7 @@ def test_teardownfails_one_function(testdir):
         ["*def teardown_function(function):*", "*1 passed*1 error*"])
 
 
-@py.test.mark.xfail
+@pytest.mark.xfail
 def test_terminate_on_hangingnode(testdir):
     p = testdir.makeconftest("""
         def pytest_sessionfinish(session):
@@ -585,7 +587,7 @@ def test_skipping(testdir):
 
 def test_issue34_pluginloading_in_subprocess(testdir):
     testdir.tmpdir.join("plugin123.py").write(
-        py.code.Source("""
+        textwrap.dedent("""
         def pytest_namespace():
             return {'sample_variable': 'testing'}
     """))

testing/test_plugin.py

@@ -31,7 +31,9 @@ def test_dist_options(testdir):
 
 def test_auto_detect_cpus(testdir, monkeypatch):
     import os
-    if hasattr(os, 'cpu_count'):
+    if hasattr(os, 'sched_getaffinity'):
+        monkeypatch.setattr(os, 'sched_getaffinity', lambda _pid: set(range(99)))
+    elif hasattr(os, 'cpu_count'):
         monkeypatch.setattr(os, 'cpu_count', lambda: 99)
     else:
         import multiprocessing

testing/test_remote.py

@@ -1,4 +1,5 @@
 import py
+import pprint
 import pytest
 from xdist.workermanage import WorkerController, unserialize_report
 from xdist.remote import serialize_report
@@ -14,7 +15,7 @@ def check_marshallable(d):
     try:
         marshal.dumps(d)
     except ValueError:
-        py.std.pprint.pprint(d)
+        pprint.pprint(d)
         raise ValueError("not marshallable")
 
 
@@ -344,7 +345,7 @@ class TestWorkerInteractor:
         for data in worker.slp.channel:
             worker.slp.process_from_remote(data)
         worker.slp.process_from_remote(worker.slp.ENDMARK)
-        py.std.pprint.pprint(hookrec.hookrecorder.calls)
+        pprint.pprint(hookrec.hookrecorder.calls)
         hookrec.hookrecorder.contains([
             ("pytest_collectstart", "collector.fspath == aaa"),
             ("pytest_pycollect_makeitem", "name == 'test_func'"),

testing/test_slavemanage.py

@@ -1,5 +1,6 @@
 import py
 import pytest
+import textwrap
 import execnet
 from _pytest.pytester import HookRecorder
 from xdist import workermanage, newhooks
@@ -198,7 +199,7 @@ class TestNodeManager:
         source.ensure("dir1", "somefile", dir=1)
         dir2.ensure("hello")
         source.ensure("bogusdir", "file")
-        source.join("tox.ini").write(py.std.textwrap.dedent("""
+        source.join("tox.ini").write(textwrap.dedent("""
             [pytest]
             rsyncdirs=dir1/dir2
         """))
@@ -217,7 +218,7 @@ class TestNodeManager:
         dir2.ensure("hello")
         source.ensure("foo", "bar")
         source.ensure("bar", "foo")
-        source.join("tox.ini").write(py.std.textwrap.dedent("""
+        source.join("tox.ini").write(textwrap.dedent("""
             [pytest]
             rsyncdirs = dir1 dir5
             rsyncignore = dir1/dir2 dir5/dir6 foo*

tox.ini

@@ -27,7 +27,7 @@ platform=
 commands=
   # always clean to avoid code unmarshal mismatch on old python/pytest
   py.cleanup -aq
-  py.test {posargs}
+  pytest {posargs}
 
 [testenv:flakes]
 changedir=
@@ -40,7 +40,18 @@ deps = readme
 skip_install = true
 commands = python setup.py check -r -s
 
+[testenv:release]
+changedir=
+decription = do a release, required posarg of the version number
+basepython = python3.6
+skipsdist = True
+usedevelop = True
+passenv = *
+deps =
+    towncrier
+commands =
+    towncrier --version {posargs} --yes
 
 [pytest]
 addopts = -rsfxX
-;; hello
+

xdist/dsession.py

@@ -17,8 +17,8 @@ class Interrupted(KeyboardInterrupt):
     """ signals an immediate interruption. """
 
 
-class DSession:
-    """A py.test plugin which runs a distributed test session
+class DSession(object):
+    """A pytest plugin which runs a distributed test session
 
     At the beginning of the test session this creates a NodeManager
     instance which creates and starts all nodes.  Nodes then emit
@@ -26,7 +26,7 @@ class DSession:
     methods.
 
     Once a node is started it will automatically start running the
-    py.test mainloop with some custom hooks.  This means a node
+    pytest mainloop with some custom hooks.  This means a node
     automatically starts collecting tests.  Once tests are collected
     it will wait for instructions.
     """
@@ -316,7 +316,7 @@ class DSession:
         self.config.hook.pytest_runtest_logreport(report=rep)
 
 
-class TerminalDistReporter:
+class TerminalDistReporter(object):
     def __init__(self, config):
         self.config = config
         self.tr = config.pluginmanager.getplugin("terminalreporter")

xdist/looponfail.py

@@ -1,5 +1,5 @@
 """
-    Implement -f aka looponfailing for py.test.
+    Implement -f aka looponfailing for pytest.
 
     NOTE that we try to avoid loading and depending on application modules
     within the controlling process (the one that starts repeatedly test
@@ -10,6 +10,7 @@
 import py
 import pytest
 import sys
+import time
 import execnet
 
 
@@ -157,7 +158,7 @@ def init_worker_session(channel, args, option_dict):
     WorkerFailSession(config, channel).main()
 
 
-class WorkerFailSession:
+class WorkerFailSession(object):
     def __init__(self, config, channel):
         self.config = config
         self.channel = channel
@@ -211,7 +212,7 @@ class WorkerFailSession:
         self.channel.send((trails, failreports, self.collection_failed))
 
 
-class StatRecorder:
+class StatRecorder(object):
     def __init__(self, rootdirlist):
         self.rootdirlist = rootdirlist
         self.statcache = {}
@@ -228,7 +229,7 @@ class StatRecorder:
             changed = self.check()
             if changed:
                 return
-            py.std.time.sleep(checkinterval)
+            time.sleep(checkinterval)
 
     def check(self, removepycfiles=True):  # noqa, too complex
         changed = False

xdist/plugin.py

@@ -5,9 +5,16 @@ import pytest
 def parse_numprocesses(s):
     if s == 'auto':
         try:
-            from os import cpu_count
+            from os import sched_getaffinity
         except ImportError:
-            from multiprocessing import cpu_count
+            try:
+                from os import cpu_count
+            except ImportError:
+                from multiprocessing import cpu_count
+        else:
+            def cpu_count():
+                return len(sched_getaffinity(0))
+
         try:
             n = cpu_count()
         except NotImplementedError:

xdist/remote.py

@@ -14,7 +14,7 @@ import _pytest.hookspec
 import pytest
 
 
-class WorkerInteractor:
+class WorkerInteractor(object):
     def __init__(self, config, channel):
         self.config = config
         self.workerid = config.workerinput.get('workerid', "?")

xdist/scheduler/each.py

@@ -4,7 +4,7 @@ from xdist.workermanage import parse_spec_config
 from xdist.report import report_collection_diff
 
 
-class EachScheduling:
+class EachScheduling(object):
     """Implement scheduling of test items on all nodes
 
     If a node gets added after the test run is started then it is

xdist/scheduler/filescope.py

@@ -20,7 +20,7 @@ class LoadFileScheduling(LoadScopeScheduling):
     This class behaves very much like LoadScopeScheduling, but with a file-level scope.
     """
 
-    def __init(self, config, log=None):
+    def __init__(self, config, log=None):
         super(LoadFileScheduling, self).__init__(config, log)
         if log is None:
             self.log = Producer('loadfilesched')

xdist/scheduler/load.py

@@ -7,7 +7,7 @@ from xdist.workermanage import parse_spec_config
 from xdist.report import report_collection_diff
 
 
-class LoadScheduling:
+class LoadScheduling(object):
     """Implement load scheduling across nodes.
 
     This distributes the tests collected across all nodes so each test

xdist/scheduler/loadscope.py

@@ -6,7 +6,7 @@ from xdist.report import report_collection_diff
 from xdist.workermanage import parse_spec_config
 
 
-class LoadScopeScheduling:
+class LoadScopeScheduling(object):
     """Implement load scheduling across nodes, but grouping test by scope.
 
     This distributes the tests collected across all nodes so each test is run

xdist/workermanage.py

@@ -1,6 +1,7 @@
 import fnmatch
 import os
 import re
+import threading
 
 import py
 import pytest
@@ -32,7 +33,7 @@ class NodeManager(object):
 
     def __init__(self, config, specs=None, defaultchdir="pyexecnetcache"):
         self.config = config
-        self._nodesready = py.std.threading.Event()
+        self._nodesready = threading.Event()
         self.trace = self.config.trace.get("nodemanager")
         self.group = execnet.Group()
         if specs is None:
@@ -184,7 +185,7 @@ class HostRSync(execnet.RSync):
 
 
 def make_reltoroot(roots, args):
-    # XXX introduce/use public API for splitting py.test args
+    # XXX introduce/use public API for splitting pytest args
     splitcode = "::"
     result = []
     for arg in args:
@@ -307,7 +308,7 @@ class WorkerController(object):
                 self.notify_inproc(eventname, node=self, **kwargs)
             elif eventname == "workerfinished":
                 self._down = True
-                self.workeroutput = kwargs['workeroutput']
+                self.workeroutput = self.slaveoutput = kwargs['workeroutput']
                 self.notify_inproc("workerfinished", node=self)
             elif eventname in ("logstart", "logfinish"):
                 self.notify_inproc(eventname, node=self, **kwargs)