Release 1.31.0

* No idea what I'm doing.

* Extend integration test to cover ac3f6a78c7401aba9b62c2c67f280a129ae84eaf.

* Remove useless output to stop failing other tests.

* Reformat.

* Upgrade virtualenv to see if it fixes py34 builds

* Add CHANGELOG entry

* Make foobarplugin part of the test only

* Fix language_version in pre-commit for rst

* Fix test in Python 2

Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>

.appveyor.yml

@@ -5,17 +5,18 @@ environment:
   - TOXENV: "py35-pytestlatest"
   - TOXENV: "py36-pytestlatest"
   - TOXENV: "py37-pytestlatest"
-  - TOXENV: "py37-pytestmaster"
+  - TOXENV: "py38-pytestlatest"
-  - TOXENV: "py37-pytestfeatures"
+  - TOXENV: "py38-pytestmaster"
+  - TOXENV: "py38-pytestfeatures"
 
 install:
-  - C:\Python37\python -m pip install -U pip setuptools
+  - C:\Python38\python -m pip install -U pip setuptools virtualenv
-  - C:\Python37\python -m pip install -U tox setuptools_scm
+  - C:\Python38\python -m pip install -U tox setuptools_scm
 
 build: false  # Not a C# project, build stuff at the test step instead.
 
 test_script:
-  - C:\Python37\python -m tox
+  - C:\Python38\python -m tox
 
 # We don't deploy anything on tags with AppVeyor, we use Travis instead, so we
 # might as well save resources

.pre-commit-config.yaml

@@ -21,4 +21,4 @@ repos:
         files: ^(CHANGELOG.rst|HOWTORELEASE.rst|README.rst|changelog/.*)$
         language: python
         additional_dependencies: [pygments, restructuredtext_lint]
-        python_version: python3.7
+        language_version: python3.7

.travis.yml

@@ -30,8 +30,8 @@ jobs:
       cache:
         directories:
           - $HOME/.cache/pre-commit
-    - python: '3.7'
+    - python: '3.8'
-      env: TOXENV=py37-pytestlatest
+      env: TOXENV=py38-pytestlatest
     - python: '2.7'
       env: TOXENV=py27-pytestlatest
 
@@ -43,12 +43,14 @@ jobs:
     - python: "3.6"
       env: TOXENV=py36-pytestlatest
     - python: "3.7"
-      env: TOXENV=py37-pytestmaster
+      env: TOXENV=py37-pytestlatest
-    - python: "3.7"
+    - python: "3.8"
-      env: TOXENV=py37-pytestfeatures
+      env: TOXENV=py38-pytestmaster
+    - python: "3.8"
+      env: TOXENV=py38-pytestfeatures
 
     - stage: deploy
-      python: '3.7'
+      python: '3.8'
       env:
       install: pip install -U setuptools setuptools_scm
       script: skip

CHANGELOG.rst

@@ -1,3 +1,42 @@
+pytest-xdist 1.31.0 (2019-12-19)
+================================
+
+Features
+--------
+
+- `#486 <https://github.com/pytest-dev/pytest-xdist/issues/486>`_: Add support for Python 3.8.
+
+
+Bug Fixes
+---------
+
+- `#491 <https://github.com/pytest-dev/pytest-xdist/issues/491>`_: Fix regression that caused custom plugin command-line arguments to be discarded when using ``--tx`` mode.
+
+
+
+pytest-xdist 1.30.0 (2019-10-01)
+================================
+
+Features
+--------
+
+- `#448 <https://github.com/pytest-dev/pytest-xdist/issues/448>`_: Initialization between workers and master nodes is now more consistent, which fixes a number of
+  long-standing issues related to startup with the ``-c`` option.
+
+  Issues:
+
+  * `#6 <https://github.com/pytest-dev/pytest-xdist/issues/6>`__: Poor interaction between ``-n#`` and ``-c X.cfg``
+  * `#445 <https://github.com/pytest-dev/pytest-xdist/issues/445>`__: pytest-xdist is not reporting the same nodeid as pytest does
+
+  This however only works with **pytest 5.1 or later**, as it required changes in pytest itself.
+
+
+Bug Fixes
+---------
+
+- `#467 <https://github.com/pytest-dev/pytest-xdist/issues/467>`_: Fix crash issues related to running xdist with the terminal plugin disabled.
+
+
 pytest-xdist 1.29.0 (2019-06-14)
 ================================
 
@@ -401,39 +440,6 @@ Improved Documentation
 - Added ``HOWTORELEASE`` documentation. (#155)
 
 
-..
-    You should *NOT* be adding new change log entries to this file, this
-    file is managed by towncrier. You *may* edit previous change logs to
-    fix problems like typo corrections or such.
-    To add a new change log entry, please see
-    https://pip.pypa.io/en/latest/development/#adding-a-news-entry
-    We named the news folder ``changelog``
-
-.. towncrier release notes start
-
-pytest-xdist 1.30.0 (2019-10-01)
-================================
-
-Features
---------
-
-- `#448 <https://github.com/pytest-dev/pytest-xdist/issues/448>`_: Initialization between workers and master nodes is now more consistent, which fixes a number of
-  long-standing issues related to startup with the ``-c`` option.
-
-  Issues:
-
-  * `#6 <https://github.com/pytest-dev/pytest-xdist/issues/6>`__: Poor interaction between ``-n#`` and ``-c X.cfg``
-  * `#445 <https://github.com/pytest-dev/pytest-xdist/issues/445>`__: pytest-xdist is not reporting the same nodeid as pytest does
-
-  This however only works with **pytest 5.1 or later**, as it required changes in pytest itself.
-
-
-Bug Fixes
----------
-
-- `#467 <https://github.com/pytest-dev/pytest-xdist/issues/467>`_: Fix crash issues related to running xdist with the terminal plugin disabled.
-
-
 1.17.0
 ------
 

README.rst

@@ -95,6 +95,59 @@ any guaranteed order, but you can control this with these options:
   in version ``1.21``.
 
 
+Making session-scoped fixtures execute only once
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``pytest-xdist`` is designed so that each worker process will perform its own collection and execute
+a subset of all tests. This means that tests in different processes requesting a high-level
+scoped fixture (for example ``session``) will execute the fixture code more than once, which
+breaks expectations and might be undesired in certain situations.
+
+While ``pytest-xdist`` does not have a builtin support for ensuring a session-scoped fixture is
+executed exactly once, this can be achieved by using a lock file for inter-process communication.
+
+The example below needs to execute the fixture ``session_data`` only once (because it is
+resource intensive, or needs to execute only once to define configuration options, etc), so it makes
+use of a `FileLock <https://pypi.org/project/filelock/>`_ to produce the fixture data only once
+when the first process requests the fixture, while the other processes will then read
+the data from a file.
+
+Here is the code:
+
+.. code-block:: python
+
+    import json
+
+    import pytest
+    from filelock import FileLock
+
+
+    @pytest.fixture(scope="session")
+    def session_data(tmp_path_factory, worker_id):
+        if not worker_id:
+            # not executing in with multiple workers, just produce the data and let
+            # pytest's fixture caching do its job
+            return produce_expensive_data()
+
+        # get the temp directory shared by all workers
+        root_tmp_dir = tmp_path_factory.getbasetemp().parent
+
+        fn = root_tmp_dir / "data.json"
+        with FileLock(str(fn) + ".lock"):
+            if fn.is_file():
+                data = json.loads(fn.read_text())
+            else:
+                data = produce_expensive_data()
+                fn.write_text(json.dumps(data))
+        return data
+
+
+The example above can also be use in cases a fixture needs to execute exactly once per test session, like
+initializing a database service and populating initial tables.
+
+This technique might not work for every case, but should be a starting point for many situations
+where executing a high-scope fixture exactly once is important.
+
 Running tests in a Python subprocess
 ------------------------------------
 

setup.py

@@ -46,5 +46,6 @@ setup(
         "Programming Language :: Python :: 3.5",
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
     ],
 )

src/xdist/workermanage.py

@@ -190,6 +190,7 @@ def make_reltoroot(roots, args):
         parts = arg.split(splitcode)
         fspath = py.path.local(parts[0])
         if not fspath.exists():
+            result.append(arg)
             continue
         for root in roots:
             x = fspath.relto(root)

testing/acceptance_test.py

@@ -186,16 +186,46 @@ class TestDistribution:
         )
         assert result.ret == 1
 
-    def test_distribution_rsyncdirs_example(self, testdir):
+    def test_distribution_rsyncdirs_example(self, testdir, monkeypatch):
+        # use a custom plugin that has a custom command-line option to ensure
+        # this is propagated to workers (see #491)
+        testdir.makepyfile(
+            **{
+                "myplugin/src/foobarplugin.py": """
+            from __future__ import print_function
+
+            import os
+            import sys
+            import pytest
+
+            def pytest_addoption(parser):
+                parser.addoption("--foobar", action="store", dest="foobar_opt")
+
+            @pytest.mark.tryfirst
+            def pytest_load_initial_conftests(early_config):
+                opt = early_config.known_args_namespace.foobar_opt
+                print("--foobar=%s active! [%s]" % (opt, os.getpid()), file=sys.stderr)
+            """
+            }
+        )
+        assert (testdir.tmpdir / "myplugin/src/foobarplugin.py").check(file=1)
+        monkeypatch.setenv(
+            "PYTHONPATH", str(testdir.tmpdir / "myplugin/src"), prepend=os.pathsep
+        )
+
         source = testdir.mkdir("source")
         dest = testdir.mkdir("dest")
         subdir = source.mkdir("example_pkg")
         subdir.ensure("__init__.py")
         p = subdir.join("test_one.py")
         p.write("def test_5():\n  assert not __file__.startswith(%r)" % str(p))
-        result = testdir.runpytest(
+        result = testdir.runpytest_subprocess(
             "-v",
             "-d",
+            "-s",
+            "-pfoobarplugin",
+            "--foobar=123",
+            "--dist=load",
             "--rsyncdir=%(subdir)s" % locals(),
             "--tx=popen//chdir=%(dest)s" % locals(),
             p,
@@ -209,6 +239,7 @@ class TestDistribution:
                 "*1 passed*",
             ]
         )
+        result.stderr.fnmatch_lines(["--foobar=123 active! *"])
         assert dest.join(subdir.basename).check(dir=1)
 
     def test_backward_compatibility_worker_terminology(self, testdir):
@@ -784,7 +815,7 @@ class TestWarnings:
             )
         )
         result = testdir.runpytest(n)
-        result.stdout.fnmatch_lines(["*this is a warning*", "*1 passed, 1 warnings*"])
+        result.stdout.fnmatch_lines(["*this is a warning*", "*1 passed, 1 warning*"])
 
     @pytest.mark.parametrize("n", ["-n0", "-n1"])
     def test_custom_subclass(self, testdir, n):
@@ -808,7 +839,7 @@ class TestWarnings:
         )
         testdir.syspathinsert()
         result = testdir.runpytest(n)
-        result.stdout.fnmatch_lines(["*MyWarning*", "*1 passed, 1 warnings*"])
+        result.stdout.fnmatch_lines(["*MyWarning*", "*1 passed, 1 warning*"])
 
     @pytest.mark.parametrize("n", ["-n0", "-n1"])
     def test_unserializable_arguments(self, testdir, n):
@@ -825,7 +856,7 @@ class TestWarnings:
         )
         testdir.syspathinsert()
         result = testdir.runpytest(n)
-        result.stdout.fnmatch_lines(["*UserWarning*foo.txt*", "*1 passed, 1 warnings*"])
+        result.stdout.fnmatch_lines(["*UserWarning*foo.txt*", "*1 passed, 1 warning*"])
 
     @pytest.mark.parametrize("n", ["-n0", "-n1"])
     def test_unserializable_warning_details(self, testdir, n):
@@ -857,7 +888,7 @@ class TestWarnings:
         testdir.syspathinsert()
         result = testdir.runpytest(n)
         result.stdout.fnmatch_lines(
-            ["*ResourceWarning*unclosed*", "*1 passed, 1 warnings*"]
+            ["*ResourceWarning*unclosed*", "*1 passed, 1 warning*"]
         )
 
 

tox.ini

@@ -1,8 +1,8 @@
 [tox]
 envlist=
   linting
-  py{27,34,35,36,37}-pytestlatest
+  py{27,34,35,36,37,38}-pytestlatest
-  py37-pytest{master,features}
+  py38-pytest{master,features}
 
 [testenv]
 passenv = USER USERNAME