Compare commits

...

72 Commits

Author SHA1 Message Date
Bruno Oliveira
cee78e9b20 CHANGELOG for 1.18.0 2017-06-26 15:16:48 -03:00
Bruno Oliveira
6e1154881d Merge pull request #164 from timyhou/master
Issue #130
2017-06-23 19:39:27 -03:00
Ronny Pfannschmidt
a96f6c9038 Merge pull request #168 from kain88-de/numprocesses-option
add '--numprocesses' as long option for '-n'
2017-06-23 09:45:46 +02:00
Bruno Oliveira
47b65716a3 Fix test for --numprocess flag 2017-06-22 21:47:34 -03:00
Max Linke
dbaf49cd69 add '--numprocesses' as long option for '-n'
This is more verbose in scripts that run pytest-xdist. It can also help
in some ci-helper scripts like to auto detect if  pytest-xdist should be
installed.
2017-06-22 21:53:16 +02:00
Bruno Oliveira
41c00ca47e Clarify PR template
Clarify that a new "file" should be added to "changelog" folder.
2017-06-22 16:41:27 -03:00
thou
f0dee10a0d Fix failing test since spawn.out does not contain "LOOPONFAILING" in pytest >= 3 2017-06-21 13:37:03 -05:00
Tim Hou
6256608669 Update tox and appveyor for CI tests to exclude pytest < 3
Update the install requires versions for pytest
2017-06-21 13:37:02 -05:00
thou
4a1f1d879b Updated serialize_report and unserialize_report to pass entire exception info using ReprExceptionInfo.
This addresses issue #130.
2017-06-21 13:36:58 -05:00
Ronny Pfannschmidt
2094157950 Merge pull request #162 from nicoddemus/let-setuptools-scm-manage-manifest
Let setuptools_scm manage the files that go into the package
2017-06-14 13:43:46 +02:00
Bruno Oliveira
7473683cdb Let setuptools_scm manage the files that go into the package
As discussed in #161, setuptools_scm will automatically
add all version controlled files to the package, so
MANIFEST.in is no longer needed

Fix #161
2017-06-14 07:58:57 -03:00
Bruno Oliveira
c48834aa69 Merge pull request #159 from nicoddemus/release-1.17.1
Release 1.17.1
2017-06-10 10:59:42 -03:00
Bruno Oliveira
6d9d6c6a34 Update HOWTORELEASE docs 2017-06-10 10:03:10 -03:00
Bruno Oliveira
1542aff326 Update CHANGELOG 2017-06-10 10:02:55 -03:00
Bruno Oliveira
cc95cf26e1 Skip uploading docs when publishing to PyPI
This should also solve the problem of not publishing wheels
2017-06-10 10:02:43 -03:00
Bruno Oliveira
eb90a7c82c Merge pull request #158 from nicoddemus/revert-pr-125
Revert PR 125 due to 1.17.0 breakage
2017-06-10 09:55:51 -03:00
Bruno Oliveira
afe8bad9ff Add news fragment
Fix #157
2017-06-10 09:44:55 -03:00
Bruno Oliveira
95a62460aa Revert "Mark test as complete at teardown."
This reverts commit 82b992a5fd.
2017-06-10 09:43:59 -03:00
Bruno Oliveira
8e2fdeee3e Revert "- Reinforce test_crashing_item to match test name instead of test"
This reverts commit 99f667380f.
2017-06-10 09:43:59 -03:00
Bruno Oliveira
fae1e02c1d Revert "Fix missing blank line."
This reverts commit 602694b817.
2017-06-10 09:43:59 -03:00
Ronny Pfannschmidt
3440f08a40 Merge pull request #156 from nicoddemus/release-howto-towncrier
Add towncrier and HOWTORELEASE docs
2017-06-10 09:00:47 +02:00
Bruno Oliveira
a9b064d992 Add HOWTORELEASE document
Fix #155
2017-06-09 19:50:54 -03:00
Bruno Oliveira
47875cbea9 Add towncrier for CHANGELOG management
Fix #154
2017-06-09 19:49:06 -03:00
Bruno Oliveira
e59c609b59 Merge pull request #153 from reginaldl/master
Fix CHANGELOG for issue #124
2017-06-09 18:52:11 -03:00
reginaldl
de257e6b83 Fix CHANGELOG for issue #124 2017-06-09 10:51:12 -07:00
Ronny Pfannschmidt
51b5a5ace4 Merge pull request #140 from xoviat/patch-1
travis: use tox-travis
2017-05-17 10:34:15 +02:00
Ronny Pfannschmidt
d86ce20d28 Merge pull request #144 from EdgeCaseLabs/master
Correcting broken URI in documentation.
2017-05-16 15:21:40 +02:00
Wes Thomas
58750f5765 Correcting broken URI in documentation. 2017-05-11 15:29:39 -05:00
xoviat
0789605d30 fixup py36 tests 2017-05-08 09:12:47 -05:00
xoviat
2b57d85550 travis: use tox-travis 2017-05-08 09:11:32 -05:00
Omer Katz
0ce4f32315 Run tests on Python 3.6 on windows. 2017-05-08 09:11:32 -05:00
Omer Katz
de7a45ef40 Test with Python 3.6 as well. 2017-05-08 09:11:32 -05:00
Ronny Pfannschmidt
e83aae6351 Merge pull request #125 from reginaldl/master
Mark test as complete at teardown.
2017-05-05 22:43:53 +02:00
Ronny Pfannschmidt
a23f572b21 Merge pull request #138 from nicoddemus/anaconda-ver
Add a badge for the anaconda package
2017-05-05 07:11:52 +02:00
Bruno Oliveira
36c0e68c23 Add a badge for the anaconda package
Also bring Travis and AppVeyor badges near each other
2017-04-24 17:04:59 -03:00
reginaldl
c79a98c624 Add note about #124 to CHANGELOG. 2017-03-07 10:15:52 -08:00
reginaldl
602694b817 Fix missing blank line. 2017-02-23 15:14:24 -08:00
reginaldl
99f667380f - Reinforce test_crashing_item to match test name instead of test
file.
- Add `test_crashing_item_teardown` to validate crash behavior at
  teardown.
2017-02-23 15:01:48 -08:00
reginaldl
82b992a5fd Mark test as complete at teardown. 2017-02-23 11:31:37 -08:00
Ronny Pfannschmidt
80f187e3a6 Merge pull request #122 from wronglink/custom_scheduler_notes
Added notes about new pytest_xdist_make_scheduler hook
2017-02-21 12:26:59 +01:00
Michael Elovskikh
2c8efb5785 Added notes about new pytest_xdist_make_scheduler hook 2017-02-21 01:39:03 +05:00
Bruno Oliveira
a867f71c7a Merge pull request #89 from wronglink/custom_scheduler
[WIP] Allow custom scheduler class implementation
2017-02-16 09:40:36 -02:00
Ronny Pfannschmidt
f3098f00eb Merge pull request #120 from KKoukiou/worker_id
Add worker_id into TestReport
2017-02-16 10:24:41 +01:00
Michael Elovskikh
4ed8161ef8 Renamed scheduller remove_item method to mark_test_complete 2017-02-15 23:54:23 +05:00
Michael Elovskikh
c809406ef6 Refactor scheduller interface, make more properties instead of methods 2017-02-15 11:43:56 +05:00
Michael Elovskikh
a99e6e7ca8 Clean up scheduller interfaces 2017-02-15 00:21:26 +05:00
Michael Elovskikh
6200123fac Made EachScheduling.numnodes countable from config 2017-02-14 20:19:55 +05:00
Michael Elovskikh
a2aaf9bf04 Rename init_distribute() to more clear schedule() 2017-02-14 20:19:55 +05:00
Michael Elovskikh
27ffbbb4f7 Unified scheduler class interface 2017-02-14 20:19:55 +05:00
Michael Elovskikh
81003df660 Fix codestyle 2017-02-14 20:19:55 +05:00
Michael Elovskikh
753f9241a7 Support older pytest versions 2017-02-14 20:19:55 +05:00
Michael Elovskikh
cf863fd38a Remove unnecessary test 2017-02-14 20:19:55 +05:00
Michael Elovskikh
2ff2772521 Use pytest_xdist_make_scheduler hook to set up scheduller instance 2017-02-14 20:19:55 +05:00
Michael Elovskikh
3165b78936 Test custom scheduler plugin config 2017-02-14 20:19:55 +05:00
Michael Elovskikh
ffcbe44685 Fix codestyle 2017-02-14 20:19:55 +05:00
Michael Elovskikh
d5ba3804bb Use __import__ instead of importlib for py26 support 2017-02-14 20:19:55 +05:00
Michael Elovskikh
7d099b65c2 Allow custom scheduler class 2017-02-14 20:19:55 +05:00
Katerina Koukiou
63431e749e Add worker_id into TestReport 2017-02-14 15:45:06 +01:00
Ronny Pfannschmidt
c3c75e405a Merge pull request #119 from danielquinn/patch-1
Fixed typo
2017-02-08 11:33:57 +01:00
Daniel Quinn
daa80a2708 Fixed typo 2017-02-08 10:29:24 +00:00
Bruno Oliveira
55a410af46 Merge pull request #116 from allanlewis/dest-default-type
xdist.plugin: Remove unnecessary `type` for --dist
2017-01-23 12:49:57 -02:00
Allan Lewis
1c3cd73f23 xdist.plugin: Remove unnecessary type for --dist
1e18f25 corrected an issue with the `type` specification for the `--dist`
argument, setting it to `str`. However, this is the default so there's little
value in specifying it.
2017-01-23 11:22:45 +00:00
Bruno Oliveira
68470d3e8f Merge pull request #108 from pombredanne/patch-1
Ensure the LICENSE is included in built wheels
2017-01-10 20:16:46 -02:00
Philippe Ombredanne
7091a58588 Ensure the LICENSE is included in built wheels
Otherwise it is not included by default as wheels do not honor the MANIFEST.in

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
2017-01-10 19:26:01 +01:00
Ronny Pfannschmidt
dc646cebcc Merge pull request #107 from iansharkey/master
Fix for eventname test
2017-01-07 09:06:34 +01:00
iansharkey
d01ef8d1be Merge pull request #1 from iansharkey/eventname-patch
Fix for eventname test
2017-01-06 20:48:18 -05:00
iansharkey
d91819114a Fix for eventname test
Fix process_from_remote to check eventname against a single-element tuple instead of a string sequence.
2017-01-06 20:47:34 -05:00
Ronny Pfannschmidt
13e601a884 Merge pull request #100 from nicoddemus/pytest-3
Test with pytest 3.0
2016-12-06 11:24:44 +01:00
Bruno Oliveira
d174e2bf89 Update CHANGELOG about supporing pytest 2.7 or later 2016-12-05 20:12:29 -02:00
Bruno Oliveira
e5d8064534 Test using pytest 3.0
Fix #99
2016-12-05 20:12:29 -02:00
Bruno Oliveira
780cd0e04f Merge pull request #94 from torcolvin/warningremove
Don't show warning from pytest 3
2016-10-24 16:15:19 -02:00
Tor Colvin
1e18f25cd0 Don't show warning from pytest 3 2016-10-24 11:28:54 -04:00
25 changed files with 565 additions and 228 deletions

17
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,17 @@
Thanks for submitting a PR, your contribution is really appreciated!
Here's a quick checklist that should be present in PRs:
- [ ] Make sure to include reasonable tests for your change if necessary
- [ ] Add a *news* file into the `changelog` folder, following these guidelines:
* Name it `$issue_id.$type` for example `588.bug`
* If you don't have an issue_id change it to the PR id after creating it
* Ensure type is one of `removal`, `feature`, `bugfix`, `vendor`, `doc` or `trivial`
* Make sure to use full sentences with correct case and punctuation, for example:
```
Fix issue with non-ascii contents in doctest text files.
```

View File

@@ -1,30 +1,21 @@
sudo: false sudo: false
language: python language: python
python: python:
- '2.6'
- '2.7'
- '3.4'
- '3.5' - '3.5'
install: pip install -U tox setuptools_scm - '3.6'
env: install: pip install tox-travis setuptools_scm
matrix: matrix:
- TESTENV=py26-pytest26 include:
- TESTENV=py26-pytest27 # note: please use "tox --listenvs" to populate the build matrix
- TESTENV=py26-pytest28 - python: "3.5"
- TESTENV=py26-pytest29 env: TOXENV=flakes
- TESTENV=py27-pytest26 - python: "3.5"
- TESTENV=py27-pytest27 env: TOXENV=readme
- TESTENV=py27-pytest28
- TESTENV=py27-pytest29 script: tox
- TESTENV=py34-pytest26
- TESTENV=py34-pytest27
- TESTENV=py34-pytest28
- TESTENV=py34-pytest29
- TESTENV=py35-pytest27
- TESTENV=py35-pytest28
- TESTENV=py35-pytest29
- TESTENV=py27-pytest28-pexpect
- TESTENV=py35-pytest28-pexpect
- TESTENV=flakes
- TESTENV=readme
script: tox --recreate -e $TESTENV
notifications: notifications:
irc: irc:
channels: channels:
@@ -37,6 +28,7 @@ notifications:
deploy: deploy:
provider: pypi provider: pypi
user: ronny user: ronny
skip_upload_docs: true
password: password:
secure: cxmSDho5d+PYKEM4ZCg8ms1P4lzhYkrw6fEOm2HtTcsuCyY6aZMSgImWAnEYbJHSkdzgcxlXK9UKJ9B0YenXmBCkAr7UjdnpNXNmkySr0sYzlH/sfqt/dDATCHFaRKxnkOSOVywaDYhT9n8YudbXI77pXwD12i/CeSSJDbHhsu0JYUfAcb+D6YjRYoA2SEGCnzSzg+gDDfwXZx4ZiODCGLVwieNp1klCg88YROUE1BaYYNuUOONvfXX8+TWowbCF6ChH1WL/bZ49OStEYQNuYxZQZr4yClIqu9VJbchrU8j860K9ott2kkGTgfB/dDrQB/XncBubyIX9ikzCQAmmBXWAI3eyvWLPDk2Jz7kW2l2RT7syct80tCq3JhvQ1qdwr5ap7siocTLgnBW0tF4tkHSTFN3510fkc43npnp6FThebESQpnI24vqpwJ9hI/kW5mYi014Og2E/cpCXnz2XO8iZPDbqAMQpDsqEQoyhfGNgPTGp4K30TxRtwZBI5hHhDKnnR16fXtRgt1gYPvz/peUQvvpOm4JzIzGXPzluuutpnCBy75v5+oiwT3YRrLL/Meims9FtDDXL3qQubAE/ezIOOpm0N5XXV8DxIom8EN71yq5ab1tqhM+tBX7owRjy4FR4If2Q8feBdmTuh26DIQt/y+qSG8VkB9Sw/JCjc7c= secure: cxmSDho5d+PYKEM4ZCg8ms1P4lzhYkrw6fEOm2HtTcsuCyY6aZMSgImWAnEYbJHSkdzgcxlXK9UKJ9B0YenXmBCkAr7UjdnpNXNmkySr0sYzlH/sfqt/dDATCHFaRKxnkOSOVywaDYhT9n8YudbXI77pXwD12i/CeSSJDbHhsu0JYUfAcb+D6YjRYoA2SEGCnzSzg+gDDfwXZx4ZiODCGLVwieNp1klCg88YROUE1BaYYNuUOONvfXX8+TWowbCF6ChH1WL/bZ49OStEYQNuYxZQZr4yClIqu9VJbchrU8j860K9ott2kkGTgfB/dDrQB/XncBubyIX9ikzCQAmmBXWAI3eyvWLPDk2Jz7kW2l2RT7syct80tCq3JhvQ1qdwr5ap7siocTLgnBW0tF4tkHSTFN3510fkc43npnp6FThebESQpnI24vqpwJ9hI/kW5mYi014Og2E/cpCXnz2XO8iZPDbqAMQpDsqEQoyhfGNgPTGp4K30TxRtwZBI5hHhDKnnR16fXtRgt1gYPvz/peUQvvpOm4JzIzGXPzluuutpnCBy75v5+oiwT3YRrLL/Meims9FtDDXL3qQubAE/ezIOOpm0N5XXV8DxIom8EN71yq5ab1tqhM+tBX7owRjy4FR4If2Q8feBdmTuh26DIQt/y+qSG8VkB9Sw/JCjc7c=
on: on:

View File

@@ -1,3 +1,67 @@
pytest-xdist 1.18.0 (2017-06-26)
================================
Features
--------
- Add long option `--numprocesses` as alternative for `-n`. (#168)
Bug Fixes
---------
- Fix serialization and deserialization dropping longrepr details. (#133)
pytest-xdist 1.17.1 (2017-06-10)
================================
Bug Fixes
---------
- Hot fix release reverting the change introduced by #124, unfortunately it
broke a number of test suites so we are reversing this change while we
investigate the problem. (#157)
Improved Documentation
----------------------
- Introduced ``towncrier`` for ``CHANGELOG`` management. (#154)
- Added ``HOWTORELEASE`` documentation. (#155)
..
You should *NOT* be adding new change log entries to this file, this
file is managed by towncrier. You *may* edit previous change logs to
fix problems like typo corrections or such.
To add a new change log entry, please see
https://pip.pypa.io/en/latest/development/#adding-a-news-entry
We named the news folder ``changelog``
.. towncrier release notes start
1.17.0
------
- fix #124: xdist would mark test as complete after 'call' step. As a result,
xdist could identify the wrong test as failing when test crashes at teardown.
To address this issue, xdist now marks test as complete at teardown.
1.16.0
------
- ``pytest-xdist`` now requires pytest 2.7 or later.
- Add ``worker_id`` attribute in the TestReport
- new hook: ``pytest_xdist_make_scheduler(config, log)``, can return custom tests items
distribution logic implementation. You can take a look at built-in ``LoadScheduling``
and ``EachScheduling`` implementations. Note that required scheduler class public
API may change in next ``pytest-xdist`` versions.
1.15.0 1.15.0
------ ------

45
HOWTORELEASE.rst Normal file
View File

@@ -0,0 +1,45 @@
======================
Releasing pytest-xdist
======================
This document describes the steps to make a new ``pytest-xdist`` release.
Version
-------
``master`` should always be green and a potential release candidate. ``pytest-xdist`` follows
semantic versioning, so given that the current version is ``X.Y.Z``, to find the next version number
one needs to look at the ``changelog`` folder:
- If there is any file named ``*.feature``, then we must make a new **minor** release: next
release will be ``X.Y+1.0``.
- Otherwise it is just a **bug fix** release: ``X.Y.Z+1``.
Steps
-----
To publish a new release ``X.Y.Z``, the steps are as follows:
#. Create a new branch named ``release-X.Y.Z`` from the latest ``master``.
#. Install ``pytest-xdist`` and dev requirements in a virtualenv::
$ pip install -e . -r dev-requirements.txt
#. Update ``CHANGELOG.rst`` file by running::
$ towncrier --version X.Y.Z
It might ask for confirmation to remove news fragments; answer yes.
#. Commit and push the branch for review.
#. Once PR is **green** and **approved**, create and push a tag::
$ export VERSION=X.Y.Z
$ git tag v$VERSION release-$VERSION
$ git push git@github.com:pytest-dev/pytest-xdist.git v$VERSION
That will build the package and publish it on ``PyPI`` automatically.

View File

@@ -1,7 +0,0 @@
include CHANGELOG
include LICENSE
include README.txt
include setup.py
include tox.ini
graft testing
prune .git

View File

@@ -35,6 +35,8 @@ The execution flow is:
fashion. The rest of the tests will be distributed later as **workers** fashion. The rest of the tests will be distributed later as **workers**
finish tests (see below). finish tests (see below).
1. Note that `pytest_xdist_make_scheduler` hook can be used to implement custom tests distribution logic.
1. **workers** re-implement `pytest_runtestloop`: pytest's default implementation 1. **workers** re-implement `pytest_runtestloop`: pytest's default implementation
basically loops over all collected items in the `session` object and executes basically loops over all collected items in the `session` object and executes
the `pytest_runtest_protocol` for each test item, but in xdist **workers** sit idly the `pytest_runtest_protocol` for each test item, but in xdist **workers** sit idly

View File

@@ -1,10 +1,14 @@
.. image:: http://img.shields.io/pypi/v/pytest-xdist.svg
:target: https://pypi.python.org/pypi/pytest-xdist
.. image:: https://anaconda.org/conda-forge/pytest-xdist/badges/version.svg
:target: https://anaconda.org/conda-forge/pytest-xdist
.. image:: https://travis-ci.org/pytest-dev/pytest-xdist.svg?branch=master .. image:: https://travis-ci.org/pytest-dev/pytest-xdist.svg?branch=master
:target: https://travis-ci.org/pytest-dev/pytest-xdist :target: https://travis-ci.org/pytest-dev/pytest-xdist
.. image:: http://img.shields.io/pypi/v/pytest-xdist.svg
:target: https://pypi.python.org/pypi/pytest-xdist
.. image:: https://ci.appveyor.com/api/projects/status/56eq1a1avd4sdd7e/branch/master?svg=true .. image:: https://ci.appveyor.com/api/projects/status/56eq1a1avd4sdd7e/branch/master?svg=true
:target: https://ci.appveyor.com/project/pytestbot/pytest-xdist :target: https://ci.appveyor.com/project/pytestbot/pytest-xdist
@@ -46,7 +50,7 @@ Install the plugin with::
pip install pytest-xdist pip install pytest-xdist
or use the package in develope/in-place mode with or use the package in develop/in-place mode with
a checkout of the `pytest-xdist repository`_ :: a checkout of the `pytest-xdist repository`_ ::
python setup.py develop python setup.py develop
@@ -209,6 +213,10 @@ defined:
* ``PYTEST_XDIST_WORKER_COUNT``: the total number of workers in this session, * ``PYTEST_XDIST_WORKER_COUNT``: the total number of workers in this session,
e.g., ``"4"`` when ``-n 4`` is given in the command-line. e.g., ``"4"`` when ``-n 4`` is given in the command-line.
The information about the worker_id in a test is stored in the TestReport as
well, under worker_id attribute.
*New in version 1.15.* *New in version 1.15.*
Specifying test exec environments in an ini file Specifying test exec environments in an ini file

View File

@@ -1,3 +1,16 @@
environment:
matrix:
# note: please use "tox --listenvs" to populate the build matrix
- TOXENV: "py26-pytest30"
- TOXENV: "py27-pytest30"
- TOXENV: "py34-pytest30"
- TOXENV: "py35-pytest30"
- TOXENV: "py36-pytest30"
- TOXENV: "py27-pytest30-pexpect"
- TOXENV: "py35-pytest30-pexpect"
- TOXENV: "flakes"
- TOXENV: "readme"
install: install:
- C:\Python35\python -m pip install tox setuptools_scm - C:\Python35\python -m pip install tox setuptools_scm

39
changelog/_template.rst Normal file
View File

@@ -0,0 +1,39 @@
{% for section in sections %}
{% set underline = "-" %}
{% if section %}
{{section}}
{{ underline * section|length }}{% set underline = "~" %}
{% endif %}
{% if sections[section] %}
{% for category, val in definitions.items() if category in sections[section] and category != 'trivial' %}
{{ definitions[category]['name'] }}
{{ underline * definitions[category]['name']|length }}
{% if definitions[category]['showcontent'] %}
{% for text, values in sections[section][category]|dictsort(by='value') %}
- {{ text }}{% if category != 'vendor' %} ({{ values|sort|join(', ') }}){% endif %}
{% endfor %}
{% else %}
- {{ sections[section][category]['']|sort|join(', ') }}
{% endif %}
{% if sections[section][category]|length == 0 %}
No significant changes.
{% else %}
{% endif %}
{% endfor %}
{% else %}
No significant changes.
{% endif %}
{% endfor %}

1
dev-requirements.txt Normal file
View File

@@ -0,0 +1 @@
towncrier

36
pyproject.toml Normal file
View File

@@ -0,0 +1,36 @@
[tool.towncrier]
package = "xdist"
filename = "CHANGELOG.rst"
directory = "changelog/"
title_format = "pytest-xdist {version} ({project_date})"
template = "changelog/_template.rst"
[[tool.towncrier.type]]
directory = "removal"
name = "Deprecations and Removals"
showcontent = true
[[tool.towncrier.type]]
directory = "feature"
name = "Features"
showcontent = true
[[tool.towncrier.type]]
directory = "bugfix"
name = "Bug Fixes"
showcontent = true
[[tool.towncrier.type]]
directory = "vendor"
name = "Vendored Libraries"
showcontent = true
[[tool.towncrier.type]]
directory = "doc"
name = "Improved Documentation"
showcontent = true
[[tool.towncrier.type]]
directory = "trivial"
name = "Trivial Changes"
showcontent = false

View File

@@ -1,2 +1,5 @@
[bdist_wheel] [bdist_wheel]
universal = 1 universal = 1
[metadata]
license_file = LICENSE

View File

@@ -20,7 +20,7 @@ setup(
], ],
}, },
zip_safe=False, zip_safe=False,
install_requires=['execnet>=1.1', 'pytest>=2.4.2', 'py>=1.4.22'], install_requires=['execnet>=1.1', 'pytest>=3.0.0', 'py>=1.4.22'],
setup_requires=['setuptools_scm'], setup_requires=['setuptools_scm'],
classifiers=[ classifiers=[
'Development Status :: 5 - Production/Stable', 'Development Status :: 5 - Production/Stable',

View File

@@ -10,7 +10,9 @@ class TestDistribution:
""") """)
result = testdir.runpytest(p1, "-n1") result = testdir.runpytest(p1, "-n1")
assert result.ret == 0 assert result.ret == 0
result.stdout.fnmatch_lines(["*1 passed*", ]) result.stdout.fnmatch_lines([
"*1 passed*",
])
def test_n1_fail(self, testdir): def test_n1_fail(self, testdir):
p1 = testdir.makepyfile(""" p1 = testdir.makepyfile("""
@@ -19,7 +21,9 @@ class TestDistribution:
""") """)
result = testdir.runpytest(p1, "-n1") result = testdir.runpytest(p1, "-n1")
assert result.ret == 1 assert result.ret == 1
result.stdout.fnmatch_lines(["*1 failed*", ]) result.stdout.fnmatch_lines([
"*1 failed*",
])
def test_n1_import_error(self, testdir): def test_n1_import_error(self, testdir):
p1 = testdir.makepyfile(""" p1 = testdir.makepyfile("""
@@ -30,7 +34,7 @@ class TestDistribution:
result = testdir.runpytest(p1, "-n1") result = testdir.runpytest(p1, "-n1")
assert result.ret == 1 assert result.ret == 1
result.stdout.fnmatch_lines([ result.stdout.fnmatch_lines([
"E ImportError: *__import_of_missing_module*", "E *Error: No module named *__import_of_missing_module*",
]) ])
def test_n2_import_error(self, testdir): def test_n2_import_error(self, testdir):
@@ -53,7 +57,9 @@ class TestDistribution:
""") """)
result = testdir.runpytest(p1, "-n1") result = testdir.runpytest(p1, "-n1")
assert result.ret == 0 assert result.ret == 0
result.stdout.fnmatch_lines(["*1 skipped*", ]) result.stdout.fnmatch_lines([
"*1 skipped*",
])
def test_manytests_to_one_import_error(self, testdir): def test_manytests_to_one_import_error(self, testdir):
p1 = testdir.makepyfile(""" p1 = testdir.makepyfile("""
@@ -62,13 +68,14 @@ class TestDistribution:
pass pass
""") """)
result = testdir.runpytest(p1, '--tx=popen', '--tx=popen') result = testdir.runpytest(p1, '--tx=popen', '--tx=popen')
assert result.ret == 1 assert result.ret in (1, 2)
result.stdout.fnmatch_lines([ result.stdout.fnmatch_lines([
"E ImportError: *__import_of_missing_module*", "E *Error: No module named *__import_of_missing_module*",
]) ])
def test_manytests_to_one_popen(self, testdir): def test_manytests_to_one_popen(self, testdir):
p1 = testdir.makepyfile(""" p1 = testdir.makepyfile(
"""
import py import py
def test_fail0(): def test_fail0():
assert 0 assert 0
@@ -108,10 +115,13 @@ class TestDistribution:
""" % str(testdir.tmpdir)) """ % str(testdir.tmpdir))
result = testdir.runpytest_subprocess(p1, "-n1") result = testdir.runpytest_subprocess(p1, "-n1")
assert result.ret == 0 assert result.ret == 0
result.stdout.fnmatch_lines(["*1 passed*", ]) result.stdout.fnmatch_lines([
"*1 passed*",
])
def test_dist_ini_specified(self, testdir): def test_dist_ini_specified(self, testdir):
p1 = testdir.makepyfile(""" p1 = testdir.makepyfile(
"""
import py import py
def test_fail0(): def test_fail0():
assert 0 assert 0
@@ -207,14 +217,14 @@ class TestDistribution:
""") """)
p1 = testdir.makepyfile("def test_func(): pass") p1 = testdir.makepyfile("def test_func(): pass")
result = testdir.runpytest("-v", p1, '-d', '--tx=popen') result = testdir.runpytest("-v", p1, '-d', '--tx=popen')
result.stdout.fnmatch_lines([ result.stdout.fnmatch_lines(
"*0*Python*", "*calculated result is 49*", "*1 passed*" ["*0*Python*", "*calculated result is 49*", "*1 passed*"])
])
assert result.ret == 0 assert result.ret == 0
def test_keyboardinterrupt_hooks_issue79(self, testdir): def test_keyboardinterrupt_hooks_issue79(self, testdir):
testdir.makepyfile(__init__="", testdir.makepyfile(
test_one=""" __init__="",
test_one="""
def test_hello(): def test_hello():
raise KeyboardInterrupt() raise KeyboardInterrupt()
""") """)
@@ -273,8 +283,9 @@ class TestDistEach:
py.test.skip("%s not found" % name) py.test.skip("%s not found" % name)
interpreters.append(interp) interpreters.append(interp)
testdir.makepyfile(__init__="", testdir.makepyfile(
test_one=""" __init__="",
test_one="""
import sys import sys
def test_hello(): def test_hello():
print("%s...%s" % sys.version_info[:2]) print("%s...%s" % sys.version_info[:2])
@@ -335,9 +346,8 @@ def test_teardownfails_one_function(testdir):
assert 0 assert 0
""") """)
result = testdir.runpytest(p, '-n1', '--tx=popen') result = testdir.runpytest(p, '-n1', '--tx=popen')
result.stdout.fnmatch_lines([ result.stdout.fnmatch_lines(
"*def teardown_function(function):*", "*1 passed*1 error*" ["*def teardown_function(function):*", "*1 passed*1 error*"])
])
@py.test.mark.xfail @py.test.mark.xfail
@@ -350,7 +360,9 @@ def test_terminate_on_hangingnode(testdir):
""") """)
result = testdir.runpytest(p, '--dist=each', '--tx=popen//id=my') result = testdir.runpytest(p, '--dist=each', '--tx=popen//id=my')
assert result.duration < 2.0 assert result.duration < 2.0
result.stdout.fnmatch_lines(["*killed*my*", ]) result.stdout.fnmatch_lines([
"*killed*my*",
])
@pytest.mark.xfail(reason="works if run outside test suite", run=False) @pytest.mark.xfail(reason="works if run outside test suite", run=False)
@@ -377,7 +389,10 @@ def test_session_hooks(testdir):
assert hasattr(sys, 'pytestsessionhooks') assert hasattr(sys, 'pytestsessionhooks')
""") """)
result = testdir.runpytest(p, "--dist=each", "--tx=popen") result = testdir.runpytest(p, "--dist=each", "--tx=popen")
result.stdout.fnmatch_lines(["*ValueError*", "*1 passed*", ]) result.stdout.fnmatch_lines([
"*ValueError*",
"*1 passed*",
])
assert not result.ret assert not result.ret
d = result.parseoutcomes() d = result.parseoutcomes()
assert d['passed'] == 1 assert d['passed'] == 1
@@ -422,7 +437,10 @@ def test_funcarg_teardown_failure(testdir):
pass pass
""") """)
result = testdir.runpytest_subprocess("--debug", p) # , "-n1") result = testdir.runpytest_subprocess("--debug", p) # , "-n1")
result.stdout.fnmatch_lines(["*ValueError*42*", "*1 passed*1 error*", ]) result.stdout.fnmatch_lines([
"*ValueError*42*",
"*1 passed*1 error*",
])
assert result.ret assert result.ret
@@ -453,7 +471,8 @@ def test_skipping(testdir):
def test_issue34_pluginloading_in_subprocess(testdir): def test_issue34_pluginloading_in_subprocess(testdir):
testdir.tmpdir.join("plugin123.py").write(py.code.Source(""" testdir.tmpdir.join("plugin123.py").write(
py.code.Source("""
def pytest_namespace(): def pytest_namespace():
return {'sample_variable': 'testing'} return {'sample_variable': 'testing'}
""")) """))
@@ -464,7 +483,9 @@ def test_issue34_pluginloading_in_subprocess(testdir):
""") """)
result = testdir.runpytest_subprocess("-n1", "-p", "plugin123") result = testdir.runpytest_subprocess("-n1", "-p", "plugin123")
assert result.ret == 0 assert result.ret == 0
result.stdout.fnmatch_lines(["*1 passed*", ]) result.stdout.fnmatch_lines([
"*1 passed*",
])
def test_fixture_scope_caching_issue503(testdir): def test_fixture_scope_caching_issue503(testdir):
@@ -486,7 +507,9 @@ def test_fixture_scope_caching_issue503(testdir):
""") """)
result = testdir.runpytest(p1, '-v', '-n1') result = testdir.runpytest(p1, '-v', '-n1')
assert result.ret == 0 assert result.ret == 0
result.stdout.fnmatch_lines(["*2 passed*", ]) result.stdout.fnmatch_lines([
"*2 passed*",
])
def test_issue_594_random_parametrize(testdir): def test_issue_594_random_parametrize(testdir):

View File

@@ -49,80 +49,84 @@ def dumpqueue(queue):
class TestEachScheduling: class TestEachScheduling:
def test_schedule_load_simple(self): def test_schedule_load_simple(self, testdir):
node1 = MockNode() node1 = MockNode()
node2 = MockNode() node2 = MockNode()
sched = EachScheduling(2) config = testdir.parseconfig("--tx=2*popen")
sched.addnode(node1) sched = EachScheduling(config)
sched.addnode(node2) sched.add_node(node1)
sched.add_node(node2)
collection = ["a.py::test_1", ] collection = ["a.py::test_1", ]
assert not sched.collection_is_completed assert not sched.collection_is_completed
sched.addnode_collection(node1, collection) sched.add_node_collection(node1, collection)
assert not sched.collection_is_completed assert not sched.collection_is_completed
sched.addnode_collection(node2, collection) sched.add_node_collection(node2, collection)
assert sched.collection_is_completed assert sched.collection_is_completed
assert sched.node2collection[node1] == collection assert sched.node2collection[node1] == collection
assert sched.node2collection[node2] == collection assert sched.node2collection[node2] == collection
sched.init_distribute() sched.schedule()
assert sched.tests_finished() assert sched.tests_finished
assert node1.sent == ['ALL'] assert node1.sent == ['ALL']
assert node2.sent == ['ALL'] assert node2.sent == ['ALL']
sched.remove_item(node1, 0) sched.mark_test_complete(node1, 0)
assert sched.tests_finished() assert sched.tests_finished
sched.remove_item(node2, 0) sched.mark_test_complete(node2, 0)
assert sched.tests_finished() assert sched.tests_finished
def test_schedule_remove_node(self): def test_schedule_remove_node(self, testdir):
node1 = MockNode() node1 = MockNode()
sched = EachScheduling(1) config = testdir.parseconfig("--tx=popen")
sched.addnode(node1) sched = EachScheduling(config)
sched.add_node(node1)
collection = ["a.py::test_1", ] collection = ["a.py::test_1", ]
assert not sched.collection_is_completed assert not sched.collection_is_completed
sched.addnode_collection(node1, collection) sched.add_node_collection(node1, collection)
assert sched.collection_is_completed assert sched.collection_is_completed
assert sched.node2collection[node1] == collection assert sched.node2collection[node1] == collection
sched.init_distribute() sched.schedule()
assert sched.tests_finished() assert sched.tests_finished
crashitem = sched.remove_node(node1) crashitem = sched.remove_node(node1)
assert crashitem assert crashitem
assert sched.tests_finished() assert sched.tests_finished
assert not sched.hasnodes() assert not sched.nodes
class TestLoadScheduling: class TestLoadScheduling:
def test_schedule_load_simple(self): def test_schedule_load_simple(self, testdir):
sched = LoadScheduling(2) config = testdir.parseconfig("--tx=2*popen")
sched.addnode(MockNode()) sched = LoadScheduling(config)
sched.addnode(MockNode()) sched.add_node(MockNode())
sched.add_node(MockNode())
node1, node2 = sched.nodes node1, node2 = sched.nodes
collection = ["a.py::test_1", "a.py::test_2"] collection = ["a.py::test_1", "a.py::test_2"]
assert not sched.collection_is_completed assert not sched.collection_is_completed
sched.addnode_collection(node1, collection) sched.add_node_collection(node1, collection)
assert not sched.collection_is_completed assert not sched.collection_is_completed
sched.addnode_collection(node2, collection) sched.add_node_collection(node2, collection)
assert sched.collection_is_completed assert sched.collection_is_completed
assert sched.node2collection[node1] == collection assert sched.node2collection[node1] == collection
assert sched.node2collection[node2] == collection assert sched.node2collection[node2] == collection
sched.init_distribute() sched.schedule()
assert not sched.pending assert not sched.pending
assert sched.tests_finished() assert sched.tests_finished
assert len(node1.sent) == 1 assert len(node1.sent) == 1
assert len(node2.sent) == 1 assert len(node2.sent) == 1
assert node1.sent == [0] assert node1.sent == [0]
assert node2.sent == [1] assert node2.sent == [1]
sched.remove_item(node1, node1.sent[0]) sched.mark_test_complete(node1, node1.sent[0])
assert sched.tests_finished() assert sched.tests_finished
def test_init_distribute_batch_size(self): def test_schedule_batch_size(self, testdir):
sched = LoadScheduling(2) config = testdir.parseconfig("--tx=2*popen")
sched.addnode(MockNode()) sched = LoadScheduling(config)
sched.addnode(MockNode()) sched.add_node(MockNode())
sched.add_node(MockNode())
node1, node2 = sched.nodes node1, node2 = sched.nodes
col = ["xyz"] * (6) col = ["xyz"] * (6)
sched.addnode_collection(node1, col) sched.add_node_collection(node1, col)
sched.addnode_collection(node2, col) sched.add_node_collection(node2, col)
sched.init_distribute() sched.schedule()
# assert not sched.tests_finished() # assert not sched.tests_finished
sent1 = node1.sent sent1 = node1.sent
sent2 = node2.sent sent2 = node2.sent
assert sent1 == [0, 2] assert sent1 == [0, 2]
@@ -131,25 +135,26 @@ class TestLoadScheduling:
assert sched.node2pending[node1] == sent1 assert sched.node2pending[node1] == sent1
assert sched.node2pending[node2] == sent2 assert sched.node2pending[node2] == sent2
assert len(sched.pending) == 2 assert len(sched.pending) == 2
sched.remove_item(node1, 0) sched.mark_test_complete(node1, 0)
assert node1.sent == [0, 2, 4] assert node1.sent == [0, 2, 4]
assert sched.pending == [5] assert sched.pending == [5]
assert node2.sent == [1, 3] assert node2.sent == [1, 3]
sched.remove_item(node1, 2) sched.mark_test_complete(node1, 2)
assert node1.sent == [0, 2, 4, 5] assert node1.sent == [0, 2, 4, 5]
assert not sched.pending assert not sched.pending
def test_init_distribute_fewer_tests_than_nodes(self): def test_schedule_fewer_tests_than_nodes(self, testdir):
sched = LoadScheduling(2) config = testdir.parseconfig("--tx=2*popen")
sched.addnode(MockNode()) sched = LoadScheduling(config)
sched.addnode(MockNode()) sched.add_node(MockNode())
sched.addnode(MockNode()) sched.add_node(MockNode())
sched.add_node(MockNode())
node1, node2, node3 = sched.nodes node1, node2, node3 = sched.nodes
col = ["xyz"] * 2 col = ["xyz"] * 2
sched.addnode_collection(node1, col) sched.add_node_collection(node1, col)
sched.addnode_collection(node2, col) sched.add_node_collection(node2, col)
sched.init_distribute() sched.schedule()
# assert not sched.tests_finished() # assert not sched.tests_finished
sent1 = node1.sent sent1 = node1.sent
sent2 = node2.sent sent2 = node2.sent
sent3 = node3.sent sent3 = node3.sent
@@ -158,17 +163,18 @@ class TestLoadScheduling:
assert sent3 == [] assert sent3 == []
assert not sched.pending assert not sched.pending
def test_init_distribute_fewer_than_two_tests_per_node(self): def test_schedule_fewer_than_two_tests_per_node(self, testdir):
sched = LoadScheduling(2) config = testdir.parseconfig("--tx=2*popen")
sched.addnode(MockNode()) sched = LoadScheduling(config)
sched.addnode(MockNode()) sched.add_node(MockNode())
sched.addnode(MockNode()) sched.add_node(MockNode())
sched.add_node(MockNode())
node1, node2, node3 = sched.nodes node1, node2, node3 = sched.nodes
col = ["xyz"] * 5 col = ["xyz"] * 5
sched.addnode_collection(node1, col) sched.add_node_collection(node1, col)
sched.addnode_collection(node2, col) sched.add_node_collection(node2, col)
sched.init_distribute() sched.schedule()
# assert not sched.tests_finished() # assert not sched.tests_finished
sent1 = node1.sent sent1 = node1.sent
sent2 = node2.sent sent2 = node2.sent
sent3 = node3.sent sent3 = node3.sent
@@ -177,14 +183,15 @@ class TestLoadScheduling:
assert sent3 == [2] assert sent3 == [2]
assert not sched.pending assert not sched.pending
def test_add_remove_node(self): def test_add_remove_node(self, testdir):
node = MockNode() node = MockNode()
sched = LoadScheduling(1) config = testdir.parseconfig("--tx=popen")
sched.addnode(node) sched = LoadScheduling(config)
sched.add_node(node)
collection = ["test_file.py::test_func"] collection = ["test_file.py::test_func"]
sched.addnode_collection(node, collection) sched.add_node_collection(node, collection)
assert sched.collection_is_completed assert sched.collection_is_completed
sched.init_distribute() sched.schedule()
assert not sched.pending assert not sched.pending
crashitem = sched.remove_node(node) crashitem = sched.remove_node(node)
assert crashitem == collection[0] assert crashitem == collection[0]
@@ -207,16 +214,16 @@ class TestLoadScheduling:
self.reports.append(report) self.reports.append(report)
collect_hook = CollectHook() collect_hook = CollectHook()
config = testdir.parseconfig() config = testdir.parseconfig("--tx=2*popen")
config.pluginmanager.register(collect_hook, "collect_hook") config.pluginmanager.register(collect_hook, "collect_hook")
node1 = MockNode() node1 = MockNode()
node2 = MockNode() node2 = MockNode()
sched = LoadScheduling(2, config=config) sched = LoadScheduling(config)
sched.addnode(node1) sched.add_node(node1)
sched.addnode(node2) sched.add_node(node2)
sched.addnode_collection(node1, ["a.py::test_1"]) sched.add_node_collection(node1, ["a.py::test_1"])
sched.addnode_collection(node2, ["a.py::test_2"]) sched.add_node_collection(node2, ["a.py::test_2"])
sched.init_distribute() sched.schedule()
assert len(collect_hook.reports) == 1 assert len(collect_hook.reports) == 1
rep = collect_hook.reports[0] rep = collect_hook.reports[0]
assert 'Different tests were collected between' in rep.longrepr assert 'Different tests were collected between' in rep.longrepr

View File

@@ -268,7 +268,7 @@ class TestFunctional:
""") """)
child = testdir.spawn_pytest("-f %s" % p) child = testdir.spawn_pytest("-f %s" % p)
child.expect("1 xpass") child.expect("1 xpass")
child.expect("### LOOPONFAILING ####") # child.expect("### LOOPONFAILING ####")
child.expect("waiting for changes") child.expect("waiting for changes")
child.kill(15) child.kill(15)

View File

@@ -14,14 +14,19 @@ class TestHooks:
def test_runtest_logreport(self, testdir): def test_runtest_logreport(self, testdir):
"""Test that log reports from pytest_runtest_logreport when running """Test that log reports from pytest_runtest_logreport when running
with xdist contain a "node" attribute. (#8) with xdist contain "node", "nodeid" and "worker_id" attributes. (#8)
""" """
testdir.makeconftest(""" testdir.makeconftest("""
def pytest_runtest_logreport(report): def pytest_runtest_logreport(report):
if hasattr(report, 'node'): if hasattr(report, 'node'):
slaveid = report.node.slaveinput['slaveid']
if report.when == "call": if report.when == "call":
print("HOOK: %s %s" % (report.nodeid, slaveid)) slaveid = report.node.slaveinput['slaveid']
if slaveid != report.worker_id:
print("HOOK: Worker id mismatch: %s %s"
% (slaveid, report.worker_id))
else:
print("HOOK: %s %s"
% (report.nodeid, report.worker_id))
""") """)
res = testdir.runpytest('-n1', '-s') res = testdir.runpytest('-n1', '-s')
res.stdout.fnmatch_lines([ res.stdout.fnmatch_lines([

View File

@@ -20,6 +20,10 @@ def test_dist_options(testdir):
check_options(config) check_options(config)
assert config.option.dist == "load" assert config.option.dist == "load"
assert config.option.tx == ['popen'] * 2 assert config.option.tx == ['popen'] * 2
config = testdir.parseconfigure("--numprocesses", "2")
check_options(config)
assert config.option.dist == "load"
assert config.option.tx == ['popen'] * 2
config = testdir.parseconfigure("-d") config = testdir.parseconfigure("-d")
check_options(config) check_options(config)
assert config.option.dist == "load" assert config.option.dist == "load"

View File

@@ -79,6 +79,20 @@ def test_remoteinitconfig(testdir):
class TestReportSerialization: class TestReportSerialization:
def test_xdist_report_longrepr_reprcrash_130(self, testdir):
reprec = testdir.inline_runsource("""
import py
def test_fail(): assert False, 'Expected Message'
""")
reports = reprec.getreports("pytest_runtest_logreport")
assert len(reports) == 3
initial_failure_report = reports[1]
d = serialize_report(initial_failure_report)
check_marshallable(d)
processed_report = unserialize_report("testreport", d)
assert 'Expected Message' \
in processed_report.longrepr.reprcrash.message
def test_itemreport_outcomes(self, testdir): def test_itemreport_outcomes(self, testdir):
reprec = testdir.inline_runsource(""" reprec = testdir.inline_runsource("""
import py import py
@@ -108,7 +122,7 @@ class TestReportSerialization:
assert newrep.when == rep.when assert newrep.when == rep.when
assert newrep.keywords == rep.keywords assert newrep.keywords == rep.keywords
if rep.failed: if rep.failed:
assert newrep.longrepr == str(rep.longrepr) assert newrep.longreprtext == rep.longreprtext
def test_collectreport_passed(self, testdir): def test_collectreport_passed(self, testdir):
reprec = testdir.inline_runsource("def test_func(): pass") reprec = testdir.inline_runsource("def test_func(): pass")

10
tox.ini
View File

@@ -1,9 +1,8 @@
[tox] [tox]
# if you change the envlist, please update .travis.yml file as well # if you change the envlist, please update .travis.yml file as well
envlist= envlist=
py{26,27,34}-pytest2{6,7,8,9} py{26,27,34,35,36}-pytest{30}
py35-pytest2{7,8,9} py{27,35}-pytest{30}-pexpect
py{27,35}-pytest28-pexpect
flakes flakes
readme readme
@@ -15,10 +14,7 @@ deps =
pycmd pycmd
# to avoid .eggs # to avoid .eggs
setuptools_scm setuptools_scm
pytest26: pytest~=2.6.1 pytest30: pytest~=3.0.5
pytest27: pytest~=2.7.2
pytest28: pytest~=2.8.7
pytest29: pytest~=2.9.1
pexpect: pexpect pexpect: pexpect
platform= platform=
pexpect: linux|darwin pexpect: linux|darwin

View File

@@ -4,7 +4,7 @@ from _pytest.runner import CollectReport
import pytest import pytest
import py import py
from xdist.slavemanage import NodeManager from xdist.slavemanage import NodeManager, parse_spec_config
queue = py.builtin._tryimport('queue', 'Queue') queue = py.builtin._tryimport('queue', 'Queue')
@@ -24,8 +24,9 @@ class EachScheduling:
assigned the remaining items from the removed node. assigned the remaining items from the removed node.
""" """
def __init__(self, numnodes, log=None): def __init__(self, config, log=None):
self.numnodes = numnodes self.config = config
self.numnodes = len(parse_spec_config(config))
self.node2collection = {} self.node2collection = {}
self.node2pending = {} self.node2pending = {}
self._started = [] self._started = []
@@ -41,10 +42,19 @@ class EachScheduling:
"""A list of all nodes in the scheduler.""" """A list of all nodes in the scheduler."""
return list(self.node2pending.keys()) return list(self.node2pending.keys())
def hasnodes(self): @property
return bool(self.node2pending) def tests_finished(self):
if not self.collection_is_completed:
return False
if self._removed2pending:
return False
for pending in self.node2pending.values():
if len(pending) >= 2:
return False
return True
def haspending(self): @property
def has_pending(self):
"""Return True if there are pending test items """Return True if there are pending test items
This indicates that collection has finished and nodes are This indicates that collection has finished and nodes are
@@ -56,21 +66,11 @@ class EachScheduling:
return True return True
return False return False
def addnode(self, node): def add_node(self, node):
assert node not in self.node2pending assert node not in self.node2pending
self.node2pending[node] = [] self.node2pending[node] = []
def tests_finished(self): def add_node_collection(self, node, collection):
if not self.collection_is_completed:
return False
if self._removed2pending:
return False
for pending in self.node2pending.values():
if len(pending) >= 2:
return False
return True
def addnode_collection(self, node, collection):
"""Add the collected test items from a node """Add the collected test items from a node
Collection is complete once all nodes have submitted their Collection is complete once all nodes have submitted their
@@ -78,7 +78,7 @@ class EachScheduling:
list. When the collection is already completed this list. When the collection is already completed this
submission is from a node which was restarted to replace a submission is from a node which was restarted to replace a
dead node. In this case we already assign the pending items dead node. In this case we already assign the pending items
here. In either case ``.init_distribute()`` will instruct the here. In either case ``.schedule()`` will instruct the
node to start running the required tests. node to start running the required tests.
""" """
assert node in self.node2pending assert node in self.node2pending
@@ -102,11 +102,11 @@ class EachScheduling:
self.node2pending[node] = pending self.node2pending[node] = pending
break break
def remove_item(self, node, item_index, duration=0): def mark_test_complete(self, node, item_index, duration=0):
self.node2pending[node].remove(item_index) self.node2pending[node].remove(item_index)
def remove_node(self, node): def remove_node(self, node):
# KeyError if we didn't get an addnode() yet # KeyError if we didn't get an add_node() yet
pending = self.node2pending.pop(node) pending = self.node2pending.pop(node)
if not pending: if not pending:
return return
@@ -115,12 +115,12 @@ class EachScheduling:
self._removed2pending[node] = pending self._removed2pending[node] = pending
return crashitem return crashitem
def init_distribute(self): def schedule(self):
"""Schedule the test items on the nodes """Schedule the test items on the nodes
If the node's pending list is empty it is a new node which If the node's pending list is empty it is a new node which
needs to run all the tests. If the pending list is already needs to run all the tests. If the pending list is already
populated (by ``.addnode_collection()``) then it replaces a populated (by ``.add_node_collection()``) then it replaces a
dead node and we only need to run those tests. dead node and we only need to run those tests.
""" """
assert self.collection_is_completed assert self.collection_is_completed
@@ -143,7 +143,7 @@ class LoadScheduling:
when all collections are received it is verified they are when all collections are received it is verified they are
identical collections. Then the collection gets divided up in identical collections. Then the collection gets divided up in
chunks and chunks get submitted to nodes. Whenever a node finishes chunks and chunks get submitted to nodes. Whenever a node finishes
an item, it calls ``.remove_item()`` which will trigger the an item, it calls ``.mark_test_complete()`` which will trigger the
scheduler to assign more tests if the number of pending tests for scheduler to assign more tests if the number of pending tests for
the node falls below a low-watermark. the node falls below a low-watermark.
@@ -170,7 +170,7 @@ class LoadScheduling:
:collection: The one collection once it is validated to be :collection: The one collection once it is validated to be
identical between all the nodes. It is initialised to None identical between all the nodes. It is initialised to None
until ``.init_distribute()`` is called. until ``.schedule()`` is called.
:pending: List of indices of globally pending tests. These are :pending: List of indices of globally pending tests. These are
tests which have not yet been allocated to a chunk for a node tests which have not yet been allocated to a chunk for a node
@@ -181,8 +181,8 @@ class LoadScheduling:
:config: Config object, used for handling hooks. :config: Config object, used for handling hooks.
""" """
def __init__(self, numnodes, log=None, config=None): def __init__(self, config, log=None):
self.numnodes = numnodes self.numnodes = len(parse_spec_config(config))
self.node2collection = {} self.node2collection = {}
self.node2pending = {} self.node2pending = {}
self.pending = [] self.pending = []
@@ -208,7 +208,20 @@ class LoadScheduling:
""" """
return len(self.node2collection) >= self.numnodes return len(self.node2collection) >= self.numnodes
def haspending(self): @property
def tests_finished(self):
"""Return True if all tests have been executed by the nodes."""
if not self.collection_is_completed:
return False
if self.pending:
return False
for pending in self.node2pending.values():
if len(pending) >= 2:
return False
return True
@property
def has_pending(self):
"""Return True if there are pending test items """Return True if there are pending test items
This indicates that collection has finished and nodes are This indicates that collection has finished and nodes are
@@ -222,11 +235,7 @@ class LoadScheduling:
return True return True
return False return False
def hasnodes(self): def add_node(self, node):
"""Return True if nodes exist in the scheduler."""
return bool(self.node2pending)
def addnode(self, node):
"""Add a new node to the scheduler. """Add a new node to the scheduler.
From now on the node will be allocated chunks of tests to From now on the node will be allocated chunks of tests to
@@ -238,18 +247,7 @@ class LoadScheduling:
assert node not in self.node2pending assert node not in self.node2pending
self.node2pending[node] = [] self.node2pending[node] = []
def tests_finished(self): def add_node_collection(self, node, collection):
"""Return True if all tests have been executed by the nodes."""
if not self.collection_is_completed:
return False
if self.pending:
return False
for pending in self.node2pending.values():
if len(pending) >= 2:
return False
return True
def addnode_collection(self, node, collection):
"""Add the collected test items from a node """Add the collected test items from a node
The collection is stored in the ``.node2collection`` map. The collection is stored in the ``.node2collection`` map.
@@ -258,7 +256,7 @@ class LoadScheduling:
assert node in self.node2pending assert node in self.node2pending
if self.collection_is_completed: if self.collection_is_completed:
# A new node has been added later, perhaps an original one died. # A new node has been added later, perhaps an original one died.
# .init_distribute() should have # .schedule() should have
# been called by now # been called by now
assert self.collection assert self.collection
if collection != self.collection: if collection != self.collection:
@@ -271,7 +269,7 @@ class LoadScheduling:
return return
self.node2collection[node] = list(collection) self.node2collection[node] = list(collection)
def remove_item(self, node, item_index, duration=0): def mark_test_complete(self, node, item_index, duration=0):
"""Mark test item as completed by node """Mark test item as completed by node
The duration it took to execute the item is used as a hint to The duration it took to execute the item is used as a hint to
@@ -335,7 +333,7 @@ class LoadScheduling:
self.check_schedule(node) self.check_schedule(node)
return crashitem return crashitem
def init_distribute(self): def schedule(self):
"""Initiate distribution of the test collection """Initiate distribution of the test collection
Initiate scheduling of the items across the nodes. If this Initiate scheduling of the items across the nodes. If this
@@ -345,8 +343,6 @@ class LoadScheduling:
This is called by the ``DSession.slave_collectionfinish`` hook This is called by the ``DSession.slave_collectionfinish`` hook
if ``.collection_is_completed`` is True. if ``.collection_is_completed`` is True.
XXX Perhaps this method should have been called ".schedule()".
""" """
assert self.collection_is_completed assert self.collection_is_completed
@@ -466,6 +462,8 @@ class DSession:
self.log = py.log.Producer("dsession") self.log = py.log.Producer("dsession")
if not config.option.debug: if not config.option.debug:
py.log.setconsumer(self.log._keywords, None) py.log.setconsumer(self.log._keywords, None)
self.nodemanager = None
self.sched = None
self.shuttingdown = False self.shuttingdown = False
self.countfailures = 0 self.countfailures = 0
self.maxfail = config.getvalue("maxfail") self.maxfail = config.getvalue("maxfail")
@@ -521,16 +519,21 @@ class DSession:
# prohibit collection of test items in master process # prohibit collection of test items in master process
return True return True
def pytest_runtestloop(self): @pytest.mark.trylast
numnodes = len(self.nodemanager.specs) def pytest_xdist_make_scheduler(self, config, log):
dist = self.config.getvalue("dist") dist = config.getvalue("dist")
if dist == "load": if dist == "load":
self.sched = LoadScheduling(numnodes, log=self.log, return LoadScheduling(config, log)
config=self.config)
elif dist == "each": elif dist == "each":
self.sched = EachScheduling(numnodes, log=self.log) return EachScheduling(config, log)
else:
assert 0, dist def pytest_runtestloop(self):
self.sched = self.config.hook.pytest_xdist_make_scheduler(
config=self.config,
log=self.log
)
assert self.sched is not None
self.shouldstop = False self.shouldstop = False
while not self.session_finished: while not self.session_finished:
self.loop_once() self.loop_once()
@@ -553,7 +556,7 @@ class DSession:
call = getattr(self, method) call = getattr(self, method)
self.log("calling method", method, kwargs) self.log("calling method", method, kwargs)
call(**kwargs) call(**kwargs)
if self.sched.tests_finished(): if self.sched.tests_finished:
self.triggershutdown() self.triggershutdown()
# #
@@ -573,7 +576,7 @@ class DSession:
if self.shuttingdown: if self.shuttingdown:
node.shutdown() node.shutdown()
else: else:
self.sched.addnode(node) self.sched.add_node(node)
def slave_slavefinished(self, node): def slave_slavefinished(self, node):
"""Emitted when node executes its pytest_sessionfinish hook. """Emitted when node executes its pytest_sessionfinish hook.
@@ -635,16 +638,16 @@ class DSession:
# tell session which items were effectively collected otherwise # tell session which items were effectively collected otherwise
# the master node will finish the session with EXIT_NOTESTSCOLLECTED # the master node will finish the session with EXIT_NOTESTSCOLLECTED
self._session.testscollected = len(ids) self._session.testscollected = len(ids)
self.sched.addnode_collection(node, ids) self.sched.add_node_collection(node, ids)
if self.terminal: if self.terminal:
self.trdist.setstatus(node.gateway.spec, "[%d]" % (len(ids))) self.trdist.setstatus(node.gateway.spec, "[%d]" % (len(ids)))
if self.sched.collection_is_completed: if self.sched.collection_is_completed:
if self.terminal and not self.sched.haspending(): if self.terminal and not self.sched.has_pending:
self.trdist.ensure_show_status() self.trdist.ensure_show_status()
self.terminal.write_line("") self.terminal.write_line("")
self.terminal.write_line("scheduling tests via %s" % ( self.terminal.write_line("scheduling tests via %s" % (
self.sched.__class__.__name__)) self.sched.__class__.__name__))
self.sched.init_distribute() self.sched.schedule()
def slave_logstart(self, node, nodeid, location): def slave_logstart(self, node, nodeid, location):
"""Emitted when a node calls the pytest_runtest_logstart hook.""" """Emitted when a node calls the pytest_runtest_logstart hook."""
@@ -658,7 +661,7 @@ class DSession:
the item from the pending list in the scheduler. the item from the pending list in the scheduler.
""" """
if rep.when == "call" or (rep.when == "setup" and not rep.passed): if rep.when == "call" or (rep.when == "setup" and not rep.passed):
self.sched.remove_item(node, rep.item_index, rep.duration) self.sched.mark_test_complete(node, rep.item_index, rep.duration)
# self.report_line("testreport %s: %s" %(rep.id, rep.status)) # self.report_line("testreport %s: %s" %(rep.id, rep.status))
rep.node = node rep.node = node
self.config.hook.pytest_runtest_logreport(report=rep) self.config.hook.pytest_runtest_logreport(report=rep)

View File

@@ -9,8 +9,9 @@ with the worker instance that executed the hook originally:
You can use this hooks just as you would use normal pytest hooks, but some care You can use this hooks just as you would use normal pytest hooks, but some care
must be taken in plugins in case ``xdist`` is not installed. Please see: must be taken in plugins in case ``xdist`` is not installed. Please see:
http://pytest.org/latest/writing_plugins.html#optionally-using-hooks-from-3rd-party-plugins http://pytest.org/en/latest/writing_plugins.html#optionally-using-hooks-from-3rd-party-plugins
""" """
import pytest
def pytest_xdist_setupnodes(config, specs): def pytest_xdist_setupnodes(config, specs):
@@ -44,3 +45,8 @@ def pytest_testnodedown(node, error):
def pytest_xdist_node_collection_finished(node, ids): def pytest_xdist_node_collection_finished(node, ids):
"""called by the master node when a node finishes collecting. """called by the master node when a node finishes collecting.
""" """
@pytest.mark.firstresult
def pytest_xdist_make_scheduler(config, log):
""" return a node scheduler implementation """

View File

@@ -20,7 +20,7 @@ def parse_numprocesses(s):
def pytest_addoption(parser): def pytest_addoption(parser):
group = parser.getgroup("xdist", "distributed and subprocess testing") group = parser.getgroup("xdist", "distributed and subprocess testing")
group._addoption( group._addoption(
'-n', dest="numprocesses", metavar="numprocesses", '-n', '--numprocesses', dest="numprocesses", metavar="numprocesses",
action="store", action="store",
type=parse_numprocesses, type=parse_numprocesses,
help="shortcut for '--dist=load --tx=NUM*popen', " help="shortcut for '--dist=load --tx=NUM*popen', "
@@ -32,7 +32,7 @@ def pytest_addoption(parser):
group._addoption( group._addoption(
'--dist', metavar="distmode", '--dist', metavar="distmode",
action="store", choices=['load', 'each', 'no'], action="store", choices=['load', 'each', 'no'],
type="choice", dest="dist", default="no", dest="dist", default="no",
help=("set mode for distributing tests to exec environments.\n\n" help=("set mode for distributing tests to exec environments.\n\n"
"each: send each test to each available environment.\n\n" "each: send each test to each available environment.\n\n"
"load: send each test to available environment.\n\n" "load: send each test to available environment.\n\n"

View File

@@ -88,6 +88,7 @@ class SlaveInteractor:
def pytest_runtest_logreport(self, report): def pytest_runtest_logreport(self, report):
data = serialize_report(report) data = serialize_report(report)
data["item_index"] = self.item_index data["item_index"] = self.item_index
data["worker_id"] = self.slaveid
assert self.session.items[self.item_index].nodeid == report.nodeid assert self.session.items[self.item_index].nodeid == report.nodeid
self.sendevent("testreport", data=data) self.sendevent("testreport", data=data)
@@ -97,10 +98,33 @@ class SlaveInteractor:
def serialize_report(rep): def serialize_report(rep):
def disassembled_report(rep):
reprtraceback = rep.longrepr.reprtraceback.__dict__
reprcrash = rep.longrepr.reprcrash.__dict__
new_entries = []
for entry in reprtraceback['reprentries']:
new_entry = entry.__dict__
for key, value in new_entry.items():
if hasattr(value, '__dict__'):
new_entry[key] = value.__dict__
new_entries.append(new_entry)
reprtraceback['reprentries'] = new_entries
return {
'reprcrash': reprcrash,
'reprtraceback': reprtraceback
}
import py import py
d = rep.__dict__.copy() d = rep.__dict__.copy()
if hasattr(rep.longrepr, 'toterminal'): if hasattr(rep.longrepr, 'toterminal'):
d['longrepr'] = str(rep.longrepr) if hasattr(rep.longrepr, 'reprtraceback') \
and hasattr(rep.longrepr, 'reprcrash'):
d['longrepr'] = disassembled_report(rep)
else:
d['longrepr'] = str(rep.longrepr)
else: else:
d['longrepr'] = rep.longrepr d['longrepr'] = rep.longrepr
for name in d: for name in d:

View File

@@ -10,6 +10,22 @@ import xdist.remote
from _pytest import runner # XXX load dynamically from _pytest import runner # XXX load dynamically
def parse_spec_config(config):
xspeclist = []
for xspec in config.getvalue("tx"):
i = xspec.find("*")
try:
num = int(xspec[:i])
except ValueError:
xspeclist.append(xspec)
else:
xspeclist.extend([xspec[i + 1:]] * num)
if not xspeclist:
raise pytest.UsageError(
"MISSING test execution (tx) nodes: please specify --tx")
return xspeclist
class NodeManager(object): class NodeManager(object):
EXIT_TIMEOUT = 10 EXIT_TIMEOUT = 10
DEFAULT_IGNORES = ['.*', '*.pyc', '*.pyo', '*~'] DEFAULT_IGNORES = ['.*', '*.pyc', '*.pyo', '*~']
@@ -62,19 +78,7 @@ class NodeManager(object):
self.group.terminate(self.EXIT_TIMEOUT) self.group.terminate(self.EXIT_TIMEOUT)
def _getxspecs(self): def _getxspecs(self):
xspeclist = [] return [execnet.XSpec(x) for x in parse_spec_config(self.config)]
for xspec in self.config.getvalue("tx"):
i = xspec.find("*")
try:
num = int(xspec[:i])
except ValueError:
xspeclist.append(xspec)
else:
xspeclist.extend([xspec[i+1:]] * num)
if not xspeclist:
raise pytest.UsageError(
"MISSING test execution (tx) nodes: please specify --tx")
return [execnet.XSpec(x) for x in xspeclist]
def _getrsyncdirs(self): def _getrsyncdirs(self):
for spec in self.specs: for spec in self.specs:
@@ -292,7 +296,7 @@ class SlaveController(object):
self._down = True self._down = True
return return
eventname, kwargs = eventcall eventname, kwargs = eventcall
if eventname in ("collectionstart"): if eventname in ("collectionstart",):
self.log("ignoring %s(%s)" % (eventname, kwargs)) self.log("ignoring %s(%s)" % (eventname, kwargs))
elif eventname == "slaveready": elif eventname == "slaveready":
self.notify_inproc(eventname, node=self, **kwargs) self.notify_inproc(eventname, node=self, **kwargs)
@@ -323,7 +327,45 @@ class SlaveController(object):
def unserialize_report(name, reportdict): def unserialize_report(name, reportdict):
def assembled_report(reportdict):
from _pytest._code.code import (
ReprExceptionInfo,
ReprFileLocation,
ReprEntry,
ReprFuncArgs,
ReprTraceback
)
if reportdict['longrepr']:
if 'reprcrash' and 'reprtraceback' in reportdict['longrepr']:
reprtraceback = reportdict['longrepr']['reprtraceback']
reprcrash = reportdict['longrepr']['reprcrash']
unserialized_entries = []
for entry in reprtraceback['reprentries']:
reprfuncargs, reprfileloc = None, None
if entry['reprfuncargs']:
reprfuncargs = ReprFuncArgs(**entry['reprfuncargs'])
if entry['reprfileloc']:
reprfileloc = ReprFileLocation(**entry['reprfileloc'])
reprentry = ReprEntry(
lines=entry['lines'],
reprfuncargs=reprfuncargs,
reprlocals=entry['reprlocals'],
filelocrepr=reprfileloc,
style=entry['style']
)
unserialized_entries.append(reprentry)
reprtraceback['reprentries'] = unserialized_entries
reportdict['longrepr'] = ReprExceptionInfo(
reprtraceback=ReprTraceback(**reprtraceback),
reprcrash=ReprFileLocation(**reprcrash),
)
return reportdict
if name == "testreport": if name == "testreport":
return runner.TestReport(**reportdict) return runner.TestReport(**assembled_report(reportdict))
elif name == "collectreport": elif name == "collectreport":
return runner.CollectReport(**reportdict) return runner.CollectReport(**assembled_report(reportdict))