Commit Graph

311 Commits

Author SHA1 Message Date
Ronny Pfannschmidt
17da7cbe59 correct the type compare the updated flake8 found 2023-08-01 09:12:00 +00:00
Bruno Oliveira
be1d5c401f Make collection progress output less verbose (#901)
Fix #555
2023-05-12 10:36:32 -03:00
Aleksandr Mezin
6abcdfc22e Fix hang caused by steal command with empty test queue
Fixes #884
2023-03-09 16:44:34 +02:00
Miro Hrončok
8fd1bfd182 Tests: Unset PYTEST_XDIST_AUTO_NUM_WORKERS when the behavior without the envvar is asserted 2023-01-17 20:10:12 +01:00
Aleksandr Mezin
e9860923bf Fix some LoadScheduling tests
The expected number of nodes didn't match throughout the test code.
2023-01-11 22:34:56 +02:00
Aleksandr Mezin
d1dfad3e92 Implement work-stealing scheduler (#862)
Closes #858
2023-01-11 08:38:57 -03:00
Aleksandr Mezin
9b0b5b1495 Add --maxschedchunk CLI option (#857)
Maximum number of tests scheduled in one step.

Setting it to 1 will force pytest to send tests to workers one by one -
might be useful for a small number of slow tests.

Larger numbers will allow the scheduler to submit consecutive chunks of tests
to workers - allows reusing fixtures.

Unlimited if not set.

Fixes #855
Fixes #255
2022-12-23 08:21:20 -03:00
Bruno Oliveira
7a0bc4c1c9 Let users configure dist mode in the configuration file
Fix #789
2022-11-30 13:57:48 -03:00
Bruno Oliveira
4a33933e30 Use ternary operator to remove mypy error
Otherwise we would get this error:

```
error: Function "Callable[[Any, bool, Optional[float]], None]" could always be true in boolean context  [truthy-function]
```
2022-11-14 23:13:38 -03:00
Bruno Oliveira
d7d396a7f9 Run pyupgrade --py37-plus 2022-11-05 12:01:07 -03:00
Ronny Pfannschmidt
0c981d3547 fix and update packaging
* drop empty setup.py which "breaks" on incomplete build envs
* use the build package to build the dist
* update the gh workflow to use it

fixup noise from pre-commit
2022-10-25 17:07:47 +02:00
Petr Viktorin
c168706fbe Get -nauto default from PYTEST_XDIST_AUTO_NUM_WORKERS (#829)
* Get `-nauto` default from `PYTEST_XDIST_AUTO_NUM_WORKERS`

A few additional tests for existing functionality are added too.
And the ``-n logical`` option is documented.

Fixes: https://github.com/pytest-dev/pytest-xdist/issues/792
2022-10-24 09:33:19 -03:00
Bruno Oliveira
8b0856a986 Deprecate looponfail plugin
Fix #826
2022-10-22 10:43:02 -03:00
Bruno Oliveira
1fbf49909f Remove --boxed and pytest-forked dependency
Fixes #468
Fixes #543
2022-10-22 07:31:34 -03:00
Aleksandr Mezin
80130d59ea Restore old initial batch distribution logic in LoadScheduling (#812)
* Restore old initial batch distribution logic in LoadScheduling

pytest orders tests for optimal sequential execution - i. e. avoiding
unnecessary setup and teardown of fixtures. So executing tests in consecutive
chunks is important for optimal performance.

Commit 09d79ace35 optimized test distribution for
the corner case, when the number of tests is less than 2 * number of nodes.
At the same time, it made initial test distribution worse for all other cases.
If some tests use some fixture, and these tests fit into the initial batch,
the fixture will be created min(n_tests, n_workers) times, no matter how many
other tests there are. With the old algorithm (before
09d79ace35), if there are enough tests not using
the fixture, the fixture was created only once.

So restore the old behavior for typical cases where the number of tests is
much greater than the number of workers (or, strictly speaking, when there
are at least 2 tests for every node).

In my test suite, where fixtures create Docker containers, this change reduces
total run time by 10-15%.

This is a partial revert of commit 09d79ace35

Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2022-10-22 07:12:16 -03:00
Bruno Oliveira
25cc1a4119 Replace py.path.local usages by pathlib.Path 2022-10-21 22:54:17 -03:00
Aleksandr Mezin
e407a3377e Cancel shutdown when crashed worker is restarted (#813)
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2022-09-06 17:37:38 -03:00
Daniel Hahler
d137384315 Add test_collection_crash
Reproduces:

    INTERNALERROR> TypeError: unhashable type: 'ExceptionChainRepr'

Ref: https://github.com/pytest-dev/pytest-xdist/issues/515
Ref: https://github.com/pytest-dev/pytest/pull/6988
2022-09-06 17:20:44 -03:00
Sylvain Marié
9236f11757 Added a couple tests for #404 (#766)
* Added a couple tests for #404
Co-authored-by: Sylvain MARIE <sylvain.marie@se.com>
2022-08-23 21:41:23 +01:00
Pepa
61132777f8 Added known limitation section to the docs (#796)
Co-authored-by: Josef <sorry.i-keep@it.private>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2022-07-22 10:44:17 -03:00
Bruno Oliveira
e3c1a3de67 Remove redundant type casts 2022-03-10 09:11:00 -03:00
pre-commit-ci[bot]
290b322a5d [pre-commit.ci] pre-commit autoupdate (#755)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-02-01 08:34:13 -03:00
Bruno Oliveira
18143275c6 Remove obsolete skip based on java (no longer supported) 2021-12-27 08:05:46 -03:00
Bruno Oliveira
d21f59fea7 Fix typing errors after updating mypy 2021-12-27 08:05:01 -03:00
Bruno Oliveira
13f39349c6 Remove unnecessary skip from test_logfinish_hook as we require pytest>=6.2
Some checks failed
build / build (ubuntu-latest, 3.10-dev, py310-pytestlatest) (push) Failing after 2s
build / build (ubuntu-latest, 3.6, py36-pytestlatest) (push) Failing after 2s
build / build (ubuntu-latest, 3.7, py37-pytestlatest) (push) Failing after 1s
build / build (ubuntu-latest, 3.8, py38-psutil) (push) Failing after 2s
build / build (ubuntu-latest, 3.8, py38-pytestlatest) (push) Failing after 1s
build / build (ubuntu-latest, 3.8, py38-pytestmain) (push) Failing after 2s
build / build (ubuntu-latest, 3.8, py38-setproctitle) (push) Failing after 2s
build / build (ubuntu-latest, 3.9, py39-pytestlatest) (push) Failing after 3s
build / build (windows-latest, 3.10-dev, py310-pytestlatest) (push) Has been cancelled
build / build (windows-latest, 3.6, py36-pytestlatest) (push) Has been cancelled
build / build (windows-latest, 3.7, py37-pytestlatest) (push) Has been cancelled
build / build (windows-latest, 3.8, py38-psutil) (push) Has been cancelled
build / build (windows-latest, 3.8, py38-pytestlatest) (push) Has been cancelled
build / build (windows-latest, 3.8, py38-pytestmain) (push) Has been cancelled
build / build (windows-latest, 3.8, py38-setproctitle) (push) Has been cancelled
build / build (windows-latest, 3.9, py39-pytestlatest) (push) Has been cancelled
build / deploy (push) Has been cancelled
2021-12-10 08:32:32 -03:00
Bruno Oliveira
c76d5622f1 Skip test_warning_captured_deprecated_in_pytest_6 in pytest>=7.1 2021-12-10 08:31:55 -03:00
baekdohyeop
62e50d0097 Address review 2021-11-29 02:59:18 +09:00
baekdohyeop
52a395888f Create new dist option 'loadgroup' 2021-11-29 02:59:18 +09:00
Bruno Oliveira
6d83034bb8 Fix test_warning_captured_deprecated_in_pytest_6
This test started to fail in the 'py38-pytestmain' environment, the
cause being PytestRemovedIn7Warning being raised by the conftest file
of the test itself. Ignoring it is fine, the purpose of the test is to ensure
the hook is not called by pytest-xdist.
2021-11-29 02:59:18 +09:00
Ran Benita
2bac85562b Merge pull request #728 from bluetech/hookimpl-decorators
Add explicit pytest.hookspec/hookimpl annotations, avoid legacy tryfirst/trylast marks
2021-11-02 12:28:36 +02:00
Ran Benita
b85b71cd22 Add explicit pytest.hookspec/hookimpl annotations, avoid legacy tryfirst/trylast marks 2021-11-02 09:42:00 +02:00
Ran Benita
89a3df3e20 Deprecate --boxed
This will allow removing the pytest-forked dependency in the future.

Refs #468.
2021-11-02 09:38:56 +02:00
Ran Benita
9ddb274f23 Update test suite to modern pytest
- Use pytest>=6.2 features (same as install_requires).
  When we can require >=7, can fix some more typing omissions and
  version checks.
- Replace testdir with pytester
- Replace py.test with pytest
- Replace tmpdir with tmp_path
- Remove (almost) all other uses of py
- Add some type annotations (not checked yet)

Ref #722.
2021-11-01 09:05:28 +02:00
Bruno Oliveira
90fc616906 Fix test_fixture_teardown_failure for pytest master
`--debug` now receives an optional file, so the way the test
was calling pytest made it seem like it was passing the python file
file as the debug file.
2021-09-01 07:54:13 -03:00
Ronny Pfannschmidt
b02a6db771 Merge pull request #667 from graingert/fix-sys-path
fix sys.path for local workers  Fixes #421
2021-06-16 11:40:55 +02:00
Ronny Pfannschmidt
7f07d50808 Merge pull request #646 from kroeschl/numprocesses-logical
Add explicit 'logical' choice to --numprocesses flag
2021-06-08 08:44:29 +02:00
Thomas Grainger
02f971d45f swap docstring 2021-06-03 21:46:21 +01:00
Thomas Grainger
56e631b4bb fix sys.path for local workers Fixes #421 2021-06-03 21:20:11 +01:00
Kyle Roeschley
4d27110beb Add 'logical' CPU count choice for --numprocesses
This can significantly speed up runs when testing is not CPU-bound.
2021-04-27 08:47:21 -05:00
xoviat
f861f1667d add new pytest_handlecrashitem hook
allows handling and rescheduling crash tests

PR #651

Co-authored-by: xoviat <xoviat@users.noreply.github.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2021-04-27 09:07:49 -03:00
Adam Johnson
f02aa70c95 Upgrade pre-commit hooks
Use `pre-commit autoupdate` to upgrade, and follow the message from `pre-commit-hooks` to move to the official `flake8` hook.
2021-03-20 11:13:59 -03:00
Ronny Pfannschmidt
166bdb4103 prepare #592: replace master with controller where we can 2021-02-07 22:08:22 +01:00
Bruno Oliveira
16d636391b Propagate internal errors to the master node
This should help users diagnose internal errors in workers like
exceptions from hooks or in pytest itself.
2020-12-13 20:05:31 -03:00
Bruno Oliveira
b3a1bf3c2e Fix test_warning_captured_deprecated_in_pytest_6
The test was failing because *other* warnings were being triggered on the
master node. This makes the test more reliable by checking only that
the warning from the worker node is not emitted.

Fix #601
2020-12-12 11:58:41 -03:00
Bruno Oliveira
607d8288d4 Add psutil extra and introduce pytest_xdist_auto_num_workers hook
This makes using psutil optional and opens up the possibility of customization
through the pytest_xdist_auto_num_workers hook, making things like #477 possible.

Fix #585
2020-08-25 08:38:28 -03:00
Bruno Oliveira
e469fc7b80 Revert "Merge pull request #560 from utapyngo/logical-cpu-count"
This reverts commit 0094b29a14, reversing
changes made to c6255faad4.
2020-08-24 14:50:36 -03:00
Christine M
c211d247a7 Noisy rsync (#473)
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2020-08-12 17:23:03 -03:00
Zac-HD
d153e0a4c4 Remove compat for pytest < 6 2020-08-06 15:48:33 +10:00
Sylvain Marié
ba8398478e New methods is_xdist_worker, is_xdist_master, get_xdist_worker_id. (#505)
Co-authored-by: Sylvain MARIE <sylvain.marie@se.com>
Co-authored-by: Zac Hatfield-Dodds <Zac-HD@users.noreply.github.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2020-08-05 20:41:54 -03:00
Bruno Oliveira
047816f33a Drop features from old pytest versions 2020-08-05 19:55:56 -03:00