kevlened / pytest-parallel Goto Github PK

View Code? Open in Web Editor NEW

312.0 10.0 59.0 157 KB

A pytest plugin for parallel and concurrent testing

Home Page: https://github.com/browsertron/pytest-parallel/issues/104#issuecomment-1293941066

License: MIT License

Python 97.98% Shell 2.02%

pytest parallel concurrent

pytest-parallel's Introduction

Maintainers needed

The project is currently unmaintained

pytest-parallel

a pytest plugin for parallel and concurrent testing

What?

This plugin makes it possible to run tests quickly using multiprocessing (parallelism) and multithreading (concurrency).

Why?

pytest-xdist is great to run tests that:

aren't threadsafe
perform poorly when multithreaded
need state isolation

pytest-parallel is better for some use cases (like Selenium tests) that:

can be threadsafe
can use non-blocking IO for http requests to make it performant
manage little or no state in the Python environment

Put simply, pytest-xdist does parallelism while pytest-parallel does parallelism and concurrency.

Requirements

Python3 version [3.6+]
Unix or Mac for --workers
Unix, Mac, or Windows for --tests-per-worker

Installation

pip install pytest-parallel

Options

workers (optional) - max workers (aka processes) to start. Can be a positive integer or auto which uses one worker per core. Defaults to 1.
tests-per-worker (optional) - max concurrent tests per worker. Can be a positive integer or auto which evenly divides tests among the workers up to 50 concurrent tests. Defaults to 1.

Examples

# runs 2 workers with 1 test per worker at a time
pytest --workers 2

# runs 4 workers (assuming a quad-core machine) with 1 test per worker
pytest --workers auto

# runs 1 worker with 4 tests at a time
pytest --tests-per-worker 4

# runs 1 worker with up to 50 tests at a time
pytest --tests-per-worker auto

# runs 2 workers with up to 50 tests per worker
pytest --workers 2 --tests-per-worker auto

Notice

Beginning with Python 3.8, forking behavior is forced on macOS at the expense of safety.

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

Source

License

MIT

pytest-parallel's People

Contributors

Stargazers

Watchers

pytest-parallel's Issues

Does't not work in Windows

I saw requirements for Unix but still would be nice if you figure that for windows

Pytest --parallel multiprocessing error

Hi Kevlened,

I've been trying to run parallel selenium tests on Browserstack using pytest parallel but I've been failing to do so due to an dependency error because of the multiprocessing package. This issue is likely due to a misunderstanding on my part, but I am unable find the solution for it on Google or Stack Overflow. I would dearly appreciate your assistance.

I will briefly describe what I am attempting to do and share the error log with you.

I was inspired by this video to run 3 tests in parallel on Browserstack and I wrote a simple login test for three different platforms (namely Windows, iOS and Anroid) in PyCharm to run with pytest.

My project structure is as follows:

The test_loginTest.py file looks like this and I am using an interpreter with version Python 3.7.3 through Anaconda:

from ....Parallel import login_test_android,login_test_ios,login_test_windows

def test_login_test_windows():
    login_test_windows()
    assert True

def test_login_test_ios():
    login_test_ios()
    assert True

def test_login_test_android():
    login_test_android()
    assert True

I use the following command on the Command Line tool to successfully run my tests:

python -m pytest test_loginTest.py
(note: somehow running the command pytest test_loginTest.py fails to run as it fails to import the selenium package)

I then later use the following command on the Command Line tool to run tests in parallel on Browserstack:
python -m pytest --workers 3 test_loginTest.py

This command causes the following errors:

===================================================================================test session starts ===================================================================================
platform win32 -- Python 3.7.3, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: C:\Users\User\PycharmProjects\testin_PoC
plugins: Faker-4.1.2, arraydiff-0.3, doctestplus-0.3.0, openfiles-0.3.2, parallel-0.1.0, remotedata-0.3.1
collected 3 items                                                                                                                                                                                                   
pytest-parallel: 1 worker (process), 1 test per worker (thread)
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\_pytest\main.py", line 213, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\_pytest\main.py", line 257, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pluggy\hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pluggy\manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pluggy\manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\site-packages\pytest_parallel\__init__.py", line 302, in pytest_runtestloop
INTERNALERROR>     process.start()
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\multiprocessing\process.py", line 112, in start
INTERNALERROR>     self._popen = self._Popen(self)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
INTERNALERROR>     return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
INTERNALERROR>     return Popen(process_obj)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
INTERNALERROR>     reduction.dump(process_obj, to_child)
INTERNALERROR>   File "C:\Users\User\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
INTERNALERROR>     ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'

===================================================================================no tests ran in 0.18 seconds ===================================================================================
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\User\Anaconda3\lib\multiprocessing\connection.py", line 312, in _recv_bytes
    nread, err = ov.GetOverlappedResult(True)
BrokenPipeError: [WinError 109] The pipe has been ended

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\Anaconda3\lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "C:\Users\User\Anaconda3\lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\User\Anaconda3\lib\site-packages\pytest_parallel\__init__.py", line 348, in process_responses
    event_name, kwargs = queue.get()
  File "<string>", line 2, in get
  File "C:\Users\User\Anaconda3\lib\multiprocessing\managers.py", line 796, in _callmethod
    kind, result = conn.recv()
  File "C:\Users\User\Anaconda3\lib\multiprocessing\connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "C:\Users\User\Anaconda3\lib\multiprocessing\connection.py", line 321, in _recv_bytes
    raise EOFError
EOFError

Problem with loading terminalreporter

I have problem to run pytest-parallel:

Traceback (most recent call last):
File "/venv/lib64/python3.6/site-packages/_pytest/main.py", line 174, in wrap_session
config._do_configure()
File "/venv/lib64/python3.6/site-packages/_pytest/config/init.py", line 607, in _do_configure
self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
File "/venv/lib64/python3.6/site-packages/pluggy/hooks.py", line 280, in call_historic
res = self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
File "/venv/lib64/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/venv/lib64/python3.6/site-packages/pluggy/manager.py", line 61, in
firstresult=hook.spec_opts.get('firstresult'),
File "/venv/lib64/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
return outcome.get_result()
File "/venv/lib64/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
raise ex[1].with_traceback(ex[2])
File "/venv/lib64/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
res = hook_impl.function(*args)
File "/venv/lib64/python3.6/site-packages/pytest_parallel/init.py", line 93, in pytest_configure
config.pluginmanager.register(ParallelRunner(config), 'parallelrunner')
File "/venv/lib64/python3.6/site-packages/pytest_parallel/init.py", line 234, in init
reporter.showfspath = False
AttributeError: 'NoneType' object has no attribute 'showfspath'

plugins: requests-mock-1.5.2, xdist-1.23.2, sugar-0.9.2, forked-0.2, django-3.4.3, django-ordering-1.0.1, cov-2.5.1, celery-4.2.1

session-scope fixtures aren't honored

Similar to what happens with pytest-xdist: pytest-dev/pytest-xdist#271

Getting "Type Error: Cannot create a consistent method resolution" during execution

Getting below error
class ThreadLocalSetupState(threading.local, _pytest.runner.SetupState):
E TypeError: Cannot create a consistent method resolution
E order (MRO) for bases _local, ThreadLocalSetupState

Versions and OS details:
OS:Windows
Python 3.6.5
pytest-5.2.2
pytest-parallel-0.0.9

Please support Python 3.9

When I run pytest . --workers auto with Python 3.9, this error occur.

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/_pytest/main.py", line 255, in wrap_session
INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pytest_parallel/__init__.py", line 219, in pytest_sessionstart
INTERNALERROR>     os.environ = ThreadLocalEnviron(os.environ)
INTERNALERROR>   File "/.../venv/lib/python3.9/site-packages/pytest_parallel/__init__.py", line 122, in __init__
INTERNALERROR>     env.putenv,
INTERNALERROR> AttributeError: '_Environ' object has no attribute 'putenv'

So I check the code, it uses os.environ.putenv but Python doesn't have the API. Then only have os.putenv.

On Python 3.8 they are exists.

Python 3.8.6 (default, Dec 11 2020, 14:45:03)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> e = os.environ
>>> e.putenv
<built-in function putenv>
>>> os.putenv == e.putenv
True

But 3.9 does not

Python 3.9.0 (default, Oct  7 2020, 23:09:01)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> e = os.environ
>>> e.putenv
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: '_Environ' object has no attribute 'putenv'

and definition of _Environ is also changed.

class _Environ(MutableMapping):
    def __init__(self, data, encodekey, decodekey, encodevalue, decodevalue):
        self.encodekey = encodekey
        self.decodekey = decodekey
        self.encodevalue = encodevalue
        self.decodevalue = decodevalue
        self._data = data

AssertionError: Expected current collector to be...

self._collectors:
  <Collector at 0x7f7a3779d8d0: CTracer>
                      <module> : /usr/local/bin/pytest @10
                          main : /usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py @60
                _prepareconfig : /usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py @195
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_cmdline_parse : /usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py @654
                         parse : /usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py @871
                     _preparse : /usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py @826
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
 pytest_load_initial_conftests : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @99
                      __init__ : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @146
                         start : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @168
                         start : /usr/local/lib/python3.7/site-packages/pytest_cov/engine.py @159
                         erase : /usr/local/lib/python3.7/site-packages/coverage/control.py @726
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a35d7cba8: CTracer>
                      <module> : /usr/local/bin/pytest @10
                          main : /usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py @79
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
           pytest_cmdline_main : /usr/local/lib/python3.7/site-packages/_pytest/main.py @242
                  wrap_session : /usr/local/lib/python3.7/site-packages/_pytest/main.py @209
                         _main : /usr/local/lib/python3.7/site-packages/_pytest/main.py @249
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
            pytest_runtestloop : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @342
                         start : /usr/local/lib/python3.7/multiprocessing/process.py @112
                        _Popen : /usr/local/lib/python3.7/multiprocessing/context.py @223
                        _Popen : /usr/local/lib/python3.7/multiprocessing/context.py @277
                      __init__ : /usr/local/lib/python3.7/multiprocessing/popen_fork.py @20
                       _launch : /usr/local/lib/python3.7/multiprocessing/popen_fork.py @74
                    _bootstrap : /usr/local/lib/python3.7/multiprocessing/process.py @290
            _run_after_forkers : /usr/local/lib/python3.7/multiprocessing/util.py @132
         multiprocessing_start : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @23
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a35ce4ef0: CTracer>
                    _bootstrap : /usr/local/lib/python3.7/threading.py @885
              _bootstrap_inner : /usr/local/lib/python3.7/threading.py @917
                           run : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @81
                      run_test : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @51
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
       pytest_runtest_protocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @78
               runtestprotocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @87
               call_and_report : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @173
             call_runtest_hook : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                     from_call : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @226
                      <lambda> : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_runtest_setup : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @287
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a35d68e10: CTracer>
                    _bootstrap : /usr/local/lib/python3.7/threading.py @885
              _bootstrap_inner : /usr/local/lib/python3.7/threading.py @917
                           run : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @81
                      run_test : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @51
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
       pytest_runtest_protocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @78
               runtestprotocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @87
               call_and_report : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @173
             call_runtest_hook : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                     from_call : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @226
                      <lambda> : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_runtest_setup : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @287
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @246
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a34483f60: CTracer>
                    _bootstrap : /usr/local/lib/python3.7/threading.py @885
              _bootstrap_inner : /usr/local/lib/python3.7/threading.py @917
                           run : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @74
                      run_test : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @51
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
       pytest_runtest_protocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @78
               runtestprotocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @87
               call_and_report : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @173
             call_runtest_hook : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                     from_call : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @226
                      <lambda> : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_runtest_setup : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @287
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a34483898: CTracer>
                    _bootstrap : /usr/local/lib/python3.7/threading.py @885
              _bootstrap_inner : /usr/local/lib/python3.7/threading.py @917
                           run : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @74
                      run_test : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @51
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
       pytest_runtest_protocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @78
               runtestprotocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @87
               call_and_report : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @173
             call_runtest_hook : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                     from_call : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @226
                      <lambda> : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_runtest_setup : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @287
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a34446ac8: CTracer>
                    _bootstrap : /usr/local/lib/python3.7/threading.py @885
              _bootstrap_inner : /usr/local/lib/python3.7/threading.py @917
                           run : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @74
                      run_test : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @51
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
       pytest_runtest_protocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @78
               runtestprotocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @87
               call_and_report : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @173
             call_runtest_hook : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                     from_call : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @226
                      <lambda> : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_runtest_setup : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @287
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
  <Collector at 0x7f7a344b8be0: CTracer>
                    _bootstrap : /usr/local/lib/python3.7/threading.py @885
              _bootstrap_inner : /usr/local/lib/python3.7/threading.py @917
                           run : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @74
                      run_test : /usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py @51
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
       pytest_runtest_protocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @78
               runtestprotocol : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @87
               call_and_report : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @173
             call_runtest_hook : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                     from_call : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @226
                      <lambda> : /usr/local/lib/python3.7/site-packages/_pytest/runner.py @198
                      __call__ : /usr/local/lib/python3.7/site-packages/pluggy/hooks.py @289
                     _hookexec : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @68
                      <lambda> : /usr/local/lib/python3.7/site-packages/pluggy/manager.py @62
                    _multicall : /usr/local/lib/python3.7/site-packages/pluggy/callers.py @187
          pytest_runtest_setup : /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py @287
                          init : /usr/local/lib/python3.7/site-packages/pytest_cov/embed.py @67
                          load : /usr/local/lib/python3.7/site-packages/coverage/control.py @675
                         _init : /usr/local/lib/python3.7/site-packages/coverage/control.py @266
                      __init__ : /usr/local/lib/python3.7/site-packages/coverage/collector.py @107
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/multiprocessing/util.py", line 265, in _run_finalizers
    finalizer()
  File "/usr/local/lib/python3.7/multiprocessing/util.py", line 189, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/site-packages/pytest_cov/embed.py", line 83, in cleanup
    _cleanup(cov)
  File "/usr/local/lib/python3.7/site-packages/pytest_cov/embed.py", line 76, in _cleanup
    cov.stop()
  File "/usr/local/lib/python3.7/site-packages/coverage/control.py", line 707, in stop
    self.collector.stop()
  File "/usr/local/lib/python3.7/site-packages/coverage/collector.py", line 328, in stop
    "Expected current collector to be %r, but it's %r" % (self, self._collectors[-1])
AssertionError: Expected current collector to be <Collector at 0x7f7a35d7cba8: CTracer>, but it's <Collector at 0x7f7a344b8be0: CTracer>

Separate logging

Is it possible to have a separate log file output for each worker?

Tavern integration

Does this project play nice with tavern?

I installed and then ran my tests with command: pytest --tests-per-worker auto, but all tests fail.

I'm running about 10 separate integration test files.

taverntesting/tavern#534

Pytest-parallel and pytest-timeout hang when used together

This isn't a bug of pytest-parallel, but I wanted to raise it as a noteworthy issue and leave up to you to decide if it's worth working around pytest-timeout somehow.

Background: https://pypi.org/project/pytest-timeout/
(Potentially) Relevant issue for python-xdist: https://bitbucket.org/pytest-dev/pytest-timeout/issues/8/pytest-xdist-interop-signal-only-works-in

If you have any questions I'll be happy to answer.

Summary:
Setup:

pip install pytest-parallel==0.0.8
pip install pytest-timeout==1.3.2

Config (filename: pytest.ini):

[pytest]
filterwarnings =
    ignore:::tensorflow
timeout = 1
addopts = --workers auto

Test File(filename: fail.py):

def test_fail():
    assert False

def test_fail_2():
    assert False

Command:

Eliass-MacBook-Pro:diseaseTools etragas$ py_test fail.py

Output:

================================================================================== test session starts ==================================================================================
platform linux -- Python 3.6.7, pytest-3.8.0, py-1.7.0, pluggy-0.8.0
rootdir: /usr/src/app, inifile: pytest.ini
plugins: snapshottest-0.5.0, timeout-1.3.2, parallel-0.0.8, mock-1.5.0, cov-2.5.1, celery-4.2.1
timeout: 1.0s
timeout method: signal
timeout func_only: False
collected 2 items
pytest-parallel: 6 workers (processes), 1 test per worker (thread)
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 80, in run
    run_test(self.session, item, None)
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 50, in run_test
    item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 284, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 182, in _multicall
    next(gen)  # first yield
  File "/usr/local/lib/python3.6/site-packages/pytest_timeout.py", line 91, in pytest_runtest_protocol
    timeout_setup(item)
  File "/usr/local/lib/python3.6/site-packages/pytest_timeout.py", line 144, in timeout_setup
    signal.signal(signal.SIGALRM, handler)
  File "/usr/local/lib/python3.6/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 80, in run
    run_test(self.session, item, None)
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 50, in run_test
    item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 284, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 182, in _multicall
    next(gen)  # first yield
  File "/usr/local/lib/python3.6/site-packages/pytest_timeout.py", line 91, in pytest_runtest_protocol
    timeout_setup(item)
  File "/usr/local/lib/python3.6/site-packages/pytest_timeout.py", line 144, in timeout_setup
    signal.signal(signal.SIGALRM, handler)
  File "/usr/local/lib/python3.6/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

cannot work with pytest‘s fixture

python 3.7.4
pytest 5.2.0
pytest-parallel 0.2.2
gevent 1.4.0

the scope of fixture, for example: session，module, they do no work.

INTERNALERROR> RuntimeError: pytest-parallel got 20 errors, raising the first from Thread-7.

platform linux -- Python 3.7.5, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
I suddenly started getting below error in my git lab CI pipeline .

INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pytest_parallel/init.py", line 86, in run
INTERNALERROR> run_test(self.session, item, None)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pytest_parallel/init.py", line 51, in run_test
INTERNALERROR> item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in call
INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/manager.py", line 87, in
INTERNALERROR> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/callers.py", line 182, in _multicall
INTERNALERROR> next(gen) # first yield
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pytest_reportportal/listener.py", line 36, in pytest_runtest_protocol
INTERNALERROR> item_id = self.PyTestService.start_pytest_item(item)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pytest_reportportal/service.py", line 232, in start_pytest_item
INTERNALERROR> self.parent_item_id = self._hier_parts[part]["item_id"]
INTERNALERROR> KeyError: 'item_id'
INTERNALERROR>
INTERNALERROR> The above exception was the direct cause of the following exception:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/_pytest/main.py", line 197, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in call
INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/manager.py", line 87, in
INTERNALERROR> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/builds/core-tech/tools/qa/cstudio_automation/venv/lib/python3.7/site-packages/pytest_parallel/init.py", line 397, in pytest_runtestloop
INTERNALERROR> six.raise_from(exc, err[1])
INTERNALERROR> File "", line 3, in raise_from
INTERNALERROR> RuntimeError: pytest-parallel got 20 errors, raising the first from Thread-7.

AttributeError: 'ThreadLocalEnviron' object has no attribute 'copy'

I was using delegator package and had this error, We probably need a copy function in ThreadLocalEnviron class, like this
def copy(self):
return dict(self)

ERROR:
c = delegator.run('ip netns')
../build/debug/test/pyshl-infra/venv/lib64/python3.6/site-packages/delegator.py:280: in run
c.run(block=block, binary=binary, cwd=cwd, env=env)
../build/debug/test/pyshl-infra/venv/lib64/python3.6/site-packages/delegator.py:146: in run
popen_kwargs = self._default_popen_kwargs.copy()

self = <Command 'ip netns'>

@property
def _default_popen_kwargs(self):
    return {

      'env': os.environ.copy(),

        'stdin': subprocess.PIPE,
        'stdout': subprocess.PIPE,
        'stderr': subprocess.PIPE,
        'shell': True,
        'universal_newlines': True,
        'bufsize': 0
    }

E AttributeError: 'ThreadLocalEnviron' object has no attribute 'copy'

../build/debug/test/pyshl-infra/venv/lib64/python3.6/site-packages/delegator.py:40: AttributeError

pytest allure report not generated when using this parallel

hello
i am generating allure report and working fine.
but when i will add --workers argument then report files are not generating.

Thanks

Can't iterate over multiprocessing.managers.DictProxy

Can't iterate over multiprocessing.managers.DictProxy. through pytest-paralell but works fine in python and pytest.

test_Run.py
---------------
from multiprocessing import Process, Manager

def f(d, l):
    d[1] = '1'
    d['2'] = 2
    d[0.25] = None
    l.reverse()

if __name__ == '__main__':
    with Manager() as manager:
    d = manager.dict()
    l = manager.list(range(10))

    p = Process(target=f, args=(d, l))
    p.start()
    p.join()

    print(d)
    print(l)

If you run the

(venv) [tivo@localhost src]$ python test_run.py 
{0.25: None, 1: '1', '2': 2}
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
(venv) [tivo@localhost src]$

for pytest:

from multiprocessing import Process, Manager
def test_f():
    d, l = {}, []
    d[1] = '1'
    d['2'] = 2
    d[0.25] = None
    print(d)

with Manager() as manager:
    d = manager.dict()
    l = manager.list(range(10))
   l.reverse()
   print(l)
   p = Process(target=f)
   p.start()
   p.join()

(venv) [tivo@localhost src]$ pytest -v -s test_run.py 
collecting ... [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
{0.25: None, 1: '1', '2': 2}
collected 1 item                                                                                                                                               

test_run.py::test_f {0.25: None, 1: '1', '2': 2}
PASSED

(venv) [tivo@localhost src]$

But If you run through pytest along with pytest-paralell package, it throws error

(venv) [tivo@localhost src]$ pytest -v -s --tests-per-worker auto --workers auto test_run.py 
===================================================================== test session starts ======================================================================
platform linux -- Python 3.4.4, pytest-4.5.0, py-1.8.0, pluggy-0.11.0 -- /home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/src, inifile: pytest.ini
plugins: xdist-1.28.0, remotedata-0.3.1, pipeline-0.3.0, parallel-0.0.9, forked-1.0.2, flake8-1.0.4, cov-2.7.1
collecting ... [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
{0.25: None, 1: '1', '2': 2}
collected 1 item                                                                                                                                               
pytest-parallel: 2 workers (processes), 1 test per worker (thread)

test_run.py::test_f {0.25: None, 1: '1', '2': 2}
PASSED
Traceback (most recent call last):
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/bin/pytest", line 10, in <module> sys.exit(main())
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/_pytest/config/__init__.py", line 79, in main return config.hook.pytest_cmdline_main(config=config)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 289, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/manager.py", line 68, in _hookexec return self._inner_hookexec(hook, methods, kwargs)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/manager.py", line 62, in <lambda> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/callers.py", line 208, in _multicall return outcome.get_result()
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/callers.py", line 80, in get_result raise ex[1].with_traceback(ex[2])
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/callers.py", line 187, in _multicall res = hook_impl.function(*args)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/_pytest/main.py", line 242, in pytest_cmdline_main return wrap_session(config, _main)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/_pytest/main.py", line 235, in wrap_session session=session, exitstatus=session.exitstatus
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 289, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/manager.py", line 68, in _hookexec return self._inner_hookexec(hook, methods, kwargs)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/manager.py", line 62, in <lambda> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/pluggy/callers.py", line 203, in _multicall gen.send(outcome)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/_pytest/terminal.py", line 678, in pytest_sessionfinish self.summary_stats()
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/_pytest/terminal.py", line 876, in summary_stats (line, color) = build_summary_stats_line(self.stats)
  File "/home/tivo/workspace/ServicePortal/autotestscripts/CAT/scripts/ServerQE/brat/venv/lib/python3.4/site-packages/_pytest/terminal.py", line 1034, in build_summary_stats_line for found_type in stats:
  File "<string>", line 2, in __getitem__
  File "/usr/local/lib/python3.4/multiprocessing/managers.py", line 747, in _callmethod raise convert_to_error(kind, result)
 KeyError: 0
(venv) [tivo@localhost src]$

I have the following packages with me

pytest-parallel==0.0.9
pytest-pipeline==0.3.0

Q. What workaround can I do to get this above code PASSED without Error logs ? The issue being the results are not giving me output as to how many test cases are PASSED.

Why is it taking pytest-parallel: 2 workers (processes), 1 test per worker (thread) ? I have provided only one function there !

HINT:

If I add flags --worker 1; this above error does not appear; but usually fails my scripts and hence I am forced to use --tests-per-worker 1 along with it. But paralellism is not present here !

Pytest parallel hangs on repeated failures

Still debugging in more detail but throwing this up in case you see an obvious fix!

platform linux -- Python 3.6.6, pytest-3.8.0, py-1.6.0, pluggy-0.7.1
rootdir: /usr/src/app, inifile: pytest.ini
plugins: snapshottest-0.5.0, timeout-1.3.2, parallel-0.0.7, mock-1.5.0, cov-2.5.1, celery-4.2.1
collected 44 items
pytest-parallel: 6 workers (processes), 1 test per worker (thread)
.xx.x....x.x...x...x....F..F...F..Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 80, in run
    run_test(self.session, item, None)
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 50, in run_test
    item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 66, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 81, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
  File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 165, in call_and_report
    hook.pytest_runtest_logreport(report=report)
  File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 402, in pytest_runtest_logreport
    self.testsfailed += 1
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int'

.F.......^C^C
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 178, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 215, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.6/site-packages/pytest_parallel/__init__.py", line 294, in pytest_runtestloop
    queue.join()
  File "<string>", line 2, in join
  File "/usr/local/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod
    kind, result = conn.recv()
  File "/usr/local/lib/python3.6/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/usr/local/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/local/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt

Does the supported Python version include 3.6?

hello,Does the supported Python version include 3.6?

Pytest UserWarning is not coming up on std out for 0.1.0 release

We have a few parallel tests that often generates user-defined warnings.
The way we use these warnings is via stdout, which are later picked up by builbot and highlighted accordingly.

The tests work perfectly with respect to warnings with library 0.0.10.
However, if we use the 0.1.0 version, the warnings are not coming up on the stdout.

TypeError when use gevent and pytest-parallel

Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/site-packages/pytest_parallel/init.py", line 81, in run
run_test(self.session, item, None)
File "/usr/local/lib/python3.7/site-packages/pytest_parallel/init.py", line 51, in run_test
item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
return outcome.get_result()
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 72, in pytest_runtest_protocol
runtestprotocol(item, nextitem=nextitem)
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 88, in runtestprotocol
reports.append(call_and_report(item, "teardown", log, nextitem=nextitem))
File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 171, in call_and_report
hook.pytest_runtest_logreport(report=report)
File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
return outcome.get_result()
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 434, in pytest_runtest_logreport
if maxfail and self.testsfailed >= maxfail:
TypeError: '>=' not supported between instances of 'SafeNumber' and 'int'

Test failing

I am packaging this for Archlinux and running the tests with python setup.py pytest.

============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /build/python-pytest-parallel/src/pytest-parallel-0.0.10
plugins: parallel-0.0.10
collected 31 items

tests/test_concurrent.py .                                               [  3%]
tests/test_general.py ......F..F..F..F..F.FFF.F....                      [ 96%]
tests/test_safenumber.py .                                               [100%]

=================================== FAILURES ===================================
_________________________ test_environ_shim[cli_args1] _________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_environ_shim1')>
cli_args = ['--workers=2']

>   ???

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.INTERNAL_ERROR len(stdout.lines)=7 len(stderr.lines)=60 duration=0.11s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_environ_shim1
plugins: parallel-0.0.10
collected 1 item
pytest-parallel: 2 workers (processes), 1 test per worker (thread)

----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 924, in runpytest_inprocess
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 890, in inline_run
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 232, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 701, in pytest_sessionfinish
    self.summary_stats()
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 912, in summary_stats
    (parts, main_color) = build_summary_stats_line(self.stats)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 1134, in build_summary_stats_line
    count = sum(
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 1134, in <genexpr>
    count = sum(
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 298, in serve_client
    send(msg)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function unpickle_exception at 0x7f67e4eb7040>: it's not the same object as tblib.pickling_support.unpickle_exception

---------------------------------------------------------------------------
_________________________ test_skip_markers[cli_args1] _________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_skip_markers1')>
cli_args = ['--workers=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_skip_markers(testdir, cli_args):
        testdir.makepyfile("""
            import pytest

            def test_1():
                assert 1 == 1

            @pytest.mark.skip(reason="because")
            def test_2():
                assert 2 == 2
        """)
        result = testdir.runpytest(*cli_args)
>       result.assert_outcomes(passed=1, skipped=1)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:125:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.INTERNAL_ERROR len(stdout.lines)=7 len(stderr.lines)=60 duration=0.11s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_skip_markers1
plugins: parallel-0.0.10
collected 2 items
pytest-parallel: 2 workers (processes), 1 test per worker (thread)

----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 924, in runpytest_inprocess
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 890, in inline_run
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 232, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 701, in pytest_sessionfinish
    self.summary_stats()
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 912, in summary_stats
    (parts, main_color) = build_summary_stats_line(self.stats)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 1134, in build_summary_stats_line
    count = sum(
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 1134, in <genexpr>
    count = sum(
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 298, in serve_client
    send(msg)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function unpickle_exception at 0x7f67e42c5a60>: it's not the same object as tblib.pickling_support.unpickle_exception

---------------------------------------------------------------------------
________________________ test_skipif_markers[cli_args1] ________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_skipif_markers1')>
cli_args = ['--workers=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_skipif_markers(testdir, cli_args):
        testdir.makepyfile("""
            import pytest

            def test_1():
                assert 1 == 1

            @pytest.mark.skipif(True, reason="because")
            def test_2():
                assert 2 == 2

            @pytest.mark.skipif(
                "config.getoption('workers') or not config.getoption('workers')",
                reason="because"
            )
            def test_3():
                assert 3 == 3
        """)
        result = testdir.runpytest(*cli_args)
>       result.assert_outcomes(passed=1, skipped=2)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:153:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.INTERNAL_ERROR len(stdout.lines)=7 len(stderr.lines)=60 duration=0.12s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_skipif_markers1
plugins: parallel-0.0.10
collected 3 items
pytest-parallel: 2 workers (processes), 1 test per worker (thread)

----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 924, in runpytest_inprocess
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 890, in inline_run
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 232, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 701, in pytest_sessionfinish
    self.summary_stats()
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 912, in summary_stats
    (parts, main_color) = build_summary_stats_line(self.stats)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 1134, in build_summary_stats_line
    count = sum(
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 1134, in <genexpr>
    count = sum(
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 298, in serve_client
    send(msg)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function unpickle_exception at 0x7f67e418f310>: it's not the same object as tblib.pickling_support.unpickle_exception

---------------------------------------------------------------------------
________________________ test_custom_markers[cli_args1] ________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_custom_markers1')>
cli_args = ['--workers=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_custom_markers(testdir, cli_args):
        testdir.makepyfile("""
            import pytest

            def test_1():
                assert 1 == 1

            @pytest.mark.marked
            def test_2():
                assert 2 == 2
        """)
        result = testdir.runpytest('-m marked', *cli_args)
>       result.assert_outcomes(passed=1, skipped=0)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.INTERNAL_ERROR len(stdout.lines)=7 len(stderr.lines)=78 duration=0.12s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_custom_markers1
plugins: parallel-0.0.10
collected 2 items / 1 deselected / 1 selected
pytest-parallel: 2 workers (processes), 1 test per worker (thread)

----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 924, in runpytest_inprocess
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 890, in inline_run
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 232, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 691, in pytest_sessionfinish
    self.config.hook.pytest_terminal_summary(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 182, in _multicall
    next(gen)  # first yield
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 707, in pytest_terminal_summary
    self.summary_warnings()
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 802, in summary_warnings
    for wr in warning_reports:
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 298, in serve_client
    send(msg)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function unpickle_exception at 0x7f67e40e6820>: it's not the same object as tblib.pickling_support.unpickle_exception

---------------------------------------------------------------------------
______________________ test_multiple_failures[cli_args1] _______________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_multiple_failures1')>
cli_args = ['--workers=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_multiple_failures(testdir, cli_args):
        testdir.makepyfile("""
            def test_0():
                assert 1 == 2

            def test_1():
                assert True == False

            def test_2():
                assert 1 == 2

            def test_3():
                assert True == False
        """)
        result = testdir.runpytest(*cli_args)
>       result.assert_outcomes(failed=4)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:198:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.INTERNAL_ERROR len(stdout.lines)=7 len(stderr.lines)=80 duration=0.15s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_multiple_failures1
plugins: parallel-0.0.10
collected 4 items
pytest-parallel: 2 workers (processes), 1 test per worker (thread)

----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 924, in runpytest_inprocess
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 890, in inline_run
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 232, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 691, in pytest_sessionfinish
    self.config.hook.pytest_terminal_summary(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 182, in _multicall
    next(gen)  # first yield
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 706, in pytest_terminal_summary
    self.summary_failures()
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 864, in summary_failures
    reports = self.getreports("failed")
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 777, in getreports
    for x in self.stats.get(name, []):
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 298, in serve_client
    send(msg)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function unpickle_exception at 0x7f67df6f1af0>: it's not the same object as tblib.pickling_support.unpickle_exception

---------------------------------------------------------------------------
_________________________ test_pytest_html[cli_args0] __________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_pytest_html0')>
cli_args = []

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_pytest_html(testdir, cli_args):
        report = testdir.tmpdir.join('report.html')
        testdir.makepyfile("""
            def test_1():
                assert 1 == 1

            def test_2():
                assert 1 == 2
        """)
        result = testdir.runpytest('--html=' + str(report), *cli_args)
>       result.assert_outcomes(passed=1, failed=1)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:217:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.USAGE_ERROR len(stdout.lines)=0 len(stderr.lines)=5 duration=0.04s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stderr call -----------------------------
ERROR: usage: setup.py [options] [file_or_dir] [file_or_dir] [...]
setup.py: error: unrecognized arguments: --html=/tmp/pytest-of-builduser/pytest-0/test_pytest_html0/report.html
  inifile: None
  rootdir: /tmp/pytest-of-builduser/pytest-0/test_pytest_html0

_________________________ test_pytest_html[cli_args1] __________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_pytest_html1')>
cli_args = ['--workers=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_pytest_html(testdir, cli_args):
        report = testdir.tmpdir.join('report.html')
        testdir.makepyfile("""
            def test_1():
                assert 1 == 1

            def test_2():
                assert 1 == 2
        """)
        result = testdir.runpytest('--html=' + str(report), *cli_args)
>       result.assert_outcomes(passed=1, failed=1)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:217:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.USAGE_ERROR len(stdout.lines)=0 len(stderr.lines)=5 duration=0.04s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stderr call -----------------------------
ERROR: usage: setup.py [options] [file_or_dir] [file_or_dir] [...]
setup.py: error: unrecognized arguments: --html=/tmp/pytest-of-builduser/pytest-0/test_pytest_html1/report.html
  inifile: None
  rootdir: /tmp/pytest-of-builduser/pytest-0/test_pytest_html1

_________________________ test_pytest_html[cli_args2] __________________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_pytest_html2')>
cli_args = ['--tests-per-worker=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_pytest_html(testdir, cli_args):
        report = testdir.tmpdir.join('report.html')
        testdir.makepyfile("""
            def test_1():
                assert 1 == 1

            def test_2():
                assert 1 == 2
        """)
        result = testdir.runpytest('--html=' + str(report), *cli_args)
>       result.assert_outcomes(passed=1, failed=1)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:217:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.USAGE_ERROR len(stdout.lines)=0 len(stderr.lines)=5 duration=0.04s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stderr call -----------------------------
ERROR: usage: setup.py [options] [file_or_dir] [file_or_dir] [...]
setup.py: error: unrecognized arguments: --html=/tmp/pytest-of-builduser/pytest-0/test_pytest_html2/report.html
  inifile: None
  rootdir: /tmp/pytest-of-builduser/pytest-0/test_pytest_html2

_______________________ test_collection_error[cli_args1] _______________________

testdir = <Testdir local('/tmp/pytest-of-builduser/pytest-0/test_collection_error1')>
cli_args = ['--workers=2']

    @pytest.mark.parametrize('cli_args', [
      [],
      ['--workers=2'],
      ['--tests-per-worker=2']
    ])
    def test_collection_error(testdir, cli_args):
        testdir.makepyfile(first_file_test="""
            def test_1():
                assert 1 == 1
        """, second_file_test="""
            raise Exception('Failed to load test file')
        """)
        result = testdir.runpytest(*cli_args)
>       result.assert_outcomes(error=1)

/build/python-pytest-parallel/src/pytest-parallel-0.0.10/tests/test_general.py:241:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <RunResult ret=ExitCode.INTERNAL_ERROR len(stdout.lines)=7 len(stderr.lines)=80 duration=0.14s>

>   ???
E   ValueError: Pytest terminal summary report not found

/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py:442: ValueError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_collection_error1
plugins: parallel-0.0.10
collected 1 item / 1 error

!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 924, in runpytest_inprocess
  File "/build/python-pytest/src/pytest-5.3.5/build/lib/_pytest/pytester.py", line 890, in inline_run
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 232, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 691, in pytest_sessionfinish
    self.config.hook.pytest_terminal_summary(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 337, in traced_hookexec
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 52, in from_call
    result = func()
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 335, in <lambda>
    outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 182, in _multicall
    next(gen)  # first yield
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 705, in pytest_terminal_summary
    self.summary_errors()
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 881, in summary_errors
    reports = self.getreports("error")
  File "/usr/lib/python3.8/site-packages/_pytest/terminal.py", line 777, in getreports
    for x in self.stats.get(name, []):
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 298, in serve_client
    send(msg)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function unpickle_exception at 0x7f67df4d14c0>: it's not the same object as tblib.pickling_support.unpickle_exception

---------------------------------------------------------------------------
========================= 9 failed, 22 passed in 3.37s =========================

os.cpu_count()

--workers auto uses os.cpu_count() which is in Python 3.4 or newer, whereas multiprocessing.cpu_count() is in Python 2.6 or newer.

Summary output to stdout mentioning each test case twice

The output of pytest shows, e.g.:

pytest-parallel: 4 workers (processes), 5 tests per worker (threads)
test/test_event_driven.py::test_foo FAILED
test/test_event_driven.py::test_foo FAILED
test/test_event_driven.py::test_bar PASSED
test/test_event_driven.py::test_bar PASSED
...

I would have expected instead:

pytest-parallel: 4 workers (processes), 5 tests per worker (threads)
test/test_event_driven.py::test_foo FAILED
test/test_event_driven.py::test_bar PASSED
...

I believe the test cases are only executed once, since the error output of a failing test case only appears once, and when I list the slowest x test durations, I do not see doublets there.

Environment:
platform linux -- Python 3.7.7, pytest-5.3.5, py-1.9.0, pluggy-0.13.1 -- /usr/local/bin/python
plugins: clarity-0.3.0a0, parallel-0.1.0

dynamically added value to addopts with pytest_configure is not reflecting in cmd line

Objective:

Adding -n [var_num] to addopts for my pytest.ini file wherein the var_num is dynamically taken accordingly from command line for pytest-xdist run.

python3.6, pytest==4.6.3, pytest-xdist==1.28.0

My project struct:

Python-package
    - pytest.ini
    - setup.py
    - src (py package)
         - conftest.py
         - config.py
         - tests
             - test_tc1.py
             - test_tc2.py

pytest.ini

[pytest]
addopts = -v -s -p no:warnings
filterwarnings =
    ignore::UserWarning

conftest.py

def pytest_configure(config):
    config.addinivalue_line("markers", "env(name): mark test to run only on named environment")
    count = len(config.getoption('--env'))
    print('count: : ', count)
    args = "-n" + str(count)
    print('args is: ', args)
    config.addinivalue_line("addopts", args)
    print(config.getini("addopts"))

output is

(venv) [root@localhost tests]$ pytest test_tc1.py --env ENV1 --env ENV2 --env ENV3
count: :  3
args is:  -n3
['-v', '-s', '-p', 'no:warnings', '-n3']
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- [ROOT_DIR]/venv/bin/python3.6
cachedir: .pytest_cache
rootdir: [ROOT_DIR],
inifile: pytest.ini
plugins: forked-1.0.2, rerunfailures-7.0, xdist-1.28.0
collecting ... [shortened data]
collected 3 items                                                                                                                                              

test_tc1.py::TestClass1::test_tc1[ENV1] 
[Shortened Data]
PASSED[Shortened Data]

test_tc1.py::TestClass1::test_tc1[ENV2] 
[Shortened Data]
PASSED[Shortened Data]

test_tc1.py::TestClass1::test_tc1[ENV3] 
[Shortened Data]
PASSED[Shortened Data]


================================================================== 3 passed in 10.49 seconds ===================================================================

Expected Output

Expectation was should be same as when I parse the -n flag to pytest.ini

pytest.ini

[pytest]
addopts = -v -s -p no:warnings -n3
filterwarnings =
    ignore::UserWarning

conftest.py

def pytest_configure(config):
    config.addinivalue_line("markers", "env(name): mark test to run only on named environment")
    print(config.getini("addopts"))

Result is shown as below

(venv) root@localhost tests]$ pytest test_tc1.py --env ENV1 --env ENV2 --env ENV3
['-v', '-s', '-p', 'no:warnings', '-n3']
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- [ROOT_DIR]/venv/bin/python3.6
cachedir: .pytest_cache
rootdir: [ROOT_DIR], inifile: pytest.ini
plugins: forked-1.0.2, rerunfailures-7.0, xdist-1.28.0
[gw0] linux Python 3.6.3 cwd: [TEST_DIR]/tests
[gw1] linux Python 3.6.3 cwd: [TEST_DIR]/tests
[gw2] linux Python 3.6.3 cwd: [TEST_DIR]/tests
[gw0] Python 3.6.3 (default, Jun 25 2019, 18:53:50)  -- [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
[gw1] Python 3.6.3 (default, Jun 25 2019, 18:53:50)  -- [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
[gw2] Python 3.6.3 (default, Jun 25 2019, 18:53:50)  -- [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
gw0 ok / gw1 ok / gw2 ok/[Shortened Data]
gw0 [3] / gw1 [3] / gw2 [3]
scheduling tests via LoadScheduling

test_tc1.py::TestClass1::test_tc1[ENV3] 
test_tc1.py::TestClass1::test_tc1[ENV2] 
test_tc1.py::TestClass1::test_tc1[ENV1] 
[gw0] PASSED  test_tc1.py::TestClass1::test_tc1[ENV1] 
[gw2] PASSED  test_tc1.py::TestClass1::test_tc1[ENV3] 
[gw1] PASSED  test_tc1.py::TestClass1::test_tc1[ENV2] 

=================================================================== 3 passed in 6.05 seconds ===================================================================

As you see even if both the pytest.ini addopts having the same list of data, the resulting behaviour is different. The first attempt is not even considering the flag -n. What is the reason for this ? How to get this fixed ?

EDIT

If I add both the pytest.ini and conftest.py files; the list data is

['-v', '-s', '-p', 'no:warnings', '-n3', '-n3']

file not found: --tests-per-worker 4

I used command line to execute is working.(ex. pytest --tests-per-worker 4)
But if i used pytest.main(['--tests-per-worker 4']) to execute will be fail.
Fail message is below (Enviroment:Windows 7)

plugins: xdist-1.26.1, parallel-0.0.9, metadata-1.8.0, json-report-1.0.3, forked-1.0.2
collecting ...
=================================================================================================== no tests ran in 0.00 seconds ===================================================================================================
ERROR: file not found: --tests-per-worker 4

Code
def test_Execute():
pytest.main(['--tests-per-worker 4'])

if name == 'main':
test_Execute()

`--junitxml` creates an empty report

Pytest Parallel skips if a test marked has a test case dependency.

Description-
Pytest Parallel skips if a test marked has another test as dependency.[using pytest-dependency plugin.].

`import pytest
import configparser
import time


@pytest.mark.dependency()
def test_script9():
    time.sleep(30)
    print("test9 executed")

@pytest.mark.order2
def test_script2():
   time.sleep(30)
   print("test2 executed")


@pytest.mark.dependency(depends=['test_script9'])
def test_script3():
   time.sleep(30)
   print("test3 executed")


@pytest.mark.order4
def test_script4():
   time.sleep(30)
   print("test4 executed")`

Fixture Scope: All scopes behave as if function scoped, even with 1 process and 1 thread

Versions:

Python 3.8.0, 3.7.5
pytest==6.0.1
pytest-parallel==0.1.0

Description

To clarify, I understand that session is not shared between processes or threads (which seems to be a common point of confusion when using either pytest-parallel or pytest-xdist), but what I'm seeing is that, even with --workers 1 --tests-per-worker 1 (and everything beyond), fixtures scoped to session, class, etc all get re-run for every single test.

For the simplest example:

conftest.py

import pytest
import uuid

@pytest.fixture("class")
def session_scoped_fixture():
    return uuid.uuid4()

test.py

import pytest

def test1(session_scoped_fixture):
    assert 1 == session_scoped_fixture

def test2(session_scoped_fixture):
    assert 1 == session_scoped_fixture

def test3(session_scoped_fixture):
    assert 1 == session_scoped_fixture

def test4(session_scoped_fixture):
    assert 1 == session_scoped_fixture

Expected results

Since the fixture returns a unique UUID when run, one would expect output like this:

$ pytest --workers 2 test.py # OR pytest --workers 1 --tests-per-worker 2
...
==================================== short test summary info ====================================
FAILED test.py::test1 - AssertionError: assert 1 == UUID('deadbeef-40f1-4042-a047-3f2bf7fc6ac1')
FAILED test.py::test2 - AssertionError: assert 1 == UUID('112294dc-7949-475c-9b32-874e1fa99164')
FAILED test.py::test3 - AssertionError: assert 1 == UUID('deadbeef-40f1-4042-a047-3f2bf7fc6ac1')
FAILED test.py::test4 - AssertionError: assert 1 == UUID('112294dc-7949-475c-9b32-874e1fa99164')

ie. there are two unique IDs, for two sessions, one per worker (or thread).

Actual results

However, whenever pytest-parallel is involved, even if set to --workers 1 --tests-per-worker 1 (which should mean no concurrency or parallelism at all), every single test function appears to effectively have its own session:

$ pytest --workers 2 test.py # OR pytest --workers 1 --tests-per-worker 2
...
==================================== short test summary info ====================================
FAILED test.py::test1 - AssertionError: assert 1 == UUID('deadbeef-40f1-4042-a047-3f2bf7fc6ac1')
FAILED test.py::test2 - AssertionError: assert 1 == UUID('112294dc-7949-475c-9b32-874e1fa99164')
FAILED test.py::test3 - AssertionError: assert 1 == UUID('2fac72e8-ab02-4f41-8403-232119f70bad')
FAILED test.py::test4 - AssertionError: assert 1 == UUID('35d7f4ac-4804-40a6-9880-6153db735de3')

ie. There are four unique UUIDs, not the expected two.

For what it's worth, I don't have this issue with pytest-xdist: When running with -n 2 for example, there are two unique UUIDs among the four tests.

Is this expected? It seems like a bug, and I can't find it documented, but perhaps I'm misunderstanding something about how this all works.

Python 2.7

pytest-parallel doesn't currently support Python 2.7

You can vote for this issue by adding a 👍 reaction:

fixture will be repeated with the case use pytest.mark.parametrize

`import pytest

@pytest.fixture(scope='module')
def a():
a = 'nihao'
print('初始化')
return a

@pytest.mark.parametrize('data',['a','b','c'])
def test_001(a,data):
print(a,data)`

this code ,the fixture will be used three times when I use the pytest-parallel

if i don't use pytest-parallel,only once.

TypeError with Python 3.6.8/pytest 5.1.3 on Ubuntu 18.04.3 LTS

Hi, all,

I am very happy to discover this wonderful plugin!

Call configuration: --workers auto

Unfortunately, soon enough I have got the following exception:

Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pytest_parallel/init.py", line 81, in run
run_test(self.session, item, None)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pytest_parallel/init.py", line 51, in run_test
item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/manager.py", line 86, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
return outcome.get_result()
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/_pytest/runner.py", line 72, in pytest_runtest_protocol
runtestprotocol(item, nextitem=nextitem)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/_pytest/runner.py", line 87, in runtestprotocol
reports.append(call_and_report(item, "call", log))
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/_pytest/runner.py", line 171, in call_and_report
hook.pytest_runtest_logreport(report=report)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/manager.py", line 86, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
return outcome.get_result()
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/home/user/.virtualenvs/apache-cs-extensions/lib/python3.6/site-packages/_pytest/main.py", line 412, in pytest_runtest_logreport
if maxfail and self.testsfTypeError: '>=' not supported between instances of 'SafeNumber' and 'int'ailed >= maxfail:
TypeError: '>=' not supported between instances of 'SafeNumber' and 'int'#

It seems that pytest-parallel defines a class SafeNumber which is then causes a pytest exception in the following method:


@hookimpl(tryfirst=True)
    def pytest_runtest_logreport(self, report):
        if report.failed and not hasattr(report, "wasxfail"):
            self.testsfailed += 1
            maxfail = self.config.getvalue("maxfail")
            if maxfail and self.testsfailed >= maxfail:
                self.shouldfail = "stopping after %d failures" % (self.testsfailed)

What I do not understand, how comes that maxfail which should be a config value becomes an instance of SafeNumber.

-x, --exitfirst not work...

Seems that "pytest-parallel" ignores "--exitfirst", isn't it?

pytest-parallel exit code always 0

Unsure if the issue is specific to me, added the info below to reproduce.

tst.py =

import pytest
def test_tst():
    assert False

etragas$ pytest tst.py
test session starts 
platform darwin -- Python 3.6.6, pytest-3.8.1, py-1.6.0, pluggy-0.7.1
rootdir: /Users/etragas/diseaseTools, inifile: pytest.ini
plugins: xdist-1.23.2, profiling-1.3.0, parallel-0.0.5, mock-1.10.0, forked-0.2, cov-2.6.0
collected 1 item

tst.py F                                                                                                                                                 [100%]

=== FAILURES ===

    def test_tst():
>       assert False
E       assert False

tst.py:3: AssertionError
1 failed in 0.08 seconds 

etragas$ echo $?
1

etragas$ pytest tst.py --workers auto
test session starts 
platform darwin -- Python 3.6.6, pytest-3.8.1, py-1.6.0, pluggy-0.7.1
rootdir: /Users/etragas/diseaseTools, inifile: pytest.ini
plugins: xdist-1.23.2, profiling-1.3.0, parallel-0.0.5, mock-1.10.0, forked-0.2, cov-2.6.0
collected 1 item
pytest-parallel: 12 workers (processes), 1 test per worker (thread)
F                                                                                                                                                        [100%]
=== FAILURES ===

    def test_tst():
>       assert False
E       assert False

tst.py:3: AssertionError
1 failed in 0.12 seconds 

HUAWEI_P10-796cab753f8233:diseaseTools etragas$ echo $?
0

Problem with "How to contribute"

What I do:

git clone [email protected]:browsertron/pytest-parallel.git
cd pytest-parallel
pipenv install --dev
pipenv run test - in this step I have next problem:

$ pipenv run test
Traceback (most recent call last):
  File "/usr/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==11.9.0', 'console_scripts', 'pipenv')()
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pipenv/cli.py", line 602, in run
    core.do_run(command=command, args=args, three=three, python=python)
  File "/usr/lib/python3/dist-packages/pipenv/core.py", line 2200, in do_run
    command = ' '.join(project.scripts[command])
  File "/usr/lib/python3/dist-packages/pipenv/project.py", line 374, in scripts
    scripts[k] = shlex.split(v, posix=True)
  File "/usr/lib/python3.7/shlex.py", line 305, in split
    return list(lex)
  File "/usr/lib/python3.7/shlex.py", line 295, in __next__
    token = self.get_token()
  File "/usr/lib/python3.7/shlex.py", line 105, in get_token
    raw = self.read_token()
  File "/usr/lib/python3.7/shlex.py", line 136, in read_token
    nextchar = self.instream.read(1)
AttributeError: 'list' object has no attribute 'read'

What should I do for fix it ?

$ uname -a

Linux isizov-HP-G6 5.0.0-36-generic #39-Ubuntu SMP Tue Nov 12 09:46:06 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

(pytest-parallel-RgK59dpO) isizov@isizov-HP-G6:~/GIT/pytest-parallel$ pipenv run pip freeze
astroid==2.3.3
attrs==19.3.0
bleach==3.1.0
certifi==2019.11.28
cffi==1.13.2
chardet==3.0.4
cryptography==2.8
docutils==0.15.2
entrypoints==0.3
filelock==3.0.12
flake8==3.7.9
idna==2.8
importlib-metadata==1.1.0
invoke==1.3.0
isort==4.3.21
jeepney==0.4.1
keyring==19.3.0
lazy-object-proxy==1.4.3
mccabe==0.6.1
more-itertools==8.0.0
packaging==19.2
pkg-resources==0.0.0
pkginfo==1.5.0.1
pluggy==0.13.1
py==1.8.0
pycodestyle==2.5.0
pycparser==2.19
pyflakes==2.1.1
Pygments==2.5.2
pylint==2.4.4
pyparsing==2.4.5
pytest==5.3.1
readme-renderer==24.0
requests==2.22.0
requests-toolbelt==0.9.1
SecretStorage==3.1.1
six==1.13.0
tblib==1.5.0
toml==0.10.0
tox==3.14.2
tox-pyenv==1.1.0
tox-travis==0.12
tqdm==4.40.0
twine==3.1.1
typed-ast==1.4.0
urllib3==1.25.7
virtualenv==16.7.8
wcwidth==0.1.7
webencodings==0.5.1
wrapt==1.11.2
zipp==0.6.0

AttributeError: 'NoneType' object has no attribute '_registry' with pytest 5.3.0

Hi there,

I run pytest with pytest-parallel (v 0.0.9) in a Gitlab CI-Pipeline.
Without changing my code the pipeline jobs stopped working. The only thing that change was the pytest version from 5.2.4 to 5.3.0.

Also the pipeline job doesn't seem to stop. It's stuck in the status running until I cancel it.

The failed command is:
- python setup.py install && pytest --workers 4
where setup.py install is our project specific setup script.

The hopefully relevant output:

============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.3.0, py-1.8.0, pluggy-0.13.0
rootdir: /builds/fsd/odx-ninja
plugins: parallel-0.0.9
collected 70 items
pytest-parallel: 4 workers (processes), 1 test per worker (thread)
....Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py", line 81, in run
    run_test(self.session, item, None)
  File "/usr/local/lib/python3.7/site-packages/pytest_parallel/__init__.py", line 51, in run_test
    item.ihook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/runner.py", line 82, in pytest_runtest_protocol
    item.ihook.pytest_runtest_logfinish(nodeid=item.nodeid, location=item.location)
  File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/terminal.py", line 468, in pytest_runtest_logfinish
    main_color, _ = _get_main_color(self.stats)
  File "/usr/local/lib/python3.7/site-packages/_pytest/terminal.py", line 1102, in _get_main_color
    for found_type in stats:
  File "<string>", line 2, in __iter__
  File "/usr/local/lib/python3.7/multiprocessing/managers.py", line 825, in _callmethod
    proxytype = self._manager._registry[token.typeid][-1]
AttributeError: 'NoneType' object has no attribute '_registry'

When the --workers 4 was removed the CI pipeline didn't fail.

To avoid this we now fixed the pytest version to 5.2.4.

I also created an issue at pytest:
pytest-dev/pytest#6254

I hope I could help.

Best regards

Does not work with allure-pytest

When I attempt to use pytest-parallel it prevents the allure report from being generated. for instance if I run:
pytest --tests-per-worker 4 --alluredir=reports/allure

no report is generated. If i take the tests per worker argument out the report works. Any idea why this would prevent the report from being saved?

Using pytest parallel with markers gives AttributeError: Can't pickle local object 'ArgumentParser.init.<locals>.identity'

When we use pytest parallel and run tests per test suite, it works like a charm.
But when we use it to send test cases which has be attributed with "markers" then we get an "Attribute Error"

py.test -sv --workers 2 -m ewlc com/tests/mock/e2e/ -k test --config device_mock_Cluster.yaml
===================================================================================================== test session starts =====================================================================================================
platform darwin -- Python 3.6.5, pytest-3.8.0, py-1.5.4, pluggy-0.7.1 -- /Users/gosurend/.virtualenvs/checkuniq/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.6.5', 'Platform': 'Darwin-16.7.0-x86_64-i386-64bit', 'Packages': {'pytest': '3.8.0', 'py': '1.5.4', 'pluggy': '0.7.1'}, 'Plugins': {'xdist': '1.23.0', 'parallel': '0.0.2', 'ordering': '0.5', 'mock': '0.10.1', 'metadata': '1.7.0', 'html': '1.14.2', 'forked': '0.2', 'dependency': '0.3.2', 'cov': '2.2.1', 'assume': '1.2', 'flaky': '3.3.0'}, 'ASSURANCE_VERSION': None}
rootdir: /Users/gosurend/Documents/Wireless/QA_Always_latest/qa-assurance-common/assurance_it, inifile: pytest.ini
plugins: xdist-1.23.0, parallel-0.0.2, ordering-0.5, mock-0.10.1, metadata-1.7.0, html-1.14.2, forked-0.2, dependency-0.3.2, cov-2.2.1, assume-1.2, flaky-3.3.0
collecting 594 items / 2 errors INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/main.py", line 214, in _main
INTERNALERROR> config.hook.pytest_collection(session=session)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/main.py", line 224, in pytest_collection
INTERNALERROR> return session.perform_collect()
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/main.py", line 432, in perform_collect
INTERNALERROR> session=self, config=self.config, items=items
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/mark/init.py", line 156, in pytest_collection_modifyitems
INTERNALERROR> deselect_by_mark(items, config)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/mark/init.py", line 150, in deselect_by_mark
INTERNALERROR> config.hook.pytest_deselected(items=deselected)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/Users/gosurend/.virtualenvs/checkuniq/lib/python3.6/site-packages/_pytest/terminal.py", line 363, in pytest_deselected
INTERNALERROR> self.stats.setdefault("deselected", []).extend(items)
INTERNALERROR> File "", line 2, in extend
INTERNALERROR> File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod
INTERNALERROR> conn.send((self._id, methodname, args, kwds))
INTERNALERROR> File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/connection.py", line 206, in send
INTERNALERROR> self._send_bytes(_ForkingPickler.dumps(obj))
INTERNALERROR> File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
INTERNALERROR> cls(buf, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.init..identity'

Any fixes to this problem will be much appreciated!

Using the command line to execute the script in windows, an error is reported

Test session starts (platform: win32, Python 3.8.2, pytest 5.4.1, pytest-sugar 0.9.4)
rootdir: D:\TestScriptDir\pyqt5_study
plugins: allure-pytest-2.8.13, forked-1.2.0, html-2.1.1, metadata-1.8.0, ordering-0.6, parallel-0.1.0, repeat-0.8.0, rerunfailures-9.0, sugar-0.9.4, xdist-1.32.0
collecting ... pytest-parallel: 1 worker (process), 2 tests per worker (threads)
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages_pytest\main.py", line 191, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages_pytest\main.py", line 247, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pluggy\hooks.py", line 286, in call
INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pluggy\manager.py", line 84, in
INTERNALERROR> self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pluggy\callers.py", line 187, in multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\site-packages\pytest_parallel_init.py", line 302, in pytest_runtestloop
INTERNALERROR> process.start()
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
INTERNALERROR> self._popen = self._Popen(self)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
INTERNALERROR> return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 326, in _Popen
INTERNALERROR> return Popen(process_obj)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in init
INTERNALERROR> reduction.dump(process_obj, to_child)
INTERNALERROR> File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump
INTERNALERROR> ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'pytest_addoption..label_type..a_label_type'

Results (0.26s):
Exception in thread Thread-1:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 107, in spawn_main
new_handle = reduction.duplicate(pipe_handle,
File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 79, in duplicate
return _winapi.DuplicateHandle(
OSError: [WinError 6] 句柄无效。
Fatal Python error: could not acquire lock for <_io.BufferedWriter name=''> at interpreter shutdown, possibly due to daemon threads
Python runtime state: finalizing (tstate=000001D863DD9070)

Thread 0x000015dc (most recent call first):
File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\threading.py", line 1202 in invoke_excepthook
File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\threading.py", line 934 in _bootstrap_inner
File "C:\Users\luozhongwen\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890 in _bootstrap

Current thread 0x00001eb4 (most recent call first):

Add environment variable with index of process/thread

pytest-xdist has PYTEST_XDIST_WORKER which we use to create a unique DB for each thread. Could pytest-parallel provide something similar?

Happy to work on this if there is interest

Not working with Pytest-Selenium

I'm quite unsure why pytest-parallel don't work with pytest-selenium (pip install pytest-selenium). I've heard Selenium is not thread safe, but according to documentation, pytest-parallel do some work to make it thread safe.

Tested with:
- Python 3.7, Debian 9, pytest-selenium, pytest, and pytest-parallel
- run with: python -m pytest --driver Firefox --driver-path ./geckodriver --tests-per-worker auto --workers auto

Mark parallelism for certain groups of tests

For instance, if I have a handful of tests that can't run in parallel, but the bulk of the test suite can, I'd want a way to mark that so that certain tests have to be run in sequence as normal, and the rest parallelized. I would want this split effective when running a single pytest command over the entire test suite.

Py.Test Can't find pytest-parallel when invoked programatically

Hi, I am trying to send parallel arguments to a programmatic invocation but pytest doesn't seem to find the arguments. But on the other hand, when I run py.test with a direct command line invocation including the same arguments, it will find and run parallel sucesfully.

ERROR: usage: invoke_pytest.py [options] [file_or_dir] [file_or_dir] [...]
invoke_pytest.py: error: unrecognized arguments: --workers 1 --tests-per-worker 4

This is my code:

import os
import sys
import pytest

sys.path.append(os.path.join(sys.path[0], 'tests', 'bdd'))
sys.path.append(os.path.join(sys.path[0], 'tests', 'bdd', 'steps'))

pytest_args = [
    "tests/bdd",
    '--rootdir=tests/bdd',
    "--workers 1",
    "--tests-per-worker 4",
    '-k login',
    "--splinter-webdriver=remote",
    "--splinter-make-screenshot-on-failure=true",
    '--splinter-remote-url=...',
    '--cache-clear', '-v',
]
result = pytest.main(pytest_args)
print(f"RESULT {result}")

Although it seems unrelated, I am also using py.test bdd and splinter for this test suite.

ci: coverage reporting

Would you like to have coverage reporting setup?
I could do it, using pytest-cov and codecov.

pytest-html does not pick up the test results when we run with pytest-parallel

We usually generate reports after every run for our tests using pytest-html.
But when we use pytest-parallel, we get 0 tests run in the report, but console correctly gives all details.
Can you please take a quick look and see if it is a problem with pytest-parallel?
I tried with xdist and that is able to collect the test correctly!

The sample command I use is
py.test --workers 2 -m "ewlc" -sv com/tests/mock/e2e/eWLC_Client_Issues/blacklist_dhcp_eWLC_Issues_test.py -k test --config device_mock_Cluster.yaml --html=local-regression.html

No test result in xml -> pytest --workers auto tests/app/ --junitxml=/app/test-reports/results.xml

The following commands executes the tests in parallel and also generates junitxml file but doesn't contain the test results.
pytest --workers auto tests/app/ --junitxml=/app/test-reports/results.xml

tests execution summary:
============================= test session starts ==============================
platform linux -- Python 3.8.0, pytest-5.3.2, py-1.8.1, pluggy-0.13.1
rootdir: /app
plugins: parallel-0.0.9, apiritif-0.9.5
collected 49 items
pytest-parallel: 4 workers (processes), 1 test per worker (thread)
.................................................
=============================== warnings summary ===============================
-------------- generated xml file: /app/test-reports/results.xml ---------------
======================== 49 passed, 1 warning in 17.43s ========================

results.xml content:
<?xml version="1.0" encoding="utf-8"?><testsuites><testsuite errors="0" failures="0" hostname="864542177d3e" name="pytest" skipped="0" tests="0" time="17.420" timestamp="2020-09-03T08:18:46.725913"></testsuite></testsuites>

Am I missing something here.

'--workers 2' raise error when working with pytest5.x

Traceback (most recent call last):
File "run_case.py", line 37, in
ret = pytest.main()
File "/project/venv/lib/python3.6/site-packages/_pytest/config/init.py", line 74, in main
return config.hook.pytest_cmdline_main(config=config)
File "/project/venv/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/project/venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/project/venv/lib/python3.6/site-packages/pluggy/manager.py", line 81, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/project/venv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
return outcome.get_result()
File "/project/venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/project/venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/project/venv/lib/python3.6/site-packages/_pytest/main.py", line 250, in pytest_cmdline_main
return wrap_session(config, _main)
File "/project/venv/lib/python3.6/site-packages/_pytest/main.py", line 243, in wrap_session
session=session, exitstatus=session.exitstatus
File "/project/venv/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/project/venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/project/venv/lib/python3.6/site-packages/pluggy/manager.py", line 81, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/project/venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
gen.send(outcome)
File "/project/venv/lib/python3.6/site-packages/_pytest/terminal.py", line 665, in pytest_sessionfinish
self.summary_stats()
File "/project/venv/lib/python3.6/site-packages/_pytest/terminal.py", line 863, in summary_stats
(line, color) = build_summary_stats_line(self.stats)
File "/project/venv/lib/python3.6/site-packages/_pytest/terminal.py", line 1014, in build_summary_stats_line
for found_type in stats:
File "", line 2, in __getitem__
File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
raise convert_to_error(kind, result)
KeyError: 0

How to specify running order (both running in parallel and running in turn) when using multi-threaded/multi-process

How to specify running order (both running in parallel and running in turn) when using multi-threaded/multi-process.

I want three test files to run in parallel, with each test file running the test function in turn. How do I design this?

AttributeError: Can't pickle local object when Running with Python 3.8

To reproduce, first have a Python 3.8 enviroment, then

pip install pytest pytest-parallel
touch test.py
pytest --workers 10 test.py

will give

platform darwin -- Python 3.8.0, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /Users/laike9m/Dev/Python/TestPython
plugins: parallel-0.0.9
collected 0 items
pytest-parallel: 10 workers (processes), 1 test per worker (thread)
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/_pytest/main.py", line 235, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pluggy/manager.py", line 83, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/site-packages/pytest_parallel/__init__.py", line 342, in pytest_runtestloop
INTERNALERROR>     process.start()
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/process.py", line 121, in start
INTERNALERROR>     self._popen = self._Popen(self)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
INTERNALERROR>     return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
INTERNALERROR>     return Popen(process_obj)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
INTERNALERROR>     super().__init__(process_obj)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
INTERNALERROR>     self._launch(process_obj)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
INTERNALERROR>     reduction.dump(process_obj, fp)
INTERNALERROR>   File "/Users/laike9m/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
INTERNALERROR>     ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'

AttributeError: Can't pickle local object on Windows 10

Python 3.8.3, pytest-parallel 0.1.0 enviroment on Windows 10
pytest --tests-per-worker 4

It works correctly with pytest-parallel 0.0.10
There isn't have ForkingPickler on Windows, as far as I know.

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\_pytest\main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\_pytest\main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pytest_parallel\__init__.py", line 302, in pytest_runtestloop
INTERNALERROR>     process.start()
INTERNALERROR>   File "C:\Users\a.bobrov\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\process.py", line 112, in start
INTERNALERROR>     self._popen = self._Popen(self)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 223, in _Popen
INTERNALERROR>     return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 322, in _Popen
INTERNALERROR>     return Popen(process_obj)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
INTERNALERROR>     reduction.dump(process_obj, to_child)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\reduction.py", line 60, in dump
INTERNALERROR>     ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'

--exitfirst pytest CLI option is being ignored

Given the following pytest.ini configuration file:

[pytest]
addopts = --exitfirst --log-cli-level=debug --workers auto

I see the following when running my test suite containing an unhandled exception.

# ...
tests/unit/test_foo.py::test_foo /home/appuser/app
PASSED                                                                                  
tests/unit/test_bar.py::test_bar /home/appuser/app
FAILED
tests/unit/test_baz.py::test_baz /home/appuser/app
PASSED
# ...
1 failed, 114 passed, 2 skipped, 9 warnings in 122.04s (0:02:02)

I've only used Python's Process API in very trivial ways, so the following may be wildly off the mark: I took a brief look at how the parallelization provided by this library works and I think you could achieve the standard pytest behavior by running a loop while the processes are in progress, checking their exitcodes and terminating all of the others (and closing the queue?) if any one returns a failure exitcode.

Is this something you'd consider doing? If so, I could try to put a PR together.

Either way, this library has been a tremendous productivity booster and reduced the run time of my test suite from ~11 minutes to ~2 minutes. So, thanks!

stdout mixed between tests in HTML report

So the output is mixed instead of be per test

kevlened / pytest-parallel Goto Github PK

pytest-parallel's Introduction

Maintainers needed

pytest-parallel

What?

Why?

Requirements

Installation

Options

Examples

Notice

License

pytest-parallel's People

Contributors

Stargazers

Watchers

Forkers

pytest-parallel's Issues

HINT:

Objective:

pytest.ini

conftest.py

Expected Output

pytest.ini

conftest.py

EDIT

Versions:

Description

Expected results

Actual results

Recommend Projects

Recommend Topics

Recommend Org