Giter VIP home page Giter VIP logo

Comments (3)

glatteis avatar glatteis commented on May 31, 2024 1

A good way to do this is to write a new scheduler for your tests. Here's some example code (feel free to do it differently)

from xdist.remote import Producer
from xdist.scheduler.loadscope import LoadScopeScheduling

class YourScheduling(LoadScopeScheduling):
    """Implements a scheduler based on the LoadScopeScheduling. If a test using
    pytest-steps is executed, it groups the nodeids such that the tests
    depending on each other are executed on the same node. If not, it just
    defaults to executing tests on any node.
    """

    TESTS_USING_STEPS = ["test_using_steps"]

    def _split_scope(self, nodeid):
        """Determine the scope (grouping) of a nodeid.
        The nodeids for the model tests look like:
            path/to/your/test.py::test_using_steps[function_name-step]
        This function will group tests with the scope determined by splitting
        at 'model' plus the decimal number following.
         In the above example, scopes will be::
            path/to/your/test.py::test_using_steps[function_name
        """

        if any([x in nodeid for x in self.TESTS_USING_STEPS]):
            split = nodeid.split("-")
            return split[0]
        # Default: Each test is its own scope => Each test can run on any node
        return nodeid

Then use this scheduler by implementing this hook in conftest.py:

@pytest.hookimpl(trylast=True)
def pytest_xdist_make_scheduler(config, log):
    dist = config.getoption("dist")
    # Replace load (default) scheduler with YourScheduling which is a load
    # scheduler that also works for stepped tests
    schedulers = {
        "each": xdist.scheduler.EachScheduling,
        "load": YourScheduling,
        "loadscope": xdist.scheduler.LoadScopeScheduling,
        "loadfile": xdist.scheduler.LoadFileScheduling,
        "loadgroup": xdist.scheduler.LoadGroupScheduling,
    }
    return schedulers[dist](config, log)

from python-pytest-steps.

aaronsmith1234 avatar aaronsmith1234 commented on May 31, 2024

FYI @smarie , love the project, but I just ran into this problem today. I use pytest-xdist in CI to speed things up a bit, but in my case this actually broke my tests, which locally, not using pytest-xdist, worked fine.

One thing that worked for my use case was adding the flag --dist=loadfile to my pytest call. This distributes the tests by file, and for my use case it solved the problem.

from python-pytest-steps.

smarie avatar smarie commented on May 31, 2024

That's so nice of you @aaronsmith1234 to share this workaround !

It is persons like you who make open source so great :) I very much appreciate.

I'll leave the ticket open but rename it accordingly.

from python-pytest-steps.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.