Dear Sylvain,

I forked python-pytest-steps and started a Travis CI build. However, the CI gets stuck at the after_success step, waiting for:

Enter passphrase for ci_tools/github_travis_rsa: 

(please have a look at the full logs on https://travis-ci.com/r4lv/python-pytest-steps)

Maybe the deployment step could be deactivated for forked repos by testing the GitHub URL in the .travis.yml?

For example

Cases: Orange, Strawberry (only for the parametrized example)

Test: test_fruit

Test steps:

peel (may fail)
mix (does not depend on peel)
eat (depends on mix)

We should provide the utility methods and explanation to create this kind of table when there are steps

It will be more readable

I have just started using your pytest extensions - they are wonderful, thank you!

I had a pytest-steps test with four steps, and each step saved the current value of a variable at the end of each step. The results_bag only had the value of the variable at the last step, and that value appeared in the row for the first step.

I fixed it by creating a per-step results bag fixture and using that in place of results_bag as the argument to my test. Is this the best way to get my desired behavior?

@pytest.fixture
@one_fixture_per_step
def step_bag(request):
    """pytest-harvest creates a results_bag per step so that it
    can calculate duration_ms per step, but results_bag seems
    to not be updated per step in my tests. This fixture
    explicitly grabs the current step's bag.

    Parameters
    ----------
    request: current running test object
        Fixture provided by pytest
    """
    return request.getfixturevalue("results_bag")

Some python 2 users might use

from __future__ import unicode_literals

see smarie/python-pytest-cases#62

This seems to happen when pytest-steps is used with pytest-cases and a LazyValue, LazyTuple, LazyTupleItem, fixture_ref, FixtureRefItem is used as parameter value.

Here is the warning: steps_parametrizer.py:94: RemovedInPytest4Warning: getfuncargvalue is deprecated, use getfixturevalue

Is it planned to be fixed?
Thanks

Python 3.7 is showing the following

pytest_steps/steps_generator.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Iterable as It

I guess that means it is broken on py38

In benchmarking situations we do not want the first run of a test to perform all the indirect imports otherwise its duration is overly big.

Therefore there is a need to run the test function "by hand" once. This is quite difficult right now when the function is decorated with @test_steps...

I found the pytest-steps and I'm trying to use it my code,
but some functions in my code is async function, so I'm curious about that this module is compatible with pytest.mark.asyncio or other async functions!

I'll wait your comment and have a nice day :)

As can seen from the image, not sure with which plugin it is not coming along, but the step names are in wrong order

• py.typed
• setup.cfg universal_wheel
• setup.py remove six dependency
• setup.py zip_safe=False
• setup.py remove tests folder from package.

ValueError: <function TestClass.test_xxx> uses no argument '________step_name_'

It seems that test functions located inside classes are not supported yet.

import pytest
from pytest_steps import test_steps

@test_steps('a', 'b')
@pytest.mark.parametrize('dummy_param', ['a', 'b'], ids="p={}".format)
def test_step_id_conflicts(dummy_param):
    yield 'a'
    yield 'b'

yields:

StepYieldError: Error collecting results from step 'b': received 'a' from the `yield` statement, which is different from the current step or step name. Please either use `yield`, `yield 'b'` or wrap your step with `with optional_step(...) as my_step:` and use `yield my_step`

The issue happens because we generate a unique id for each combination of parameters except step. This unique id is based on the string id right now, where a conflict happens because the string id of the step parameter happends in the string id of the other.

At line 86 of steps_pytest_harvest_utils.py, the columns have a single level index on the left and a two level index on the right. This is causing a pandas deprecation warning.

Test case: insert the following into tests/test_steps_harvest.py at line 64 and run the library test suite.

import warnings
warnings.warn("error")

You could perhaps fix the warning with the flatten_multilevel_columns function, but the column name change might affect existing tests.

Maybe this is not so bad: maybe there is a way to tell pytest-xdist that the group of steps have to be executed in the same distributed process ?

Hello, is it possible somehow when using generator steps to mark a specific step, for example, with the @pytest.mark.xfail, skip or other marks? Thank you in advance.

from pytest_steps import test_steps
from seleniumbase import BaseCase

class MMMM(BaseCase):

@test_steps('step_a', 'step_b', 'step_c')
def test_suite(self):
    # Step A
    print("step a")
    assert not False  # replace with your logic
    intermediate_a = 'hello'
    yield

    # Step B
    print("step b")
    assert not False  # replace with your logic
    yield

    # Step C
    print("step c")
    new_text = intermediate_a + " ... augmented"
    print(new_text)
    assert len(new_text) == 56
    yield

Hey, first of all - big fan of your plugin.
I'm trying to use it on top of a parametrized test, however, all combinations use the common setup which i obviously try to move to the pytest fixture, to be performed just once. However, this way i can't mark those actions as a step, so in the final report nobody sees they were happening.

what i'd like to achieve is to end up with the following report:
(given test_e2e with params ['param1', 'param2'] and steps ['step1', 'step2']and a fixture fixture1:

collected 5 items                                                                            

test.py::test_e2e[fixture1]     PASSED                                           [ 20%]
test.py::test_e2e[step1-param1] PASSED                                           [ 40%]
test.py::test_e2e[step1-param2] PASSED                                           [ 60%]
test.py::test_e2e[step2-param1] PASSED                                           [ 80%]
test.py::test_e2e[step2-param2] PASSED                                           [100%]

=============================== 4 passed in 0.02 seconds ===============================

do you think this is achievable in any way? (i don't insist on using the fixtures, however i don't want to generate that step for every parameter)

From reading the code I realize that there is nothing preventing pytest to execute the fixture teardown after the first step. So even if you decorate the fixture with @cross_steps_fixture, the value will be reused but it might be disfunctional.

Teardown hook should therefore be executed after all steps

Challenges:

• there are several teardown hooks for fixtures as of today. We have to make sure that we capture all of them and replace them with dummy ones - and finally call them after the last step
• we have to know the name of the last step also...

Contrary to the legacy mode, with the new mode adding the 'request' argument in a test function seems to fail.

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

First just run pytest:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-steps-1.8.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-steps-1.8.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/tkloczko/rpmbuild/BUILD/python-pytest-steps-1.8.0, configfile: setup.cfg, testpaths: pytest_steps/tests/
plugins: steps-1.8.0
collected 0 items / 1 error

================================================================================== ERRORS ==================================================================================
______________________________________________________________________ ERROR collecting test session _______________________________________________________________________
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1014: in _gcd_import
    ???
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:961: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:219: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1014: in _gcd_import
    ???
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:973: in _find_and_load_unlocked
    ???
E   ModuleNotFoundError: No module named 'pytest_steps.tests'
========================================================================= short test summary info ==========================================================================
ERROR  - ModuleNotFoundError: No module named 'pytest_steps.tests'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================= 1 error in 0.22s =============================================================================

Because pytest_steps.tests is not installed I've been trying to preload it using --pyargs:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-steps-1.8.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-steps-1.8.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --pyargs pytest_steps.tests
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/tkloczko/rpmbuild/BUILD/python-pytest-steps-1.8.0, configfile: setup.cfg
plugins: steps-1.8.0
collected 0 items

========================================================================== no tests ran in 0.02s ===========================================================================
ERROR: module or package not found: pytest_steps.tests (missing __init__.py?)

In older versions of pytest (2.x) the @pytest.fixture decorator does not have the name= argument required by current code.

this happened today : log

pytest_steps/tests/test_all.py::test_run_all_tests[test_docs_example_with_harvest.py] Segmentation fault (core dumped)

From the various configurations I tried, my best guess is that the meta-testing plugin (pytester) is causing the segfault when there are too many tests run especially inside "big" conda environments with pandas, etc.

Workaround: try to put as many tests as possible not in meta-test.

i have some code

# some inputs
@pytest.mark.parametrize(
    "real_number, precision, expected_result",
    [
        (123.123, 2, "123.12"),
        ("xasd123b.x", 2, "123.123"),
        ("x.x", 2, "123.123"),
    ]
)
# some steps from your package (great btw, i love it)
@test_steps(
    "correct number of decimals ?",
    "is type str ?",
    "result == expected ?"
)
# some params for my custom function
def test_fixed_set_precision_str(
    real_number: float | str,
    precision: int,
    expected_result: float,
):
    try:
        # we know that this raises some errors 
        # when the @real_number its not valid
        result = fixed_set_precision_str(real_number, precision)
        print(result)

    except TypeError as error:
        # here its the problem with pytest-steps
        # cuz here its not yield and pytest raises error
        # because it receives StopIteration
        print(error)

    except ValueError as error:
        # same thing
        # here its the problem with pytest-steps
        # cuz here its not yield and pytest raises error
        # because it receives StopIteration
        print(error)
    else:
        _decimals = get_total_decimals(result)
        
        # step 1
        # correct number of decimals ?
        assert _decimals == precision
        yield

        # step 2
        # is type str ?
        assert isinstance(result, str)
        yield

        # step 3
        # result == expected ?
        assert result == expected_result
        yield

in pytest output i do get this error:

ytest_steps.steps_generator.StepExecutionError: Error executing step 'correct number of decimals ?': could not reach the next `yield` statement (received
 `StopIteration`). This may be caused by use of a `return` statement instead of a `yield`, or by a missing `yield`

i understand why, but i dont know how to fix this, because steps is always expecting a generator.

thanks in advance.

In generator mode, the tests decorated with @test_steps are not in the same order as usual ; depending n the platform they appear at the beginning of the module tests, or at the end.

Trying out the example from documentation python-pytest-steps/, a hint is given to add the step name to yield.
The code on the documentation shown below needs to be complemented with an extra yield (line 13) in the following image.

By default pivot_steps_on_df thinks that all fixtures are cross-steps. It should only rely on explicitly cross-step fixtures for this.

pytest-harvest >= 1.1 offers default fixtures session_results_df and module_results_df. It makes sense to also offer pivoted versions in case when steps are present.

It would seem cool/intuitive to be able to breakdown a test function into steps by just using yield in the test function, thus defining the end of each step.

Same as in pyfields, pytest-cases, makefun, genbadge, etc.

Both decorators are only useful for function-scope fixtures

As of today

• In generator mode all fixtures are "cross-steps", however the fixture function is still called several times, once for each step. This is useless, because the corresponding values are just lost.
• In parametrizer mode all fixtures are "one per step", and there is no way to declare a fixture as cross-steps.
• When pytest-harvest is used, the pivot should take into account if a fixture is in one mode or the other. As of today it does not have a clue.