Support marker keyword arguments in marker expression (test selection via `-m`) about pytest HOT 4 CLOSED

If we are to implement this, I propose we keep it simple, as it would be possible to also write my_device_under_test(with_serial=1234) or my_device_under_test(with_serial=5678).

from pytest.

I can see the appeal of this, and I agree in theory it makes sense to keep things close to a Python-like syntax.

However, at the same time, I'm also worried this might be a bottomless pit. Where do we stop in terms of types of arguments we support? Allowing my_device_under_test(with_serial=1234) or use_case("ABC-1234") seems obvious (and would indeed cover things I've seen various companies implement custom arguments/filtering via plugin hooks for).

However, what about, say, enums? Surely there's an argument to be made for enums, as in size(Size.end2end). But then where is Size coming from? I'm playing devil's advocate here, but if we allow enums, why wouldn't we also allow, I don't know, custom dataclass instances (in the sense of device(Multimeter(...)))? Etc. etc., and then at some point we're back to essentially reimplementing the eval() solution we had at some point.

Maybe though, we could limit this to int / str / bool / None or somesuch, or perhaps what ast.literal_eval supports. But still, while I can absolutely see the need and usefulness, it seems very tempting to take this too far over time.

from pytest.

Running into this thought myself, I was wondering if we could instead consider a bit of a tangent to avoid pitfalls mentioned by @The-Compiler. How about introducing a new interface called pytest.mark.annotated(*args: str)?

It would look like:

@pytest.mark.annotated("annotation1", "annotation2", ...)
def test_my_function(...):
    pass

The selection can be with a slightly specialized syntax:

pytest -m [annotation1]

which can be combined with other non-annotated markers if needed:

pytest -m "awesome and [annotation1] and [annotation2]"

This is surely limiting, but maybe that's not a bad thing. For my use case, this would help with adding bug number/jira ticket annotations to test cases.

from pytest.

The feature request makes sense to me, I think it would be great to allow matching on mark arguments (there are two kinds -- positional and keyword arguments), and something like the function call syntax seems natural for this.

For the match expression grammar, it shouldn't pose a big challenge to add. We would need to decide whether the argument values (the 1234 in your example) can be just idents (simple), or more complex, e.g. to support my_device_under_test(with_serial=1234 or 5678) etc., which would be more complex.

The bigger implementation issue would be the evaluation of the expression. Currently, to optimize the evaluation, we compile the expression to a python AST, then eval the AST with an environment ("locals") where every ident returns either True (if matches) or False. This will need to be extended to support the function-call syntax. As a quick thought, we could replace the locals hack with injecting some $match symbol into the eval environment which has the form matcher(name, *args: str, **kwargs: str) (assuming we only allow idents).

from pytest.