marketsquare / assertionengine Goto Github PK

Easy way to perform assertions on Robot Framework assertions in libraries

License: Apache License 2.0

Python 85.38% RobotFramework 14.62%

assertionengine's Introduction

Assertion Engine

Generic way to create meaningful and easy to use assertions for the Robot Framework libraries. This tools is spin off from Browser library project, where the Assertion Engine was developed as part of the of library.

Supported Assertions

Currently supported assertion operators are:

Operator	Alternative Operators	Description	Validate Equivalent
==	equal, equals, should be	Checks if returned value is equal to expected value.	value == expected
!=	inequal, should not be	Checks if returned value is not equal to expected value.	value != expected
>	greater than	Checks if returned value is greater than expected value.	value > expected
>=		Checks if returned value is greater than or equal to expected value.	value >= expected
<	less than	Checks if returned value is less than expected value.	value < expected
<=		Checks if returned value is less than or equal to expected value.	value <= expected
*=	contains	Checks if returned value contains expected value as substring.	expected in value
	not contains	Checks if returned value does not contain expected value as substring.	expected not in value
^=	should start with, starts	Checks if returned value starts with expected value.	re.search(f"^{expected}", value)
$=	should end with, ends	Checks if returned value ends with expected value.	re.search(f"{expected}$", value)
matches		Checks if given RegEx matches minimum once in returned value.	re.search(expected, value)
validate		Checks if given Python expression evaluates to True.
evaluate	then	When using this operator, the keyword does return the evaluated Python expression.

Supported formatters:

Formatter	Description
normalize spaces	Substitutes multiple spaces to single space from the value
strip	Removes spaces from the beginning and end of the value
apply to expected	Applies rules also for the expected value
case insensitive	Converts value to lower case

Usage

When library developers wants to do an assertion inline with the keyword call, then AssertionEngine provides automatic validation within single keyword call. Keyword method should get value, example from page, database or from anything which the library interacts and then use verify_assertion method from AssertionEngine to perform the validation. The verify_assertion methods needs three things to perform the assertion: value from the system, assertion_operator how the validation is performed and assertion_expected which represent the expected value. It is also possible to provide custom error message and prefix the default error message.

Example library can contain keyword:

def keyword(
    arg_to_get_value: str,
    assertion_operator: Optional[AssertionOperator] = None,
    assertion_expected: Any = None,
    message: str = None,
):
    value = method_to_get_value(arg_to_get_value)
    return verify_assertion(
        value,
        assertion_operator,
        assertion_expected,
        "Prefix message",
        message,
    )

AssertionEngine provides an interface to define scope for the formatters, but because scoping is a library specific implementation, it is up to the library to decide how scoping is actually implemented. AssertionEngine Formatter class is an ABC which provides get_formatter and set_formatter interface methods for library developers. The AssertionEngine atest libraries has examples how interface can be implemented in practice.

assertionengine's People

Contributors

Stargazers

Watchers

Forkers

aaltat mkorpela xylix robinmatz mhhenriksen

assertionengine's Issues

assertionEngine can't be used with robotframework 4.0

When upgrading robotframework 3.2.2 to 4.0 a message is displayed preventing the upgrade.

"robotframework-assertion-engine 0.0.4 requires robotframework(4.0.0,>=3.2.2, but you have robotframework 4.0 which is incompatible)"

I guess pip is not liking 4.0.0 instead of 4.0

Issue from Browser: Assertion should fail if expected value is given without operator

"Would definitely be better if there was a failure if assertion_expected is not None and assertion_operator is None.

Example in the keyword documentation would obviously help with correct usage as well."

MarketSquare/robotframework-browser#1142

Small request: "equals"

Would it be possible to add 'equals' to the list of possible operators?

${var} equals 5

feels more natural than
${var} equal 5

list_verify_assertion() does return single value in case of len(list) == 1

When the list, put into list_verify_assertion has only one element it is converted to a single element.
When the list is empty, the value is converted to None

This is maybe a bit too much magic happening here...

assertionengine.assertion_engine.map_list

def map_list(selected: List):
    if not selected or len(selected) == 0:
        return None
    elif len(selected) == 1:
        return selected[0]
    else:
        return selected

Compatibility Robot Framework 6.0 release candidate

When trying to install robotframework-browser with the latest release candidate of robotframework (6.0.rc1), pip gives following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. robotframework-assertion-engine 0.5.1 requires robotframework<6.0.0,>=4.1.3, but you have robotframework 6.0rc1 which is incompatible.

When trying to install the latest robotframework-browser with robot framework 5.1b2, the issue does not occur, so it looks like the dependencies of robotframework-assertion-engine needs to be updated to also allow robot framework 6.x versions.

Extend Get Text with white spaces support

Regarding issue MarketSquare/robotframework-browser#1325.

We may consider to add a bit similar argument for Get Text like we have in Should Contain - collapse_spaces. It could process string before assertions by replacing all whitespace characters with space character. It will really make our work easier 😃

from BuiltIn.py:

If collapse_spaces is given a true value (seeBoolean arguments) and both arguments are strings, the comparison is done with all white spaces replaced by a single space character.

Taking the opportunity we may consider adding other useful operations like strip_spaces and ignore_case.

Drop Python 3.7 support

Drop Python 3.7 to allow easier update of dependencies and fixing security problems.

Support PLC 4.0.0

Add case-insensitive assertion

Recreated issue from Browser#2951.

Currently there's no direct way of doing a case-insensitive assertion and you need to do something like Get Text .my-element validate value.lower() == "some text".

Instead of "having" to use validate there could be a case-insensitive assertion something like

Get Text    .my-element   should be case-insensitive   some text    # option 1
Get Text    .my-element    ==!    some text    # option 2

There's the formatters for assertion, so this could probably be implemented as a formatter instead of a separate assertion as well.

Drop RF 4.0 support and add RF 5.0 support

Bug: ``value`` and ``expected`` are set to ``True`` if Assertion Formatter ``apply to expected``

How to reproduce

In atests/asserion_equal.robot, create the following test case

Test With Assertion Formatter Apply To Expected
    Set Assertion Formatters    
    ...    {"Is Equal": ["normalize spaces", "apply to expected"]}
    Is Equal    A${SPACE*2}B    equal    A${SPACE*4}C