nitrokey / nitrokey-websmartcard-tests Goto Github PK

When running the tests, they end up hanging at the second test. The tests don't even stop after detaching the usbip device.

Most of the times, test execution freezes at different test cases, e. g. test_login_wrong_attempt_counter or test_ping.

Environment:

• Pipfile modified to use Python 3.9
• firmware 9343983 with DEVELOP=1

===================================== 64 failed, 5 passed in 3.39s =====================================

Environment:

• tests 871460
• firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
• REAL_HARDWARE=1 TRANSPORT=U2F

report

When running the FIDO2 test suite over NFC, the run fails after approx. 2/3 of the tests – e. g. at test_initialize or test_restore – with a Transaction failed error. I also encountered this error in my previous tests occasionally without being able to narrow it down.

Environment:

• tests 871460
• firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
• REAL_HARDWARE=1 TRANSPORT=FIDO2

report

Follow-up from #2 – most issues seem to be fixed, except:

FAILED test_comm.py::test_logout - assert True == False
FAILED test_comm.py::test_factory_reset - assert 253 == 240

Environment:

• tests 90ad5e6
• firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
• REAL_HARDWARE=1 TRANSPORT=U2F

report

64 failed, 5 passed in 5.08s 

Environment:

• Pipfile modified to use Python 3.9 and patched pynitrokey for NFC support
• firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
• REAL_HARDWARE=1 TRANSPORT=FIDO2

Log: https://cloud.nitrokey.com/s/rkNjNb35cLsdKr9

============================================================================================= short test summary info =============================================================================================
FAILED test_comm.py::test_generate_from_data - assert False == True
FAILED test_comm.py::test_initialize - assert False == True
FAILED test_comm.py::test_restore[test_input0] - assert False == True
FAILED test_comm.py::test_restore[test_input1] - assert False == True
FAILED test_comm.py::test_restore[test_input2] - assert False == True
FAILED test_comm.py::test_logout - assert True == False
FAILED test_comm.py::test_factory_reset - assert False == True
========================================================================================== 7 failed, 62 passed in 20.34s ==========================================================================================

Environment:

• Pipfile modified to use Python 3.9
• firmware 9343983 with DEVELOP=1
• REAL_HARDWARE=1 TRANSPORT=FIDO2

Log: https://cloud.nitrokey.com/s/nyip4TwAesAwaff

Edit the same as local archives:
webcrypt-report-fido2-usb.zip
webcrypt-report-u2f-usb.zip

======================================= short test summary info ========================================
FAILED test_comm.py::test_decrypt[True] - assert False == True
==================================== 1 failed, 68 passed in 22.13s =====================================

• tests 4871460
• firmware 7de4dc0 with DEVELOP=1
• configurations
  • U2F over USB
  • FIDO2 over NFC

reports.zip