Giter VIP home page Giter VIP logo

nitrokey-websmartcard-tests's Introduction

Nitrokey Webcrypt Tests

Setting up

Tests

  1. Python 3.10 with pipenv

To install Python dependencies for this project only it suffices to call:

pipenv install

Building simulation

Please refer to the main project readme for details:

Running tests

Simulation

It is possible to test the implementation using simulation without the actual hardware. The process and tests communicate via UDP.

Running

Please refer to the main project readme for details regarding how to run the UDP simulation:

Running actual tests:

pipenv run pytest test_comm.py

Same, but write live logs to output:

pipenv run pytest test_comm.py  -svx --log-cli-level=DEBUG

Hardware

Similarly to the simulation case, but with --hardware switch and additional environment variables:

  • env REAL_HARDWARE=1 TRANSPORT=FIDO2 for FIDO2 transport
  • env REAL_HARDWARE=1 TRANSPORT=U2F for U2F transport

E.g. for the U2F transport:

env REAL_HARDWARE=1 TRANSPORT=U2F pipenv run pytest --hardware test_comm.py

CI

For the CI use a Makefile is prepared, which automatically setups and runs both the simulation and tests. The UDP simulation has to be executed before that. Execution:

make

nitrokey-websmartcard-tests's People

Contributors

robin-nitrokey avatar szszszsz avatar

Stargazers

avatar

Watchers

avatar avatar avatar

Forkers

robin-nitrokey bellyfat

nitrokey-websmartcard-tests's Issues

USB, FIDO2: Tests freeze

Most of the times, test execution freezes at different test cases, e. g. test_login_wrong_attempt_counter or test_ping.

Environment:

  • Pipfile modified to use Python 3.9
  • firmware 9343983 with DEVELOP=1

Test failures 

============================================================================================= short test summary info =============================================================================================
FAILED test_comm.py::test_generate_from_data - assert False == True
FAILED test_comm.py::test_initialize - assert False == True
FAILED test_comm.py::test_restore[test_input0] - assert False == True
FAILED test_comm.py::test_restore[test_input1] - assert False == True
FAILED test_comm.py::test_restore[test_input2] - assert False == True
FAILED test_comm.py::test_logout - assert True == False
FAILED test_comm.py::test_factory_reset - assert False == True
========================================================================================== 7 failed, 62 passed in 20.34s ==========================================================================================

Environment:

  • Pipfile modified to use Python 3.9
  • firmware 9343983 with DEVELOP=1
  • REAL_HARDWARE=1 TRANSPORT=FIDO2

Log: https://cloud.nitrokey.com/s/nyip4TwAesAwaff

Edit the same as local archives:
webcrypt-report-fido2-usb.zip
webcrypt-report-u2f-usb.zip

USB, U2F: test_logout and test_factory_reset fail

Follow-up from #2 – most issues seem to be fixed, except:

FAILED test_comm.py::test_logout - assert True == False
FAILED test_comm.py::test_factory_reset - assert 253 == 240

Environment:

  • tests 90ad5e6
  • firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
  • REAL_HARDWARE=1 TRANSPORT=U2F

report

NFC, FIDO2: Transaction failed

When running the FIDO2 test suite over NFC, the run fails after approx. 2/3 of the tests – e. g. at test_initialize or test_restore – with a Transaction failed error. I also encountered this error in my previous tests occasionally without being able to narrow it down.

Environment:

  • tests 871460
  • firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
  • REAL_HARDWARE=1 TRANSPORT=FIDO2

report

test_decrypt[True] fails 

======================================= short test summary info ========================================
FAILED test_comm.py::test_decrypt[True] - assert False == True
==================================== 1 failed, 68 passed in 22.13s =====================================
  • tests 4871460
  • firmware 7de4dc0 with DEVELOP=1
  • configurations
    • U2F over USB
    • FIDO2 over NFC

reports.zip

NFC, U2F: Not enough data to read 

===================================== 64 failed, 5 passed in 3.39s =====================================

Environment:

  • tests 871460
  • firmware 9343983 with DEVELOP=1 and updated to use fido-authenticator v0.1.1
  • REAL_HARDWARE=1 TRANSPORT=U2F

report

Tests hang

When running the tests, they end up hanging at the second test. The tests don't even stop after detaching the usbip device.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.