Add end to end tests + benchmark across browsers about zk-email-verify HOT 1 CLOSED

rmccrear, Re: "many browsers" testing i.e. https://www.lambdatest.com/online-browser-testing ?

From what I understand about lambdatest, is that it requires a deployed site,
then you give lambdatest your deployed url and the tests, and it runs the tests against the deployment.

You could set up render.com to deploy on each commit to main. Then have lambdatest run against your zkemaill.onrender.com (deployed as a static website).

So the workflow could look like this:

1. pre-commit: npm run tests:e2e --> get your local results for firefox, chromium webkit printed to command line (and to the junit.xml files)
2. on PR...run the tests in three browsers on circleci to record results for firefox, chromium, webkit (but you might need a paid account to run cpu intensive tests)
3. On merge to main, automatically deploy to render.com, then run lambdatest or browserstack to run on the environments you choose. (Probably need a paid account.)

But, if firefox, chromium, webkit are enough for you, then you won't need lambdatest/browswerstack, and could get away with just local/circleci. In that case you would just do 1) and 2) and skip 3).

There are a few problems with running perf tests on the cloud, though. One is that you won't have complete control over the instances. You won't know what chip they are running, you are sharing the machine which the VM runs with others, the provider might allow you to "burst" which would change the results. So, pass/fail tests would work fine, but I'm not so sure perf testing on these platforms would be accurate. (I can't find any services that specifically advertise perf testing for web apps.)

I tried to setup the tests on LambdaTest. They ran up until the zkp started running, then get cut off with signal=SIGSEGV.
I actually had a call with a demo sales rep, and he said that something long running like that probably wouldn't run on their servers. You can contact sales about that, if you want to pursue it further. Sorry about that! But, here is a PR with the LambdaTest, if you want to try it yourself.

The PR is here: #37

You just set the env in your bash profile before running yarn test:

export LT_USERNAME="username"
export LT_ACCESS_KEY="lksajdflkjadsfyourkey"

cd e2e-lambdatest
yarn
yarn test

Then go to the LambdaTest console to take a look at the screen recording.

Done with #36

from zk-email-verify.