treosh / exthouse Goto Github PK

Currently we track additional CPU work with kinda Long Tasks API and intention to do that for Firefox is having this API implemented.

In case we can relay on js-self-profiling API, we have second option to have Firefox support sooner (depends on which API implemented sooner).

js-self-profiling under implementation in Chrome, hence we should just enable it via puppeteer.

Motivation:

Honey extension requires authentication, but even without login, it affects perf.

It will be really interesting to know the results after authentication, compare how FID and other values changes between these 2 cases.

I tried to evaluate the final version of Block Origin.

1. Download extension using https://crxextractor.com/ (Because on https://chrome-extension-downloader.com/, when I input the URL of Block Origin. (https://chrome.google.com/webstore/detail/ublock-origin/cjpalhdlnbpafiamejdnhcphjbkeiagm?hl=en), It shows the. error: Empty Extension!)

2). Copy path to the ublockOrigin.crx and pass to cli: exthouse MY_EXTENTION.crx --runs=3
It shows error:

Unexpected crx format version number.
Run:
exthouse --help

Do you have any suggestions for me?
Note that, when I used the extension in your folder. (https://github.com/treosh/exthouse/tree/master/extensions), it works ok.
.

Hello,

First of all thanks for the work you've done ! it looks very promising !
I had some question about how you guys calculate the new score ? I imagine there's some sort of comparaison between the score with and without the extension ?

I wonder how the url you pass in as test url can impact the new score !
Indeed I tried it with a plugin we've created that display a bar when you arrive on a partner website.

If I go on a website that is not a partner it does nothing.

What I expected was to have a difference between the partner and non partner site as there is more javascript executed on partners' sites.

Here is the scrore I had by running the exthouse command with --runs=3

• Non partner => 98 (pretty good)

• Partner 1 => 53

• Partner 2 => 98

• Partner 3 => 70

The script executed should be the same between the three partners !

Do you guys have any idea how this can happen ?

Thanks again for the great work !

The purpose of the project improving speed on a web. All results we were gathering related only to Chromium mostly. We’d like to extend results and see extensions (Add-ons) influence for Firefox users.

Some part of work was already done.
We are able to install add-ons using [Puppeteer(https://github.com/GoogleChrome/puppeteer).
Thanks for the help to extend API for web-ext repo to @aslushnikov @rpl @Rob--W.

More info can be found puppeteer/puppeteer#4162 and puppeteer/puppeteer#4162

Next step - use performance API which isn’t implemented yet in Firefox.
Related bugtracking issues:

• https://bugzilla.mozilla.org/show_bug.cgi?id=1463069
• https://bugzilla.mozilla.org/show_bug.cgi?id=1348405

All experiments are stored separate branch - https://github.com/treosh/exthouse/tree/firefox-experimental

Add support for custom login script to evaluate extensions requiring authentication (more than 50% of all extensions).

Solution

Puppeteer powers Exthouse and allows us to get an extension page. A custom script could do some extra steps to finish the authentication and extension install.

Possible API

login.js

exports.handler = async (extension, page) => {
  // perform steps for login
}

Use the login script:

exthouse my-ext.crx --require=login.js

After simple run of exthouse recived an exception:

Error: Protocol error (Tracing.end): Tracing failed
    at Function.fromProtocolMessage (/Users/gregoryshehet/.nvm/versions/node/v12.14.1/lib/node_modules/exthouse/node_modules/lighthouse/lighthouse-core/lib/lh-error.js:132:19)
    at /Users/gregoryshehet/.nvm/versions/node/v12.14.1/lib/node_modules/exthouse/node_modules/lighthouse/lighthouse-core/gather/connections/connection.js:123:25
    at processTicksAndRejections (internal/process/task_queues.js:94:5) {
  protocolMethod: 'Tracing.end',
  protocolError: 'Tracing failed',
  friendlyMessage: undefined
}
Cannot destructure property `extFile` of 'undefined' or 'null'.

So I tried to fix it locally by wrapping extFile destructor inside if condition:

 if (completeRes[medianIndex]) {
      const { extFile } = completeRes[medianIndex]
      const medianFileName = join(tmpDir, `median-${extFile}`.replace(/-\d+(?=\.\w+$)/, ''))
      await remove(medianFileName)
      await symlink(extFile, medianFileName)
}

But got another error, and think to create an issue :)

ENOENT: no such file or directory, open '<SOME_PATH>/median-result-collamark.json'

So I created such a file manually. Some note, script creates another json file median-result-default.json with a new line symbol in the end '\n' of the file name.

After I received an error:

Error: Protocol error (Tracing.end): Tracing failed
    at Function.fromProtocolMessage (/Users/gregoryshehet/work/extension-perfomance/node_modules/lighthouse/lighthouse-core/lib/lh-error.js:132:19)
    at /Users/gregoryshehet/work/extension-perfomance/node_modules/lighthouse/lighthouse-core/gather/connections/connection.js:123:25
    at processTicksAndRejections (internal/process/task_queues.js:94:5) {
  protocolMethod: 'Tracing.end',
  protocolError: 'Tracing failed',
  friendlyMessage: undefined
}
Cannot read property 'main-thread-tasks' of undefined

It seems there is some bugs in the library.

I got an error when installing exthouse, saying that

ERROR: Failed to download Chromium r686378! Set "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" env variable to skip download. Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/exthouse/node_modules/puppeteer/.local-chromium'

There is a workaround for people who also got this error. Try:

sudo npm install --global exthouse --unsafe-perm=true --allow-root

Need to figure out final options and response as array or object.

• Social Fixer - 179,196 users, probably has influence on user interaction. Since social media used a lot, looks like a good candidate.

anyone has more?