robohornet / robohornet Goto Github PK

Just a note to update benchmark.js to v1.0. I'll try to contribute more PR's once I get the MS "ok".

2D canvas drawing is slower than it should be in general.

Some of the tests (for example the ES5 getters/setters test) feature simple for loops that do the same computation repeatedly with a constant number of iterations. Some of them don't even use the result.

Loops like this can be trivially optimized down to a no-op by a runtime. You should either replace them with 'real' code that uses the result, or take steps to prevent optimization (for example, a try/finally block wrapping each iteration would prevent optimization in current revs of V8 and SpiderMonkey, so optimizations wouldn't occur across that boundary. You'd need to wrap with functions though, and it'd add a lot of overhead...)

Within the harness.

Performance issue #9 tests the DOM Range API, which IE9 doesn't support (although IE10 will).

We need to add an admissible workaround for IE9.

The following benchmarks fail to make progress or complete (they don't use a lot of CPU; they don't appear to do anything):

canvasdrawline
canvastodataurl
createtable
table_scrolltop
resizecol
svgresize
range

Affects:

• Google Calendar

We should add a benchmark to track this, based on Sorin's test: http://jsperf.com/postmessageperf

The test appears to make progress, but then after a few seconds gets stuck and no progress can be observed for minutes. When you close the window, the main tab with RoboHornet in it spontaneously refreshes, saying "A problem with this page caused Internet Explorer to close and reopen the tab."

Running the test in standalone reports that it completes successfully in 2 ms.

No script errors are observed.

Line 84 fails because getElementsByClassName is not supported.

As documented in https://github.com/robohornet/robohornet/wiki/Benchmark-Compatibility . Core should be treated as the "All" label, with "Extended" being the "real" "all" label. "Core" should by default be selected.

Nothing happens when you run the scrollTop test in Opera using the harness, which means the test just sits there and never completes. Same thing happens in Safari.

ES5 Property accessors are useful, but they are slower than doing normal normal properties or methods.

see also https://bugzilla.mozilla.org/show_bug.cgi?id=626021

Adding columns to a table dynamically is slow.

Currently tests in standalone mode don't use benchmark.js for simplicity. However, benchmark.js does many nice things like avoid the "script unresponsive" error.

Ensure IE is running in its edge compat modes. I noticed some tests have failed in the past because API expected to exist did not, because IE was running in an older compat mode. We should ensure it's running the latest.

Often when running some of the longer-running tests, it can be difficult to distinguish whether a test is doing useful work or is stuck. Some kind of progress indicator would make that easier to distinguish, although we'd have to be carful not to influence the performance of the tests unnecessarily.

Adding many rows to a table dynamically is slow. Apps like Google Spreadsheets would like this to be fast.

Canvas toDataURL performance is slow. Apps like Google Presentations would like it to be faster.

Potentially on other browser/platform combinations as well.

Chrome on Mac will not allow a popup window to go even partially off-screen; it appears to have no such restrictions on Win 7.

Ensuring that the test window is fully visible is important for accurate performance testing.

At the completion of the test we should upload the results to browserscope so that they can be compared to other folks over time.

Figuring out how to handle new versions of the suite in a way that makes sense with historical values will be a challenge.

If the test window is closed by the user, we should note the benchmark's status as "Closed by user" and move on to the next benchmark, treating it like a failed benchmark.

This is hard to do correctly; a naive implementation apparently had a race condition between the parent and child windows.

The best way to do this is probably to have the parent test runner poll for the existence of the window every couple of seconds. We would have to be careful not to have it poll too often or it could effect the score.

Reading from localStorage can be very slow.

Currently the first test is loaded in response to a user action so it's not blocked, but later ones will be. This leads to someone thinking it's working, then leaving while the tests run, only to come back and find only the first ran.

I haven't had the chance to dig into the code, but something is causing the SVG resize test to take ~15 minutes or more to finish, which is waaaaaaaaaay to long.

Is there something in the code that waits for it to stabilize or smth that would cause the test to take so long to finish?

Clearing a 2D canvas is typically very slow, anyone who uses the canvas for games or creative coding will want this to be much faster.

The propery_accessors test appears to run correctly in IE, but when it finishes and the window closes, the main window gives it a status of "Aborted by user" even though it finished naturally and closed itself.

Original title: Firefox: SVG Resize leads to unresponsive script warning, never completes

The test takes a long time; after awhile the unresponsive script dialog shows up. Clicking wait leads to Firefox just stalling forever until you force close it.

Not confirmed yet, but should happen based on reviewing the code today.

Creating tables via innerHTML is very slow. Apps like Google Spreadsheets would like this to be fast.

The descendant selector in CSS can be slow, but it is very useful and should be faster.

David Hyatt (long ago) and others (more recently) have reported that css's descendant selectors are among the slowest, yet they remain some of the most popular selectors used by authors. Webkit's Antti has proved that there are huge optimization gains to be had here.

Resizing columns in a table can be very slow. Apps like Google Spreadsheets would like for this to be faster. Google Spreadsheets in particular runs into this problem in a very real way with their defaults of 100 rows by 20 columns.

Version 6.0 on OS X 10.7.4.

Safari crashes hard, bringing down the entire browser. Before it would just do a crash and refresh.

Others have reported this test works on Safari 6.

Line 26: r.createContextualFragment fails because the object does not support that property or method.

When there are a lot of animated gifs on the page (for example, on some forums), the performance of the page can get very slow in some browsers.

An unconfirmed report that the 2D Canvas Test (#4 in the list) crashes Chrome Canary on Windows 7.

To run the suite you need to whitelist robohornet.org with your popup blocker. Ideally when we detect that popups are blocked, we'd let the users know immediately and instruct them how to fix it.

Applications like Ember.js would like to make more use of the DOM Range API, but it is too slow.

Handlebars has to go with a hacky implementation of script tags to define component boundaries because the range API is so slow in current browsers. http://www.w3.org/TR/DOM-Level-2-Traversal-Range/ranges.html

The test just appears to get stuck and never complete, not necessarily from using too much CPU.

Scrolling large tables can be slow. Apps like Google Spreadsheets would like this to be faster.

Even for fixed-width tables.

I noticed several tests have loops. Those can usually be avoided because Benchmark.js will has it's own test loop.

Only in Safari, when the Safari 2D test is finished, the popup spontaneously refreshes, running in standalone mode. When it finishes, the main window doesn't notice, so the whole harness stops moving forward to the next test.

Creating varargs javascript functions (which requires accessing the arguments array) can be extremely slow.

The instantiation of the arguments array is slow. It makes functions with variable # of arguments inefficient. Example: goog.dom.classes.add would be twice as fast in Chrome if it expected a single className argument.

It continues to run, but often shows a "Not responding" warning in the titlebar that stays for many seconds and then goes away. Firefox's "unresponsive script" warning does not show. After 15 minutes, the test had still not completed.

Resizing SVG can be slow. Apps like Google Presentations would like for this to be faster.

Because of the use of popup windows, Robohornet's tests are blocked on mobile safari. So right now we can't get results there.

It's possible to get results on Chrome on Android, once you allow the popups.

FF Mobile, Opera Mobile untested so far.

Accessing certain DOM properties (like offsetTop, offsetLeft, offsetWidth, and offsetHeight) can cause a synchronous reflow of the document, often with severe performance implications. The best practice is to batch up these property reads and sets to minimize the number of reflows that must occur, but in many cases it's just not possible to avoid synchronous reflows. Reflow performance is critical and should be improved.

Pure math is far slower than compiled C/C++

Google Maps:

At some points we do significant amounts of pure math-based operations, such as determining as subset of lines that don't intersect for label placement. This is essentially pure math as it doesn't depend on the DOM, browser, or any significant features of JavaScript. That is, we could write a C++ program to compute the same results with essentially a list of numbers as input and output, and it would be much faster. It seems there are inefficiencies in how JS technically should be executed that make it so much slower, even though we could get the same results in our particular case if they were ignored.

Raw javascript performance can be the bottleneck in certain applications.

The canvas clearing test is a complete fantasy as long as it never actually draws anything. It's trivial to optimize clearing an already cleared render target (modern GPUs make a clear inexpensive if not free) and the fact that you never draw anything to it after the clear means that browsers can trivially identify that they have no need to do any actual work either. Furthermore, you're not doing anything with the results of the rendering operation, and some browsers will trivially optimize rendering operations that are never presented (for example, rendering to an offscreen canvas and never using the output can be deferred in some Chrome configurations).

This test should not be included or scored until it's real.

Writing to local storage can be very slow in some browsers.