|
| 1 | +Scrollbench |
| 2 | +========= |
| 3 | + |
| 4 | +Scrollbench is a browser scrolling performance test. Scrollbench: |
| 5 | + |
| 6 | + - Runs as a bookmarklet so it can work widely (including mobile), or can be invoked from JS on a page. |
| 7 | + - Uses the best frame time measurement methodology available on the platform; most commonly `requestAnimationFrame` callbacks, hi-resolution timer `window.performance.now()`, and `window.scrollBy()`. |
| 8 | + - Still doesn't give very accurate results in many cases (see below for why). |
| 9 | + |
| 10 | + |
| 11 | +## About |
| 12 | + |
| 13 | +Complex DOM structure, sophisticated styling, and how the rendering engine chooses to deal with them all influence framerates during scrolling. Scrollbench tries to measure the framerate of a scroll and reports the average frame time as well as how many frames went significantly beyond that average (to give a sense of how steady the framerate is, which can be just as important as the average). |
| 14 | + |
| 15 | +Scrollbench was originally a little piece of a performance test in the Chromium project that was ripped out to live on its own here so it could be run in other browsers and contexts. |
| 16 | + |
| 17 | +## Methodology |
| 18 | + |
| 19 | +### Current Best Practices |
| 20 | + |
| 21 | +It is important to note that you shouldn't compare browsers that support different set of technologies. The benchmark is performed using the best API we can find on the platform, but a browser that supports requestAnimationFrame will have very different results compared to one that only has setTimeout. |
| 22 | + |
| 23 | +For this reason at the end of the test you'll get an index which defines the browser "resolution". |
| 24 | + |
| 25 | +Presently we have 3 resolutions: High, Medium and Low. You should be comparing browsers at the same resolution index only. Clicking on the Resolution value in the report you'll also get a detailed list of the technologies used to perform the benchmark. |
| 26 | + |
| 27 | +Don't mix apples and oranges! |
| 28 | + |
| 29 | +### Test Reliability |
| 30 | + |
| 31 | +The more sophisticated your browser the more reliable the benchmark. On recent browsers we can rely on requestAnimationFrame which grants the test a good resolution but that's only a part of the equation. We also need a high definition timer. A good candidate in this case would be window.performance.now. |
| 32 | + |
| 33 | +So if you have both, the test result is considered reliable. |
| 34 | + |
| 35 | +If one feature is missing we fall back to the closest alternative, which in case of requestAnimationFrame is setTimeout (notoriously a low definition timer). |
| 36 | + |
| 37 | +In some cases, though, a JavaScript-driven scroll is never going to emulate a user-initiated scroll very accurately. For instance, on many browsers on touch devices scrolling is done asynchronously (the user flings the page and it keeps going). For the test to be reliable on such devices we need support for triggering an asyncronous scroll from JavaScript, _and_ we need a way measure framerate from the part of the browser that's moving the page (rather than measuring with requestAnimationFrame callbacks, which might be blocked by other JavaScript while the browser's compositor or UI thread cheerfully continues to scroll the page). |
| 38 | + |
| 39 | +Unfortunately today we lack both an asynchronous scroll API and way of measuring framerate that doesn't . This is a hard problem, because if frame rates from the perspective of JavaScript (i.e. requestAnimationFrame) and the user (i.e. what the browser is currently putting on-screen) are different... what does frame rate even mean? |
| 40 | + |
| 41 | +So don't mix apples and oranges on the current technology being used, but also don't be surprised if the apples taste like papayas. |
| 42 | + |
| 43 | +## Quick start |
| 44 | + |
| 45 | +The easiest way to use Scrollbench.js is through the bookmarklet. |
| 46 | + |
| 47 | +1. Make a new bookmark (e.g. just bookmark this site). |
| 48 | +2. Edit the bookmarklet to have the name "scrollbench" and set its URL to this: |
| 49 | + |
| 50 | + `TODO` |
| 51 | + |
| 52 | +3. There is no 3, just run the bookmarklet: |
| 53 | + * On desktop browsers just click it. |
| 54 | + * On iOS Safari open the bookmarks list and select it. |
| 55 | + * On Chrome for Android, start typing "scrollbench" in the omnibox and hit "scrollbench" when it shows up in the suggestions list. |
| 56 | + |
| 57 | +#### iOS |
| 58 | +Note that on iOS, it's a little tricky to edit the bookmarklet: first save the bookmark, then open the bookmarks panel and edit the bookmark you just added. You are now able to modify the site address. Replace the address with the bookmarklet code and voilà! |
| 59 | + |
| 60 | +#### IE10 |
| 61 | +No bookmarklets in IE10, but you can invoke it through the F12 tools console: |
| 62 | + |
| 63 | +1. Open developer tools with "F12" and go to "Console" |
| 64 | +2. Execute: |
| 65 | + |
| 66 | + ```javascript |
| 67 | + document.head.innerHTML+="<script src='http://domain.com/fil.js'></script>"; |
| 68 | + ``` |
| 69 | + |
| 70 | +3. Execute: |
| 71 | + |
| 72 | + ```javascript |
| 73 | + var sb = new ScrollBench(); sb.start(); |
| 74 | + ``` |
| 75 | + |
| 76 | +## Interpretting Results |
| 77 | + |
| 78 | +*TODO* |
| 79 | + |
| 80 | +## Per-page configuration |
| 81 | + |
| 82 | +Sometimes a page doesn't scroll the body, but instead scroll a container element (that has an `overflow: scroll` property set). It's hard to know which element to scroll automatically, so we keep a list of pages to handle specially in `src/config.js` |
| 83 | + |
| 84 | +We welcome additions! To add a new one, copy an existing example -- the basic format is: |
| 85 | + |
| 86 | +```javascript |
| 87 | +// Site name |
| 88 | +config.pages.push({ |
| 89 | + url: '^https://www.example.com/', |
| 90 | + element: document.getElementById('containerElement') |
| 91 | + }); |
| 92 | +``` |
| 93 | + |
| 94 | +You can also use `scrollElementFn` instead of `element` to write a function that finds the element that scrolls. See the documentation below for more on how that works. |
| 95 | + |
| 96 | +Feel free to send a pull request with additional pages added to the config file. |
| 97 | + |
| 98 | +## JavaScript Documentation |
| 99 | + |
| 100 | +Scrollbench can be used from a web page like so: |
| 101 | + |
| 102 | +```html |
| 103 | +<head> |
| 104 | +... |
| 105 | +<script src="scrollbench.js" type="text/javascript"></script> |
| 106 | +</head> |
| 107 | +``` |
| 108 | + |
| 109 | +You can then invoke the scrollbench from inside your code or the debug console with the following: |
| 110 | + |
| 111 | +```javascript |
| 112 | +var sb = new ScrollBench(); |
| 113 | +sb.start(); |
| 114 | +``` |
| 115 | + |
| 116 | +Scrollbench.js (SBJS) is a class, to use it you have to create an instace with `new`. The script initiates itself but does not perform any actual benchmark until you call the start method. |
| 117 | + |
| 118 | +### Options |
| 119 | + |
| 120 | +SBJS accepts one paramenter for special configuration. The parameter must be an object and the accepted values are: |
| 121 | + |
| 122 | +1. **element**: reference to a node object. SBJS can scroll both the whole page or a DOM element. Default: document.documentElement. |
| 123 | +1. **initViewport**: boolean. If true the viewport is set to initial scale 1.0. Default: false. |
| 124 | +1. **iterations**: an integer representing the number of passes to perform. Minimum suggested and default: 2. |
| 125 | +1. **loadConfig**: boolean or string. Loads per site configurations from either the default config file or an user defined file. Default: 2. |
| 126 | +1. **onCompletion**: function. Returns the results to a custom function. Default: null |
| 127 | +1. **scrollableElementFn**: function. Function to be executed to find the scrollable element. Default: null |
| 128 | +1. **scrollStep**: integer value. In case of synchronous scroll, the amount of pixels to scroll per cycle. Default: 100 |
| 129 | +1. **waitForFn**: function. Holds back the benchmark execution until the passed function returns true. Default: null |
| 130 | + |
| 131 | +#### Options: Element |
| 132 | + |
| 133 | +SBJS scrolls the DOM element with id="scrollme": |
| 134 | + |
| 135 | +```javascript |
| 136 | +new ScrollBench({ |
| 137 | + element: document.getElementById('scrollme') |
| 138 | +}).start(); |
| 139 | +``` |
| 140 | + |
| 141 | +By default SBJS scrolls the whole page (namely: document.documentElement). There are special cases where the content is contained inside an element and scrolling the document wouldn't have any effect. In those cases you can specify the element that needs to be scrolled. |
| 142 | + |
| 143 | +#### Options: initViewport |
| 144 | + |
| 145 | +Sets the viewport to scale factor 1.0: |
| 146 | + |
| 147 | +```javascript |
| 148 | +new ScrollBench({ |
| 149 | + initViewport: true |
| 150 | +}).start(); |
| 151 | +``` |
| 152 | + |
| 153 | +This may be useful on mobile devices. If the website doesn't define the viewport size, the mobile browser tries to fit the page on the screen, but each screen/device will have a different scale factor. To get consistent benchmark results you can force the viewport scale factor to 1.0. |
| 154 | + |
| 155 | +#### Options: iterations |
| 156 | + |
| 157 | +Performs the test 5 times instead of the default 2: |
| 158 | + |
| 159 | +```javascript |
| 160 | +new ScrollBench({ |
| 161 | + iterations: 5 |
| 162 | +}).start(); |
| 163 | +``` |
| 164 | + |
| 165 | +For better reliability the test is performed 2 times by default, but this value can be raised for higher resolution. The browser often needs a "warm up" period to reach the highest performance level, so the first pass has sometimes a lower performance rate. |
| 166 | + |
| 167 | +#### Options: loadConfig |
| 168 | + |
| 169 | +Loads the default per site config file |
| 170 | + |
| 171 | +javascriptnew ScrollBench({ |
| 172 | + loadConfig: true |
| 173 | +}).start(); |
| 174 | +Any of the SBJS options can be passed on a per site basis from an user defined configuration file. By default no file is loaded, passing true SBJS loads the default config. You may alternatively pass an URL the configuration file is loaded from. |
| 175 | + |
| 176 | +The configuration is a javascript file that sets the scrollbench_config variable. See the default config for reference. |
| 177 | + |
| 178 | +Note that the bookmarklet loads the default config file but this is not the default SBJS behavior. |
| 179 | + |
| 180 | +#### Options: onCompletion |
| 181 | + |
| 182 | +Bypasses the default report and simply shows an alert box with the results: |
| 183 | + |
| 184 | +```javascript |
| 185 | +new ScrollBench({ |
| 186 | + onCompletion: function (result) { |
| 187 | + alert(JSON.stringify(result, null, ' ')) |
| 188 | + } |
| 189 | +}).start(); |
| 190 | +``` |
| 191 | + |
| 192 | +The parameter returned by the function has the following keys: |
| 193 | + |
| 194 | +* numAnimationFrames: total number of frames of the animation |
| 195 | +* droppedFrameCount: number of frames dropped (ie: performed with low frame rate) |
| 196 | +* totalTimeInSeconds: runtime |
| 197 | +* resolution: test reliability based on browser specs |
| 198 | +* timer: timer used for the test |
| 199 | +* animation: technology used to perform the animation |
| 200 | +* avgTimePerPass: average time needed for each pass |
| 201 | +* framesPerSecond: frames per second the animation is performed at (estimated) |
| 202 | +* travel: pixels travelled by the animation |
| 203 | + |
| 204 | +A value can be followed by a !, - or + meaning respectively: bad, average, good result. |
| 205 | + |
| 206 | +The results are shown by default in an overlaying iFrame, you may want to pass them to a custom function for further inspection. The results will be passed as first parameter to the specified fuction. The default internal report is not executed. |
| 207 | + |
| 208 | +#### Options: scrollableElementFn |
| 209 | + |
| 210 | +Tricky code to get the scrollable element on Gmail: |
| 211 | + |
| 212 | +```javascript |
| 213 | +new ScrollBench({ |
| 214 | + scrollableElementFn: function (callback) { |
| 215 | + gmonkey.load('2.0', |
| 216 | + function (api) { |
| 217 | + callback(api.getScrollableElement()); |
| 218 | + } |
| 219 | + ); |
| 220 | + } |
| 221 | +}).start(); |
| 222 | +``` |
| 223 | + |
| 224 | +In some circumstances it's not possible to foresee the element that should be scrolled. In those cases we can rely on a custom function. The element must be sent as first parameter of a callback function. |
| 225 | + |
| 226 | +#### Options: scrollStep |
| 227 | + |
| 228 | +Increase the number of frames by reducing the scrolling step from the default 100 to 50: |
| 229 | + |
| 230 | +```javascript |
| 231 | +new ScrollBench({ |
| 232 | + scrollStep: 50 |
| 233 | +}).start(); |
| 234 | +``` |
| 235 | + |
| 236 | +The scrolling is normally performed with a 100 pixels step. Each step is a frame. You may want to increase of decrease the number of frames by altering the scrollStep. |
| 237 | + |
| 238 | +#### Options: waitForFn |
| 239 | + |
| 240 | +Wait for the element waitForMe to be loaded before starting the benchmark. |
| 241 | + |
| 242 | +```javascript |
| 243 | +new ScrollBench({ |
| 244 | + waitForFn: function () { |
| 245 | + return document.getElementById('waitForMe'); |
| 246 | + } |
| 247 | +}).start(); |
| 248 | +``` |
| 249 | + |
| 250 | +The passed function is executed repeatedely until it doesn't returs true. When it is finally true, the benchmark begins. |
| 251 | + |
| 252 | + |
| 253 | +## Getting Involved |
| 254 | + |
| 255 | +Scrollbench could be way better! Here are some ways to help: |
| 256 | + |
| 257 | +1. Test it on various sites! If you find one that doesn't work, either file an issue or better yet, add a custom handler for it in `src/config.js` and send us a pull request (see "Per-page configuration" above for more info). |
| 258 | +2. Test it on various browsers! If you find a page that only fails in one browser, file an issue. |
| 259 | +3. Make the code better! Pull requests welcome, or if you have ideas on how this could be improved to be more accurate and want to talk about it, file an issue. |
| 260 | + |
| 261 | +## Updating the bookmarklet code |
| 262 | + |
| 263 | +To update the bookmarklet code (e.g. to change where the source files are served from) you need node and uglify-js installed first. Then modify src/bookmarklet.js as you see fit, and then in the `build` direction run `node makebm.js`, which will minify src/bookmarklet.js and (theoretically) update anywhere the bookmarklet code appears with an updated copy. |
0 commit comments