Please send any feedback on this article to Klemen Slavič
UPDATE: I'm currently in the process of updating the article, as my assumptions about the inner workings of WebKit are incorrect. I will update this article with the relevant facts and provide concrete information in place of my guesstimates below.
I've recently stumbled upon an interesting discovery regarding image rendering performance in most WebKit browsers. Namely, I've been developing a sprite animation component to implement a GIF animation replacement with better compression and performance, where I noticed that some animations appeared to be janky when using multi-frame spritesheets and clipping rectangles. Here's what I found out.
But first, a quick rundown of the basic functioning of the WebKit engine as I understand it.
Disclaimer: I have not hacked on WebKit source yet, I'm making these claims based on empirical evidence, reading through correspondence on the WebKit dev mailing list and on Levi Weintraub's Fluent Conf 2012 talk.
Every element that takes up space in the document (ie. having display set to anything other than none) has an object representation inside the rendering tree. Every block element is represented by a single object while text nodes are split into the largest non-breaking inline blocks. These blocks are then measured and laid out according to their calculated CSS properties. Inline blocks are lined left-to-right (or, if RTL is flipped, right-to-left) using their vertical-align and broken across lines in CR-LF style when the right (or left) edge of the container is reached or if laying down the element would intersect that element with any floated elements on the same float level.
The next step involves creating or updating the rendering tree according to the layout calculated above. Here, elements are inserted into, updated or removed from the rendering tree according to the changeset calculated. Elements that aren't visible (and don't take up space in the document!) are removed from the rendering tree, while the object representing the DOM element is discarded. This includes elements that are wholly outside of their parent clipping rectangle intersections, which I'm assuming is an optimisation worth the expense.
I made a simple JavaScript object that takes an animation description (width, height, sheet dimensions, frame count, FPS and image URLs), builds an animation container and the supporting DOM elements that are then animated using requestAnimationFrame and -webkit-transform according to the calculated run time, synced to the FPS. The result is something akin to GIF animations, but with optional events, compression and scaling.
Due to iOS image size limitations I've split my frames into sprite sheets no larger than 1024×1024 to avoid iOS devices from downsampling larger images and losing resolution (multiple images below or equal to that size work just fine). Using JPEG as the format for sprite sheets, the performance was smooth for modestly-sized frames (eg. covering about 2/3 of the screen realestate or less). This works well on the vast majority of Android and iOS devices, with older and low-CPU profiled devices having a bit of a problem keeping up with the usual 25 FPS animations we tested with.
The problem of jankiness comes with using PNG as the format of choice for sprite sheets of similar size to the above mentioned JPEGs. At first, I thought this might be a compositing problem, since compositing a partially opaque image is computationally more expensive than simply blitting it to the buffer.
I was in for a surprise.
The jankiness rears its ugly head
When testing across devices, I noticed a 1-2 frame skip happening very regularly during the animation, but I couldn't quite tell from looking at it on the devices whether there was a specific point at which the jank happened or not.
Random frame-skipping on very old hardware aside, it seemed that the janks appeared very consistently across different devices. This prompted me to test in a desktop browser to see if I could experience the same kind of jankiness on mobiles. To my surprise, my generously powered laptop with Chrome on Windows displayed the same type of jankiness within the animation. Thinking CPU and GPU power were adequate to produce smooth animations at this size and rate, I was quite taken aback to discover it suffered the same performance problems that plagued the mobile platforms.
The first thing I did was fire up Chrome and open up the Timeline panel in Frame mode. I played the animation with the PNG sprite sheets and hit the record button to see if I could see just what caused the jankiness. Here's a shot of the panel during the recording of a PNG sprite sheet animation:
You can immediately spot the weirdness: there's a substantial periodic spike in the paint operation length which causes the visible jank in the animation. This would have been hardly noticeable at 15FPS, but anything higher than that causes lag because of the paint operation.
Seeing that these happened at regular intervals, I immediately inserted some console.timeStamp() calls to see which frames were rendered during which screen paint in the timeline view and, surely enough, every time a previously invisible sprite sheet came into view, the long paint operation occured.
I initially suspected this might be due to using alpha channels in the PNGs, but after testing without transparency using the same component, I only saw a decrease of 5-10% in the paint operation length. Also, transparency would increase compositing time for all frames, not just the ones coming into view, so I dismissed that idea.
Then it got me thinking about the discarded rendering objects in WebKit – what if discarding the render tree block element representing the sprite sheet meant it had to decode the image every time it needed to create a new rendering object each time it came into view?
I tested my hypothesis using JPEG as the encoding format for the same sprite sheets and it turns out that the paint operation was roughly 4-5 times faster than for PNG. The paint operation was well below the 25FPS limit (still above 60FPS, though), so the animation played out smoothly.
The next step was to confirm that breaking down the sprite sheet into individual frames should shorten the paint operation by reducing the overhead of decoding the huge multi-sheet images. It worked:
The same effects were reproduced on all the test devices that I've tried this method on. All of them worked completely fine after eliminating the large PNG decoding overhead every time the sprite sheet came into view.
It seems that WebKit currently discards the decoded images pruned from the rendering tree and needs to decode the image from its source before painting it to the screen buffer at a later time. Each time this happens, you get the decoding overhead, which in case of PNG, can affect rendering performance of graphically intensive operations like my sprite animation example.
To counteract this effect, here are a few guidelines:
- use JPEG where applicable since it is faster to decode,
- if you have to use PNGs as sprite sheets, make sure to split the sheets into smaller pieces than JPEG sprite sheets, typically under the 512×512px limit to have decoding speed applicable for frame rates around 25FPS,
- experimental, untested: hack your animation script to keep at least 1 pixel of each sprite sheet visible at all times.
Having presented my conclusions I would love to hear from someone on the WebKit team to comment on the above findings and fill in the gaps in my knowledge on the causes of these artifacts. I will update this document as more information is available.
To explore the example yourself, I've put up a test page with the sprite sheets and images so you can see the effects yourself:
Note: while the test page itself doesn't use requestAnimationFrame() and instead uses a setInterval() to set up animation at the desired FPS, the results are the same, but there are less frames to display on the Timeline panel.