Rasterization API proposal

proposal.md

Rasterization API

Abstract

Rasterization API defines interface for making snapshots from HTML elements.

Introduction

Getting a snapshot of HTML element is a common task used for making thumbnails, snapshots during integration testing, printing HTML content, GPU rendering, etc. Previous attempts to implement rasterization like Mozilla`s drawWindow API were lacking of flexibility, and have some security concerns unsolved. However rasterization is not something we should be afraid of since it is already available using third - party libraries such as html2canvas or rasterizeHTML. They work pretty well but have preformance issues and are not well maintained. Current proposition is aiming to provide native API for nodes rasterization in simple, flexible and performant manner.

API interface

Since rasterization result is affected by all document styles it should belong to document. Proposed method name simply reflects intention.

document.rasterize(element, optionsObject): ImageData

###Parameters

• element - HTMLElement or HTMLDocument which should be rasterized
• optionsObject - (optional) rasterization options

###optionsObject properties (all optional)

• async: Boolean - determines is call asynchronous or not. Default - false
• callback: Function - used to handle async rasterization calls. Default - undefined
• waitForResourcesLoading: Boolean - determines should we wait for resources used in Element to load or not. They are images, fonts etc. Default - true
• width: Number - width of resulting ImageData. Do we need it?
• height: Number - height of resulting ImageData. Do we need it?
• scale: Number - zoom factor
• ignoreSelector - if element matches this selector it will be not rendered

###Return value

When called in synchronous mode it returns ImageData which could be displayed via Image tag, be drawn on Canvas or converted to Blob. If called in async mode API returns promise.

###Callback function arguments and scope

Two parameters are sent to callback function:

• ImageData - rasterization result
• HTMLElement - is an element rasterized

Security concerns

• Rasterization API should ignore input fields values set and represent them as empty or just created.
• Iframes rendered accordingly to same origin politics.

Why not element.rasterize(options)?

There's a lot more security issues than just input fields and iframes. Images not from the same origin can not be rendered. Canvases that are dirty (ie, they've had an image drawn in them that is from another origin), can not be rendered. anchor elements can not have their visited style applied.

On top of that you'd need lots of options. You mentioned width and height. Yes of course you'd need them. What size does a <div style="width: 100%;"/> render if it's not in the DOM? You'd probably also need a devicePixelRatio. Then you'd probably also need query functions so you can ask if the content is going to fit for a certain render size. You'd probably need pagination features so you can render a portion of the element, in other words you need scroll offsets. The browser couldn't give you back an ImageData for a long page that might be 1280 x 20000 pixels (which is not uncommon for a page with a long discussion thread).

These are just some of the reasons why no one has implemented it. To actually implement and make it safe would require a ton of work. Mozilla's example API was a privileged only API meaning it could only be run by extensions, not a web page since it wouldn't have been safe. Then, on top of that it's probably not worth the effort. As neat as it would be to play with it's unlikely more than a few experimental websites would use it. Look at 3D CSS, except a few example pages and fun demos it's used almost no where.

I'm not trying to discourage you. Just pointing out there's a many issues to be aware of

@greggman I think it has potential to make the web a viable contender on mobile. We need to be able to animate DOM much faster and a library built on top of something like this would enable that. There are other proposals as well, but I actually think from a performance perspective this is the most promising.

@mattecapu There is a reason to keep rasterization document method. Since it should respect document styles. You option could be added as well, but is there a situation you would like to get a snapshot of node which is somewhere in the limbo, not in document?

@greggman Not sure about images. If you can do it right now with html2canvas, why should you care about the rest? Or I am wrong?

As for useful / not useful. Huh, acceleration events are used rarely also, but would I like to remove it? Nope! I would say that you will always have a bunch of criticism when API is not there, but also some appreciation when somebody find a right use case for thing which is already implemented. For this one I already listed some use cases in the article. Lets think about more...

The plan at the moment is to develop polyfill and then try to move it forward. Probably it will be based on html2canvas but definitely should have retina support since html2canvas do not have for years =(

does this support RTL?

I use this -- it works.

http://cburgmer.github.io/rasterizeHTML.js/

@PixelsCommander An inserted node is anyway adopted by a document. The ownerDocument property of a node says to what document the node is inserted. So, it is possible to say to what document the node belongs.

All rendered things like text nodes, HTML- and SVG-elements, documents, all inherit from Node’s methods and properties so node.rasterize(options); makes more sense, when a node is really inserted into the DOM.

I cry when I think that this is not a thing :'(

what happened to css shaders, does anyone know?

Have a look at immersive-web/dom-overlays#28, maybe there's a path to similar functionality via new WebXR efforts — something like this will be clearly needed 🤔