ASCII Mode in the Vimeo Player

ascii.md

ASCII Mode in the Vimeo Player

When ASCII mode is activated, the resolution of the rendering is calculated based on the size of the player. The color resolution is half that of the monochrome. The frame rate is matched to the video, but it's capped at 30fps.

Rendering a frame

1. The <video> is rendered into an invisible backing canvas at the render resolution (one pixel for each character).
2. The image data is retrieved from the backing canvas.
3. It iterates over each pixel and
   1. Determines the character to use for each pixel based on the brightness of the pixel.
   2. Pushes the character on to an array in the case of monochrome, or pushes a <span> with inline style for color on to the array (as a string).
4. The array is joined to create a single string.
5. The innerHTML is set on the output <pre>.

This works perfectly fine for monochrome because the color is the same for every character, so the paint is very fast. With color, however, the paint exceeds the frame budget unless a majority of the pixels are black or white (black is rendered as a space, and white is the color set on the element, so no <span> needed).

I didn't have a ton of time to experiment with other things, but here's what I remember trying to get the color working well:

Using other methods to construct the string

I think I tried using string concatenation instead of pushing the characters to an array, but everything was already so fast that it didn't make any difference. The bottleneck was with rendering the output, not with the calculations.

Rendering with <canvas> text

Not only did it look really bad, it was slower than using the any of the DOM solutions.

Creating a <span> for each character up front

If I remember correctly, creating a <span> for each character up front, storing a reference to each one, and updating the style and innerText of each one wasn't meaningfully faster than using innerHTML. It may have been worse (I can't remember), but it definitely wasn't any faster.

Other solutions

I didn't try reducing the number of colors, but I don't think it would have helped much unless it resulted in a meaningful reduction to the number of spans needed (so pixels next to each other, but only within a row).

Reducing the resolution would have helped, but too much below where I ended up with it and you couldn't really tell what was going on in the video.

Unfortunately, I didn't keep much documentation on the things I tried. The only thing I could find was a screen recording of a video that it worked really well on: https://vimeo.com/249626098/0f882d1fb6