Anti-aliasing (FXAA) with headless-gl and three.js

README.md

Aliased

Anti-aliased

Getting the code

git clone https://gist.github.com/0d79ce3c7384cf6d1bb6.git

Executing the code

$ npm i
$ node_modules/.bin/coffee cmd_antialias.coffee -i test_aliased.png -o out.png
THREE.WebGLRenderer 72
THREE.WebGLRenderer: OES_texture_float extension not supported.
THREE.WebGLRenderer: OES_texture_float_linear extension not supported.
THREE.WebGLRenderer: OES_texture_half_float extension not supported.
THREE.WebGLRenderer: OES_texture_half_float_linear extension not supported.
THREE.WebGLRenderer: OES_standard_derivatives extension not supported.
THREE.WebGLRenderer: ANGLE_instanced_arrays extension not supported.
THREE.WebGLRenderer: OES_element_index_uint extension not supported.
THREE.WebGLRenderer: EXT_texture_filter_anisotropic extension not supported.
Image written: out.png

Those warnings are harmless for our test case, but might be problematic for some folks. Support for extension is planned and coming -> stackgl/headless-gl#5

If you are on Linux you will need Xvfb. One way to do it:

$ xvfb-run -s "-ac -screen 0 1280x1024x24” node_modules/.bin/coffee cmd_antialias.coffee -i test_aliased.png -o out.png

More infos here -> https://github.com/stackgl/headless-gl#how-can-headless-gl-be-used-on-a-headless-linux-machine

Inspecting the output

Tada ! You just created an image thanks to OpenGL and many awesome libraries. How cool is that. Now open the output image. On a Mac you can just do that:

open out.png

Bummer, the sample is in coffee-script

npm run compile

This will compile the .coffee file to javascript and print it in your terminal. It's almost the same as the .coffee.

aafrag.glsl

//
// Assembled (de-glslify'ed) from https://github.com/mattdesl/glsl-fxaa
//

#ifndef FXAA_REDUCE_MIN
    #define FXAA_REDUCE_MIN   (1.0/ 128.0)
#endif
#ifndef FXAA_REDUCE_MUL
    #define FXAA_REDUCE_MUL   (1.0 / 8.0)
#endif
#ifndef FXAA_SPAN_MAX
    #define FXAA_SPAN_MAX     8.0
#endif

// optimized version for mobile, where dependent 
// texture reads can be a bottleneck
vec4 fxaa(sampler2D tex, vec2 fragCoord, vec2 resolution,
            vec2 v_rgbNW, vec2 v_rgbNE, 
            vec2 v_rgbSW, vec2 v_rgbSE, 
            vec2 v_rgbM) {
    vec4 color;
    vec2 inverseVP = vec2(1.0 / resolution.x, 1.0 / resolution.y);
    vec3 rgbNW = texture2D(tex, v_rgbNW).xyz;
    vec3 rgbNE = texture2D(tex, v_rgbNE).xyz;
    vec3 rgbSW = texture2D(tex, v_rgbSW).xyz;
    vec3 rgbSE = texture2D(tex, v_rgbSE).xyz;
    vec4 texColor = texture2D(tex, v_rgbM);
    vec3 rgbM  = texColor.xyz;
    vec3 luma = vec3(0.299, 0.587, 0.114);
    float lumaNW = dot(rgbNW, luma);
    float lumaNE = dot(rgbNE, luma);
    float lumaSW = dot(rgbSW, luma);
    float lumaSE = dot(rgbSE, luma);
    float lumaM  = dot(rgbM,  luma);
    float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
    float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));
    
    vec2 dir;
    dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
    dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));
    
    float dirReduce = max((lumaNW + lumaNE + lumaSW + lumaSE) *
                          (0.25 * FXAA_REDUCE_MUL), FXAA_REDUCE_MIN);
    
    float rcpDirMin = 1.0 / (min(abs(dir.x), abs(dir.y)) + dirReduce);
    dir = min(vec2(FXAA_SPAN_MAX, FXAA_SPAN_MAX),
              max(vec2(-FXAA_SPAN_MAX, -FXAA_SPAN_MAX),
              dir * rcpDirMin)) * inverseVP;
    
    vec3 rgbA = 0.5 * (
        texture2D(tex, fragCoord * inverseVP + dir * (1.0 / 3.0 - 0.5)).xyz +
        texture2D(tex, fragCoord * inverseVP + dir * (2.0 / 3.0 - 0.5)).xyz);
    vec3 rgbB = rgbA * 0.5 + 0.25 * (
        texture2D(tex, fragCoord * inverseVP + dir * -0.5).xyz +
        texture2D(tex, fragCoord * inverseVP + dir * 0.5).xyz);

    float lumaB = dot(rgbB, luma);
    if ((lumaB < lumaMin) || (lumaB > lumaMax))
        color = vec4(rgbA, texColor.a);
    else
        color = vec4(rgbB, texColor.a);
    return color;
}

void texcoords(vec2 fragCoord, vec2 resolution,
            out vec2 v_rgbNW, out vec2 v_rgbNE,
            out vec2 v_rgbSW, out vec2 v_rgbSE,
            out vec2 v_rgbM) {
    vec2 inverseVP = 1.0 / resolution.xy;
    v_rgbNW = (fragCoord + vec2(-1.0, -1.0)) * inverseVP;
    v_rgbNE = (fragCoord + vec2(1.0, -1.0)) * inverseVP;
    v_rgbSW = (fragCoord + vec2(-1.0, 1.0)) * inverseVP;
    v_rgbSE = (fragCoord + vec2(1.0, 1.0)) * inverseVP;
    v_rgbM = vec2(fragCoord * inverseVP);
}

vec4 apply(sampler2D tex, vec2 fragCoord, vec2 resolution) {
    vec2 v_rgbNW;
    vec2 v_rgbNE;
    vec2 v_rgbSW;
    vec2 v_rgbSE;
    vec2 v_rgbM;

    // compute the texture coords
    texcoords(fragCoord, resolution, 
              v_rgbNW, v_rgbNE, v_rgbSW, v_rgbSE, v_rgbM);
    
    // compute FXAA
    return fxaa(tex, fragCoord, resolution, 
                v_rgbNW, v_rgbNE, v_rgbSW, v_rgbSE, v_rgbM);
}

uniform vec2 resolution;
uniform sampler2D dataTexture;
varying vec2 vUv;

void main() {
    vec2 fragCoord = vUv * resolution;
    gl_FragColor = apply(dataTexture, fragCoord, resolution);
}

aavert.glsl

varying vec2 vUv;

void main() {
    vUv = uv;
    gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
}

antialias.coffee

#
# headless-gl does not support anti-aliasing yet, so we do it
# ourself when running in node.
# Adapted from https://www.npmjs.com/package/three-shader-fxaa
#

fs    = require('fs')
THREE = require('three')

offscreen_renderer = require './offscreen_renderer'

# create a namespace to export our public methods
antialias = exports? and exports or @antialias = {}

class antialias.AntiAliaser

    execute: (rndr, data, width, height) ->

        # create a pixel buffer of the correct size
        pixels = new Uint8Array(4 * width * height)

        if rndr.nogl
            return pixels

        renderer = rndr.renderer

        dataTexture = new THREE.DataTexture(data, width, height,
                                            THREE.RGBAFormat)
        dataTexture.needsUpdate = true
        dataTexture.minFilter = THREE.LinearFilter

        # THREE.js business starts here
        scene = new THREE.Scene()

        # 2D rendering, use an ortho camera
        camera = new THREE.OrthographicCamera( -1, 1, 1, -1, 0, 1 )

        resolution = new THREE.Vector2(width, height)
        obj =
            vertexShader:   fs.readFileSync('aavert.glsl').toString()
            fragmentShader: fs.readFileSync('aafrag.glsl').toString()
            uniforms:
                dataTexture: { type: "t", value: dataTexture }
                resolution:  { type: "v2", value: resolution }

        material = new THREE.ShaderMaterial(obj)

        quad = new THREE.Mesh(new THREE.PlaneBufferGeometry(2, 2), material)

        # Create the mesh and add it to the scene
        scene.add(quad)

        # Let's create a render target object where we'll be rendering
        rtTexture = new THREE.WebGLRenderTarget(
            width, height, {
                minFilter: THREE.LinearFilter,
                magFilter: THREE.NearestFilter,
                format: THREE.RGBAFormat
        })

        # render
        renderer.render(scene, camera, rtTexture, true)

        # read render texture into buffer
        gl = renderer.getContext()

        # read back in the pixel buffer
        renderer.readRenderTargetPixels(rtTexture, 0, 0,
                                        width, height, pixels)

        return pixels

cmd_antialias.coffee

#
# Command line driver for the antialising code
#

ArgumentParser = require('argparse').ArgumentParser
PNG = require('pngjs').PNG
fs = require('fs')
colors = require('colors')

antialias = require './antialias'
offscreen_renderer = require './offscreen_renderer'

parser = new ArgumentParser
    version: '0.0.1'
    addHelp:true
    description: 'Argparse examples: sub-commands'

parser.addArgument(
    [ '-i', '--input' ],
        action: 'store'
        required: true
        help: 'the input image.'
)
parser.addArgument(
    [ '-o', '--output' ],
        action: 'store'
        help: 'the output image where images will be written'
)

# FIXME: code duplicated
writePng = (path, width, height, pixels) ->

    png = new PNG
        width: width
        height: height

    for i in [0...pixels.length]
        png.data[i] = pixels[i]

    buff = PNG.sync.write(png)
    fs.writeFileSync(path, buff)
    console.log("Image written: " + "#{ path }".cyan)

args = parser.parseArgs()

png = new PNG
stream = fs.createReadStream args.input
stream.pipe png

png.on 'parsed', () ->
    width  = png.width
    height = png.height
    data   = png.data

    rndr = new offscreen_renderer.OffscreenRenderer(width, height)
    antialiaser = new antialias.AntiAliaser()
    antiAliasedPixels = antialiaser.execute(rndr, data, width, height)
    writePng(args.output, width, height, antiAliasedPixels)

#
# Test with
#
test = """
rm -f out.png && \
node_modules/.bin/coffee cmd_antialias.coffee -i test_aliased.png -o out.png
"""

offscreen_renderer.coffee

#
# Small class to encapsulate a THREE.js + headless.gl
# Offscreen Rendering context
#
# Obtaining one can fails on Linux if OpenGL is missing or Xvfb is not running,
# we set the nogl member to true in those cases.
#

THREE = require('three')

# create a namespace to export our public methods
offscreen_renderer = exports? and exports or @offscreen_renderer = {}

class offscreen_renderer.OffscreenRenderer

    constructor: (width, height) ->

        @renderer = null
        @nogl = false

        try
            gl = require("gl")()
        catch e
            @nogl = true
            return

        #
        # This stub is used to prevent an irrelevant THREE.js warning for the
        # headless code that I got tired of seing printed on the terminal.
        #
        dummyAddEventListener = (a, b, c) -> x = 1
        canvas =
            addEventListener: dummyAddEventListener

        @renderer = new THREE.WebGLRenderer
            antialias: true
            width: width
            height: height
            canvas: canvas
            context: gl

package.json

{
  "name": "antialiasing-headless-gl",
  "version": "1.0.0",
  "private": true,
  "dependencies": {
    "argparse": "latest",
    "colors": "^1.1.2",
    "gl": "2.1.4",
    "pngjs": "^2.2.0",
    "three": "0.72.0"
  },
  "devDependencies": {
    "coffee-script": "^1.10.0"
  }
}

test_aliased.png

@bsergean, thanks for sharing! What versions of Mesa/OpenGL/GLSL were you using?

I tried with Mesa 12.0.3, OpenGL 3.0. GLSL 1.3 in a Ubuntu docker container and get a compilation error for aafrag.glsl:
THREE.WebGLShader: Shader couldn't compile.
THREE.WebGLShader: gl.getShaderInfoLog() vertex 0:2(1): error: syntax error, unexpected NEW_IDENTIFIER
I was wondering if this is a version problem.

I have no memories of those versions, I could look it up ... I was using CentOS 7.1 back then, and the stock mesa/etc... I would get from yum install.