Anti-aliasing (FXAA) with headless-gl and three.js

README.md

Aliased

Anti-aliased

Getting the code

git clone https://gist.github.com/0d79ce3c7384cf6d1bb6.git

Executing the code

$ npm i
$ node_modules/.bin/coffee cmd_antialias.coffee -i test_aliased.png -o out.png
THREE.WebGLRenderer 72
THREE.WebGLRenderer: OES_texture_float extension not supported.
THREE.WebGLRenderer: OES_texture_float_linear extension not supported.
THREE.WebGLRenderer: OES_texture_half_float extension not supported.
THREE.WebGLRenderer: OES_texture_half_float_linear extension not supported.
THREE.WebGLRenderer: OES_standard_derivatives extension not supported.
THREE.WebGLRenderer: ANGLE_instanced_arrays extension not supported.
THREE.WebGLRenderer: OES_element_index_uint extension not supported.
THREE.WebGLRenderer: EXT_texture_filter_anisotropic extension not supported.
Image written: out.png

Those warnings are harmless for our test case, but might be problematic for some folks. Support for extension is planned and coming -> stackgl/headless-gl#5

If you are on Linux you will need Xvfb. One way to do it:

$ xvfb-run -s "-ac -screen 0 1280x1024x24” node_modules/.bin/coffee cmd_antialias.coffee -i test_aliased.png -o out.png

More infos here -> https://github.com/stackgl/headless-gl#how-can-headless-gl-be-used-on-a-headless-linux-machine

Inspecting the output

Tada ! You just created an image thanks to OpenGL and many awesome libraries. How cool is that. Now open the output image. On a Mac you can just do that:

open out.png

Bummer, the sample is in coffee-script

npm run compile

This will compile the .coffee file to javascript and print it in your terminal. It's almost the same as the .coffee.

aafrag.glsl

//
// Assembled (de-glslify'ed) from https://github.com/mattdesl/glsl-fxaa
//

#ifndef FXAA_REDUCE_MIN
    #define FXAA_REDUCE_MIN   (1.0/ 128.0)
#endif
#ifndef FXAA_REDUCE_MUL
    #define FXAA_REDUCE_MUL   (1.0 / 8.0)
#endif
#ifndef FXAA_SPAN_MAX
    #define FXAA_SPAN_MAX     8.0
#endif

// optimized version for mobile, where dependent 
// texture reads can be a bottleneck
vec4 fxaa(sampler2D tex, vec2 fragCoord, vec2 resolution,
            vec2 v_rgbNW, vec2 v_rgbNE, 
            vec2 v_rgbSW, vec2 v_rgbSE, 
            vec2 v_rgbM) {
    vec4 color;
    vec2 inverseVP = vec2(1.0 / resolution.x, 1.0 / resolution.y);
    vec3 rgbNW = texture2D(tex, v_rgbNW).xyz;
    vec3 rgbNE = texture2D(tex, v_rgbNE).xyz;
    vec3 rgbSW = texture2D(tex, v_rgbSW).xyz;
    vec3 rgbSE = texture2D(tex, v_rgbSE).xyz;
    vec4 texColor = texture2D(tex, v_rgbM);
    vec3 rgbM  = texColor.xyz;
    vec3 luma = vec3(0.299, 0.587, 0.114);
    float lumaNW = dot(rgbNW, luma);
    float lumaNE = dot(rgbNE, luma);
    float lumaSW = dot(rgbSW, luma);
    float lumaSE = dot(rgbSE, luma);
    float lumaM  = dot(rgbM,  luma);
    float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
    float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));
    
    vec2 dir;
    dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
    dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));
    
    float dirReduce = max((lumaNW + lumaNE + lumaSW + lumaSE) *
                          (0.25 * FXAA_REDUCE_MUL), FXAA_REDUCE_MIN);
    
    float rcpDirMin = 1.0 / (min(abs(dir.x), abs(dir.y)) + dirReduce);
    dir = min(vec2(FXAA_SPAN_MAX, FXAA_SPAN_MAX),
              max(vec2(-FXAA_SPAN_MAX, -FXAA_SPAN_MAX),
              dir * rcpDirMin)) * inverseVP;
    
    vec3 rgbA = 0.5 * (
        texture2D(tex, fragCoord * inverseVP + dir * (1.0 / 3.0 - 0.5)).xyz +
        texture2D(tex, fragCoord * inverseVP + dir * (2.0 / 3.0 - 0.5)).xyz);
    vec3 rgbB = rgbA * 0.5 + 0.25 * (
        texture2D(tex, fragCoord * inverseVP + dir * -0.5).xyz +
        texture2D(tex, fragCoord * inverseVP + dir * 0.5).xyz);

    float lumaB = dot(rgbB, luma);
    if ((lumaB < lumaMin) || (lumaB > lumaMax))
        color = vec4(rgbA, texColor.a);
    else
        color = vec4(rgbB, texColor.a);
    return color;
}

void texcoords(vec2 fragCoord, vec2 resolution,
            out vec2 v_rgbNW, out vec2 v_rgbNE,
            out vec2 v_rgbSW, out vec2 v_rgbSE,
            out vec2 v_rgbM) {
    vec2 inverseVP = 1.0 / resolution.xy;
    v_rgbNW = (fragCoord + vec2(-1.0, -1.0)) * inverseVP;
    v_rgbNE = (fragCoord + vec2(1.0, -1.0)) * inverseVP;
    v_rgbSW = (fragCoord + vec2(-1.0, 1.0)) * inverseVP;
    v_rgbSE = (fragCoord + vec2(1.0, 1.0)) * inverseVP;
    v_rgbM = vec2(fragCoord * inverseVP);
}

vec4 apply(sampler2D tex, vec2 fragCoord, vec2 resolution) {
    vec2 v_rgbNW;
    vec2 v_rgbNE;
    vec2 v_rgbSW;
    vec2 v_rgbSE;
    vec2 v_rgbM;

    // compute the texture coords
    texcoords(fragCoord, resolution, 
              v_rgbNW, v_rgbNE, v_rgbSW, v_rgbSE, v_rgbM);
    
    // compute FXAA
    return fxaa(tex, fragCoord, resolution, 
                v_rgbNW, v_rgbNE, v_rgbSW, v_rgbSE, v_rgbM);
}

uniform vec2 resolution;
uniform sampler2D dataTexture;
varying vec2 vUv;

void main() {
    vec2 fragCoord = vUv * resolution;
    gl_FragColor = apply(dataTexture, fragCoord, resolution);
}

aavert.glsl

varying vec2 vUv;

void main() {
    vUv = uv;
    gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
}

antialias.coffee

#
# headless-gl does not support anti-aliasing yet, so we do it
# ourself when running in node.
# Adapted from https://www.npmjs.com/package/three-shader-fxaa
#

fs    = require('fs')
THREE = require('three')

offscreen_renderer = require './offscreen_renderer'

# create a namespace to export our public methods
antialias = exports? and exports or @antialias = {}

class antialias.AntiAliaser

    execute: (rndr, data, width, height) ->

        # create a pixel buffer of the correct size
        pixels = new Uint8Array(4 * width * height)

        if rndr.nogl
            return pixels

        renderer = rndr.renderer

        dataTexture = new THREE.DataTexture(data, width, height,
                                            THREE.RGBAFormat)
        dataTexture.needsUpdate = true
        dataTexture.minFilter = THREE.LinearFilter

        # THREE.js business starts here
        scene = new THREE.Scene()

        # 2D rendering, use an ortho camera
        camera = new THREE.OrthographicCamera( -1, 1, 1, -1, 0, 1 )

        resolution = new THREE.Vector2(width, height)
        obj =
            vertexShader:   fs.readFileSync('aavert.glsl').toString()
            fragmentShader: fs.readFileSync('aafrag.glsl').toString()
            uniforms:
                dataTexture: { type: "t", value: dataTexture }
                resolution:  { type: "v2", value: resolution }

        material = new THREE.ShaderMaterial(obj)

        quad = new THREE.Mesh(new THREE.PlaneBufferGeometry(2, 2), material)

        # Create the mesh and add it to the scene
        scene.add(quad)

        # Let's create a render target object where we'll be rendering
        rtTexture = new THREE.WebGLRenderTarget(
            width, height, {
                minFilter: THREE.LinearFilter,
                magFilter: THREE.NearestFilter,
                format: THREE.RGBAFormat
        })

        # render
        renderer.render(scene, camera, rtTexture, true)

        # read render texture into buffer
        gl = renderer.getContext()

        # read back in the pixel buffer
        renderer.readRenderTargetPixels(rtTexture, 0, 0,
                                        width, height, pixels)

        return pixels

cmd_antialias.coffee

#
# Command line driver for the antialising code
#

ArgumentParser = require('argparse').ArgumentParser
PNG = require('pngjs').PNG
fs = require('fs')
colors = require('colors')

antialias = require './antialias'
offscreen_renderer = require './offscreen_renderer'

parser = new ArgumentParser
    version: '0.0.1'
    addHelp:true
    description: 'Argparse examples: sub-commands'

parser.addArgument(
    [ '-i', '--input' ],
        action: 'store'
        required: true
        help: 'the input image.'
)
parser.addArgument(
    [ '-o', '--output' ],
        action: 'store'
        help: 'the output image where images will be written'
)

# FIXME: code duplicated
writePng = (path, width, height, pixels) ->

    png = new PNG
        width: width
        height: height

    for i in [0...pixels.length]
        png.data[i] = pixels[i]

    buff = PNG.sync.write(png)
    fs.writeFileSync(path, buff)
    console.log("Image written: " + "#{ path }".cyan)

args = parser.parseArgs()

png = new PNG
stream = fs.createReadStream args.input
stream.pipe png

png.on 'parsed', () ->
    width  = png.width
    height = png.height
    data   = png.data

    rndr = new offscreen_renderer.OffscreenRenderer(width, height)
    antialiaser = new antialias.AntiAliaser()
    antiAliasedPixels = antialiaser.execute(rndr, data, width, height)
    writePng(args.output, width, height, antiAliasedPixels)

#
# Test with
#
test = """
rm -f out.png && \
node_modules/.bin/coffee cmd_antialias.coffee -i test_aliased.png -o out.png
"""

offscreen_renderer.coffee

#
# Small class to encapsulate a THREE.js + headless.gl
# Offscreen Rendering context
#
# Obtaining one can fails on Linux if OpenGL is missing or Xvfb is not running,
# we set the nogl member to true in those cases.
#

THREE = require('three')

# create a namespace to export our public methods
offscreen_renderer = exports? and exports or @offscreen_renderer = {}

class offscreen_renderer.OffscreenRenderer

    constructor: (width, height) ->

        @renderer = null
        @nogl = false

        try
            gl = require("gl")()
        catch e
            @nogl = true
            return

        #
        # This stub is used to prevent an irrelevant THREE.js warning for the
        # headless code that I got tired of seing printed on the terminal.
        #
        dummyAddEventListener = (a, b, c) -> x = 1
        canvas =
            addEventListener: dummyAddEventListener

        @renderer = new THREE.WebGLRenderer
            antialias: true
            width: width
            height: height
            canvas: canvas
            context: gl

package.json

{
  "name": "antialiasing-headless-gl",
  "version": "1.0.0",
  "private": true,
  "dependencies": {
    "argparse": "latest",
    "colors": "^1.1.2",
    "gl": "2.1.4",
    "pngjs": "^2.2.0",
    "three": "0.72.0"
  },
  "devDependencies": {
    "coffee-script": "^1.10.0"
  }
}

test_aliased.png

It is great,bsergean.
However, I wonder why they put antialias option in headless-gl? It must be so great if this feature works as built-in.
Thank you

@ bsergean : As you know, I am succed to support FXAA with gl-headless in my project.
However, I see quality is not as good as webgl on browser (same code because my code run both on server and browser).
Someone suggest that WebGL use MSAA. So, is it possible for us to support MSAA in gl-headless?
Thank you so much.

@bsergean, thanks for sharing! What versions of Mesa/OpenGL/GLSL were you using?

I tried with Mesa 12.0.3, OpenGL 3.0. GLSL 1.3 in a Ubuntu docker container and get a compilation error for aafrag.glsl:
THREE.WebGLShader: Shader couldn't compile.
THREE.WebGLShader: gl.getShaderInfoLog() vertex 0:2(1): error: syntax error, unexpected NEW_IDENTIFIER
I was wondering if this is a version problem.

I have no memories of those versions, I could look it up ... I was using CentOS 7.1 back then, and the stock mesa/etc... I would get from yum install.