Skip to content

Instantly share code, notes, and snippets.

@crabmusket
Last active December 11, 2024 21:59
Show Gist options
  • Save crabmusket/b164c9b9d3c43db9bddbfb83afde0319 to your computer and use it in GitHub Desktop.
Save crabmusket/b164c9b9d3c43db9bddbfb83afde0319 to your computer and use it in GitHub Desktop.
Headless rendering with THREE.js

Headless THREE

Created with

Make sure you install headless-gl's dependencies, then run with XVFB like so:

Xvfb :99 -screen 0 1200x1200x16 &
DISPLAY=:99.0 node three_headless.js

You should be able to view the resulting test.ppm file in most OSes, or open it with e.g. GIMP and convert it to a JPEG. An example is attached below.

If you want to output formats other than the simple P3 text-based format, the result of extractPixels is suitable for use with Sharp.

For a package with full support for all THREE.js's features, many of which use the DOM, try three-universal, or do it yourself with JSDOM. If you don't want to do that, you'll have to implement most of your own loaders for images, geometry etc. or monkey-patch Node's globals or THREE's built-in loaders.

Prior art:

const gl = require("gl"); // https://npmjs.com/package/gl v4.9.0
const THREE = require("three"); // https://npmjs.com/package/three v0.124.0
const fs = require("fs");
const {scene, camera} = createScene();
const renderer = createRenderer({width: 200, height: 200});
renderer.render(scene, camera);
const image = extractPixels(renderer.getContext());
fs.writeFileSync("test.ppm", toP3(image));
process.exit(0);
function createScene() {
const scene = new THREE.Scene();
const box = new THREE.Mesh(new THREE.BoxBufferGeometry(), new THREE.MeshPhongMaterial());
box.position.set(0, 0, 1);
box.castShadow = true;
scene.add(box);
const ground = new THREE.Mesh(new THREE.PlaneGeometry(100, 100), new THREE.MeshPhongMaterial());
ground.receiveShadow = true;
scene.add(ground);
const light = new THREE.PointLight();
light.position.set(3, 3, 5);
light.castShadow = true;
scene.add(light);
const camera = new THREE.PerspectiveCamera();
camera.up.set(0, 0, 1);
camera.position.set(-3, 3, 3);
camera.lookAt(box.position);
scene.add(camera);
return {scene, camera};
}
function createRenderer({height, width}) {
// THREE expects a canvas object to exist, but it doesn't actually have to work.
const canvas = {
width,
height,
addEventListener: event => {},
removeEventListener: event => {},
};
const renderer = new THREE.WebGLRenderer({
canvas,
antialias: false,
powerPreference: "high-performance",
context: gl(width, height, {
preserveDrawingBuffer: true,
}),
});
renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.PCFSoftShadowMap; // default PCFShadowMap
// This is important to enable shadow mapping. For more see:
// https://threejsfundamentals.org/threejs/lessons/threejs-rendertargets.html and
// https://threejsfundamentals.org/threejs/lessons/threejs-shadows.html
const renderTarget = new THREE.WebGLRenderTarget(width, height, {
minFilter: THREE.LinearFilter,
magFilter: THREE.NearestFilter,
format: THREE.RGBAFormat,
type: THREE.UnsignedByteType,
});
renderer.setRenderTarget(renderTarget);
return renderer;
}
function extractPixels(context) {
const width = context.drawingBufferWidth;
const height = context.drawingBufferHeight;
const frameBufferPixels = new Uint8Array(width * height * 4);
context.readPixels(0, 0, width, height, context.RGBA, context.UNSIGNED_BYTE, frameBufferPixels);
// The framebuffer coordinate space has (0, 0) in the bottom left, whereas images usually
// have (0, 0) at the top left. Vertical flipping follows:
const pixels = new Uint8Array(width * height * 4);
for (let fbRow = 0; fbRow < height; fbRow += 1) {
let rowData = frameBufferPixels.subarray(fbRow * width * 4, (fbRow + 1) * width * 4);
let imgRow = height - fbRow - 1;
pixels.set(rowData, imgRow * width * 4);
}
return {width, height, pixels};
}
function toP3({width, height, pixels}) {
const headerContent = `P3\n# http://netpbm.sourceforge.net/doc/ppm.html\n${width} ${height}\n255\n`;
const bytesPerPixel = pixels.length / width / height;
const rowLen = width * bytesPerPixel;
let output = headerContent;
for (let i = 0; i < pixels.length; i += bytesPerPixel) {
// Break output into rows
if (i > 0 && i % rowLen === 0) {
output += "\n";
}
for (let j = 0; j < 3; j += 1) {
// This is super inefficient but hey
output += pixels[i + j] + " ";
}
}
return output;
}
@crabmusket
Copy link
Author

crabmusket commented Jun 10, 2022

Right, so I assume the call to gl returned undefined, i.e. there was an error creating the context. I do remember having issues creating the context sometimes, but I don't remember how I debugged them; it was quite frustrating! I haven't actually run this code for some time so it's not fresh. Do make sure that XVFB is running, but aside from that I don't have any suggestions, sorry!

Oh, you may need to fiddle with the XVFB launching command. 1200x1200x16 worked in my environment but there are other possibilities I think.

@Jackychen-bluescape
Copy link

Jackychen-bluescape commented Jun 10, 2022

Right, so I assume the call to gl returned undefined, i.e. there was an error creating the context. I do remember having issues creating the context sometimes, but I don't remember how I debugged them; it was quite frustrating! I haven't actually run this code for some time so it's not fresh. Do make sure that XVFB is running, but aside from that I don't have any suggestions, sorry!

Oh, you may need to fiddle with the XVFB launching command. 1200x1200x16 worked in my environment but there are other possibilities I think.

No worries! Interesting, I'd assume my machine is running X11 by default and that's why the gl context worked off the bat. Perhaps I'd need to somehow run that on my server as well - not entirely sure how XVFB works tbh, but I'll look into it. Thanks :)

@skerit
Copy link

skerit commented Jun 22, 2022

I just keep getting the navigator is not defined error. And three-universal seems to be abandoned.

@crabmusket
Copy link
Author

crabmusket commented Jun 23, 2022

@skerit it looks like a check on navigator was added recently (somewhere around r139-r141?). So I imagine getting this code running with recent THREE versions will require patching global to add a dummy navigator.

(Search results suggest this is the only place navigator is currently used.)

@ZenithRogue
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment