Compile Xvfb for AWS Lamba

README.md

Compile Xvfb for lambda.

Referenes

Based on the following guide: https://ericdraken.com/running-xvfb-on-a-shared-host-without-x/

Using the following docker image: https://hub.docker.com/r/lambci/lambda/

Start up and attach to docker container to do the build on

It's an Amazon Linux image that matches the environment of Lambda tasks.

# On my Mac, from the directory where I'll be working on my lambda function.
$ docker run -v "$PWD":/var/task -it lambci/lambda:build bash
# bash-4.2#

Install Xvfb and some other dependencies

We'll need some utils that it comes with.

$ yum install xorg-x11-server-Xvfb.x86_64 wget

Download xorg-server source code and unpack

$ cd /tmp
$ wget https://www.x.org/releases/X11R7.7/src/xserver/xorg-server-1.12.2.tar.gz
$ tar -xvf xorg-server-1.12.2.tar.gz
$ cd xorg-server-1.12.2

Modify some files for keymap compiling.

Edit xkb/xkbInit.c at line 754 in XkbProcessArguments()

        return 2;
    }

    // ADDED - Change xkbcomp bin directory with an environment variable
    char *xkbBinDir = getenv("XKB_BINDIR");
    if (xkbBinDir) {
        XkbBinDirectory = Xstrdup(xkbBinDir);
    }

    // ADDED - Change base xkb directory with an environment variable
    char *xkbBaseDir = getenv("XKBDIR");
    if (xkbBaseDir) {
        XkbBaseDirectory = Xstrdup(xkbBaseDir);
    }

    return 0;
}

Edit xkb/ddxLoad.c at line 188 in XkbDDXCompileKeymapByNames()

    const char *emptystring = "";
    char *xkbbasedirflag = NULL;
    const char *xkbbindir = emptystring;
    const char *xkbbindirsep = emptystring;

#ifdef WIN32
    char tmpname[PATH_MAX];
    const char *xkmfile = tmpname;
#else
    // MODIFICATION - Now using 'default.xkm' file to satisfy xkbcomp
    // const char *xkmfile = "-";
    const char *xkmfile = "default.xkm";
#endif

Create default.xkb

Contents for default.xkb:

xkb_keymap "default" {
    xkb_keycodes             { include "evdev+aliases(qwerty)" };
    xkb_types                { include "complete" };
    xkb_compatibility        { include "complete" };
    xkb_symbols              { include "pc+us+inet(evdev)" };
    xkb_geometry             { include "pc(pc105)" };
};

Create and compile it:

$ cd /tmp
$ vim default.xkb
# Insert contents above and save
$ xkbcomp -xkm default.xkb

Copy it into your lambda function /lib

$ mkdir /var/task/lib
$ cp default.xkm /var/task/lib

Install devel dependencies

$ yum install pixman-devel.x86_64 \
    libdrm-devel.x86_64 \
    libX11-devel.x86_64 \
    mesa-libGL-devel.x86_64 \
    openssl-devel.x86_64 \
    xorg-x11-xtrans-devel.noarch \
    libxkbfile-devel.x86_64 \
    libXfont-devel.x86_64 \
    libpciaccess-devel.x86_64 \
    xcb-util-keysyms-devel.x86_64

Configure and Make

$ ./configure
$ make

Copy Xvfb to lambda lib

$ cp hw/vfb/Xvfb /var/task/lib

In a new terminal, boot up another docker mounted to our task dir

# cd (your task directory)
$ docker run -v "$PWD":/var/task -it lambci/lambda:build bash

Check that we have any linked libs

Looks ok?

$ ls -la /var/task/lib
total 10812
drwxr-xr-x 5 root root      170 Dec  2 23:55 .
drwxr-xr-x 4 root root      136 Dec  2 23:28 ..
-rw-r--r-- 1 root root      314 Dec  2 23:43 default.xkb
-rw-r--r-- 1 root root    11248 Dec  2 23:43 default.xkm
-rwxr-xr-x 1 root root 11054660 Dec  2 23:55 Xvfb

$ cd /var/task/lib

$ ldd ./Xvfb
  linux-vdso.so.1 =>  (0x00007ffdc6f1a000)
  libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f10f11ca000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007f10f0fc5000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f10f0da9000)
  libpixman-1.so.0 => /usr/lib64/libpixman-1.so.0 (0x00007f10f0afe000)
  libXfont.so.1 => /usr/lib64/libXfont.so.1 (0x00007f10f08c9000)
  libXau.so.6 => /usr/lib64/libXau.so.6 (0x00007f10f06c6000)
  libm.so.6 => /lib64/libm.so.6 (0x00007f10f03c4000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f10f0001000)
  libz.so.1 => /lib64/libz.so.1 (0x00007f10efdeb000)
  /lib64/ld-linux-x86-64.so.2 (0x000056118b113000)
  libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x00007f10efb4f000)
  libfontenc.so.1 => /usr/lib64/libfontenc.so.1 (0x00007f10ef947000)

Try to start Xvfb

First, set the environment variables for keymap:

$ export XKB_BINDIR=/usr/bin
$ export XKBDIR=/var/task/lib

Fails with keymap error.

$ ./Xvfb :99
[dix] Could not init font path element /usr/share/fonts/X11/misc/, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/TTF/, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/OTF/, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1/, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/, removing from list!
XKB: Failed to compile keymap
Keyboard initialization failed. This could be a missing or incorrect setup of xkeyboard-config.

Things I'm not sure about

1. I assume the target machine needs XKBDIR and XKB_BINDIR, but since the machine in question doesn't have X11, it doesn't have xkbcomp, so does that mean I need to compile that from source also?
2. Font errors above, not sure about that.

// ADDED - Change xkbcomp bin directory with an environment variable
char *xkbBinDir = getenv("XKB_BINDIR");

// ADDED - Change base xkb directory with an environment variable
char *xkbBaseDir = getenv("XKBDIR");

You will have to supply these environment variables if you made those changes in xkbInit.c. The first is the location to the xkbcomp binary, and the second one overwrites the XkbBaseDirectory variable which is otherwise baked-in and unchangeable.

As for missing fonts, they aren't critical errors, but to silence them you can install those fonts and then copy them to your home folder on the server.

@ericdraken Yeah thats what I thought, will try to get xkbcomp binary working on the server and set those env's.

@ericdraken Tried setting the env variables (first trying to get it to run on the same container as I'm building it on). No luck. Note that the function mentioned in your original article in ddxLoad.c doesn't exist, and I edited the (similar looking) code at the location mentioned above. Wonder if that is a factor. Or it could be something else different between the lambda environment and the CentOS you were originally working with.

WIsh I had more experience with how to debug this stuff (even adding printf or something in the source code).

If you do get a chance to look at this any more I would love to send you an AMZ gift-card or something!

Note that I set the variables to:

bash-4.2# export XKBDIR=/var/task/lib
bash-4.2# export XKB_BINDIR=/usr/bin

I also get the keyboard initialization failure as above. What is the trick to getting the environmental variables set correctly so it will find the /var/task/lib/default.xkm file? Thanks!

@rubelw I never did work it out and have had to stick with my EC2 stack for now. If you know of anyone to forward this to (or want to tweet it), I'd still love to figure out a working solution!

Great work.. so close!

@cpsubrian what is your use case for xvfb in lambda? I was able to get Xvfb running in lambda but with some caveats.

@nisaacson I'm interested in hearing how you got Xvfb to work in lambda and the caveats. My use case is running OpenFrameworks code on lambda to do some image filtering.

@ericdraken Could you possibly post your portable binary in order to help speed along others through this process?

@RubenSandwich I put together a demonstration repo with the needed binaries and libraries. Check it out here https://github.com/nisaacson/aws-lambda-xvfb

Thats awesome @nisaacson! Missed your earlier question. My usecase is basically screenshotting websites via an electron app that I created. I wanted to run the electron app in lambda with a pretty high RAM config (so the screencap runs as fast as possible). Currently I run the electron app on an EC2 instance but its a huge waste of resources because it captures maybe 200 images a month but we pay for it 24/7 heh.

@cpsubrian take a look at this repo: https://github.com/dimkir/nightmare-lambda-tutorial

In case anyone else is wondering AWS now allows using custom docker containers.

So running Xvfb on lambda is as simple as this now:

FROM ubuntu:18.04
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install unzip wget php php-cgi xvfb \
 libcairo2-dev libjpeg-dev libpango1.0-dev libgif-dev build-essential g++ libgl1-mesa-dev libxi-dev \
 libx11-dev dbus-x11 pulseaudio curl udev

COPY entry.sh /
RUN chmod 755 /entry.sh

ENTRYPOINT [ "/entry.sh" ]
CMD [ "app.handler" ]

This could be your entry.sh

#!/bin/sh

while true
do
  HEADERS="$(mktemp)"
  # Get an event
  EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
  REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

  nohup Xvfb :99 -ac -screen 0 1024x768x24 -nolisten tcp &
  RESPONSE=$(ps auxwww)

  # Send the response
  curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response"  -d "$RESPONSE"
done

@san-kumar
Does this xvfb wrap any nodejs program? It doesn't look like it does.

@san-kumar - While Xfvb indeed starts running on a lambda environment, I'm finding that windows are never produced (i.e. running xwininfo -root -tree lists no child windows). However, an identical docker image ran anywhere else works fine, I see all the different windows of the applications I open as expected. Very strange and frustrating, I guess it has something to do with the restrictions of the lambda environment. Have you found the same or does your Ubuntu 16 image there work fine?