Headless VNC Server Setup Reference

headless-vnc.md

Headless VNC Server Setup Reference

AlmaLinux / RHEL 8+ — Single shared session via x11vnc on a dedicated virtual display

---

Architecture

xinit (~/.xinitrc + Xvfb :10) → x11vnc (port 5910)
     └── openbox-session (looping)
     └── your-app &

• Xvfb — virtual framebuffer (no physical display required)
• xinit — starts Xvfb and executes ~/.xinitrc in one shot
• Openbox — window manager, loops on exit to keep session alive
• x11vnc — shares the Xvfb display over VNC
• Port convention — display :10 maps to port 5910 (5900 + 10)

---

1. Install Dependencies

sudo dnf install -y epel-release
sudo dnf install -y x11vnc xorg-x11-server-Xvfb openbox xterm

xterm prevents Openbox pipe-menu errors on minimal installs. Alternatively, suppress the error with a custom menu.xml (see section 6).

---

2. User Account

Use a real user account (not a system account) so ~/.xinitrc and home directory work correctly:

sudo useradd -m vncuser
sudo passwd vncuser

---

3. Session Script /home/vncuser/start-session.sh

#!/bin/bash

export HOME=/home/vncuser

# xinit starts Xvfb :10 and executes ~/.xinitrc
xinit -- /usr/bin/Xvfb :10 -screen 0 1920x1080x24 -ac &

# Wait for display to be ready
sleep 1

export DISPLAY=:10

# x11vnc in foreground — keeps script alive
# When x11vnc exits, systemd restarts the whole stack
exec x11vnc \
  -display :10 \
  -forever \
  -shared \
  -nopw \
  -rfbport 5910 \
  -noxdamage \
  -repeat \
  -noipv6

sudo chown vncuser:vncuser /home/vncuser/start-session.sh
chmod +x /home/vncuser/start-session.sh

Key flags:

Flag	Purpose
-forever	Keep running after last client disconnects
-shared	All clients see the same session simultaneously
-nopw	No password (remove if auth is needed)
-noxdamage	Avoids rendering glitches with Xvfb

---

4. User Session /home/vncuser/.xinitrc

#!/bin/bash

# Launch your app
/path/to/your-app &

# Example: Launch xterm
xterm &

# Loop openbox — if user exits the session it restarts
# This keeps xinit alive and the VNC session persistent
while true; do
    openbox-session
    sleep 1
done

chmod +x /home/vncuser/.xinitrc

Critical: openbox-session must loop. If it exits without a loop, xinit exits, x11vnc loses its display, and the VNC session dies.

---

5. Systemd Service /etc/systemd/system/vncuser-session.service

[Unit]
Description=vncuser VNC Session
After=network.target

[Service]
Type=simple
User=vncuser
ExecStart=/home/vncuser/start-session.sh
ExecStop=/bin/kill -TERM $MAINPID
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable vncuser-session
sudo systemctl start vncuser-session

---

6. Openbox Menu (suppress pipe-menu errors)

On minimal installs without terminal emulators, create a static menu:

mkdir -p /home/vncuser/.config/openbox

/home/vncuser/.config/openbox/menu.xml:

<?xml version="1.0" encoding="UTF-8"?>
<openbox_menu xmlns="http://openbox.org/3.4/menu">
  <menu id="root-menu" label="Menu">
    <item label="Reconfigure">
      <action name="Reconfigure"/>
    </item>
    <item label="Exit">
      <action name="Exit"/>
    </item>
  </menu>
</openbox_menu>

chown -R vncuser:vncuser /home/vncuser/.config

---

7. Boot to Text Mode (no GUI on physical console)

# Set default target to text mode
sudo systemctl set-default multi-user.target

# Disable GDM (if installed)
sudo systemctl disable gdm.service
sudo systemctl stop gdm.service

# Verify
systemctl get-default
# -> multi-user.target

The vSphere/KVM console will show a plain login: prompt. The VNC session runs fully headless and independently.

---

8. Firewall and SELinux

# Open port 5910
sudo firewall-cmd --permanent --add-port=5910/tcp
sudo firewall-cmd --reload

# Allow custom VNC port in SELinux (Might not be need in AlmaLinux/RHEL)
sudo dnf install -y policycoreutils-python-utils
sudo semanage port -a -t vnc_port_t -p tcp 5910

---

9. Verification

# Service status
systemctl status vncuser-session

# Confirm x11vnc is listening
ss -tlnp 'src = :5910'

# Live logs
journalctl -u vncuser-session -f

---

10. Connecting

Direct (same network):

<server-ip>:5910

Via SSH tunnel (NAT/KVM):

ssh -L 5910:<vm-ip>:5910 user@<vm-ip> -N &
# Then connect VNC client to localhost:5910

---

File Summary

File	Owner	Purpose
/home/vncuser/start-session.sh	vncuser	Starts xinit + x11vnc
/home/vncuser/.xinitrc	vncuser	Defines X session (WM + app)
/home/vncuser/.config/openbox/menu.xml	vncuser	Static Openbox menu
/etc/systemd/system/vncuser-session.service	root	Systemd unit

---

KVM/libvirt Local Testing

# Create VM
virt-install \
  --name almalinux-vnc-test \
  --ram 2048 --vcpus 2 \
  --disk path=/var/lib/libvirt/images/almalinux-vnc-test.qcow2,size=20 \
  --os-variant almalinux9 \
  --cdrom ~/Downloads/almalinux9-minimal.iso \
  --network network=default \
  --graphics spice --video qxl \
  --console pty,target_type=serial

# Get VM IP
virsh domifaddr almalinux-vnc-test

# Simulate vSphere text console
virsh console almalinux-vnc-test   # Ctrl+] to exit

# SSH tunnel to reach x11vnc
ssh -L 5910:<vm-ip>:5910 vncuser@<vm-ip> -N &
vncviewer localhost:5910

---

Troubleshooting

Symptom	Fix
x11vnc can't connect to display	Increase sleep 1 in script to sleep 2
Openbox pipe-menu error	Install xterm or add custom menu.xml
VNC session dies when user exits Openbox	Ensure openbox-session is in a while true loop in .xinitrc
Port 5910 refused	Check firewall-cmd and semanage port
SELinux denials	Run ausearch -m avc -ts recent and use audit2allow
Black screen on connect	Check journalctl -u vncuser-session for Xvfb/xinit errors

[Unit]
Description=Headless VNC Session
After=network.target

[Service]
Type=simple
User=vncuser
Group=vncuser
WorkingDirectory=/home/vncuser
Environment=DISPLAY=:10
Environment=LC_ALL=C

# 1. xinit starts Xvfb securely, then runs ~/.xinitrc
ExecStart=/usr/bin/xinit /home/vncuser/.xinitrc -- /usr/bin/Xvfb :10 -screen 0 1920x1080x24 -nolisten tcp

# 2. Wait for the X11 socket to exist, THEN start x11vnc
ExecStartPost=/bin/sh -c 'while [ ! -e /tmp/.X11-unix/X10 ]; do sleep 0.1; done; /usr/bin/x11vnc -display :16 -forever -shared -localhost -rfbport 5916 -rfbauth /home/vncuser/.vnc/passwd -noxdamage -noxrecord -noxfixes -nosel -bg'

Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

#!/bin/bash
#/home/vncuser/.xinitrc
# Launch your application and capture its PID
/path/to/your-app &
APP_PID=$!

# Loop Openbox
while true; do
    openbox-session
    
    # If the main APP dies, break the loop to restart the session
    if ! kill -0 $APP_PID 2>/dev/null; then
        break
    fi
    sleep 1
done

[Unit]
Description=vncuser VNC Session
After=network.target

[Service]
Type=simple
User=vncuser
Group=vncuser
WorkingDir=/home/vncuser
Environment=HOME=/home/vncuser
Environment=DISPLAY=:1
Environment=LC_ALL=C
ExecStart=/usr/bin/xinit -- /usr/bin/Xvnc :1 -geometry 1920x1080 -depth 24 -rfbport 5900 -SecurityTypes None -AlwaysShared -ac -extension GLX +extension RENDER 
ExecStop=/bin/kill -TERM $MAINPID
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

https://www.wirelog.net/posts/2025-12-25-jetson-orin-nano-complete-headless-setup/