Experiments in running Sunshine on Sway WM headless (virtual display) on demand (WOL request) on nvidia 565 drivers on Arch Linux.

README.md

Headless Sway Sunshine stuff

This setup allows you to start a single Sway remote desktop session on a virtual display directly from Moonlight by sending the Wake PC request, waiting a few seconds and connecting to a fresh new desktop. Or you can just have a desktop running constantly in the background. This requires no dummy plugs or input devices attached to the host, but will also allow you to sometimes log in locally and and have that session available on sunshine too. Some of this stuff probably also works with other wlroots-based compositors with small changes.

I'm assuming you've first read and memorized the Sunshine documentation, it probably includes some very important stuff I didn't put here. The stuff about udev rules is probably important, cap_sys_admin not so much.

Here's a simpler thing to just use on demand custom res virtual screen on a real sway desktop.

This is not supposed to be a complete guide to set up headless sunshine streaming and I assume you already know how to set up a sway desktop with hardware acceleration, but some (Arch) packages you probably also want are:

sway swaybg foot wmenu nvidia libvdpau libva-nvidia-driver cuda egl-wayland libva-utils nvtop pavucontrol pipewire-pulse steam gamescope

Obviously you'll need sunshine from the official repository, or maybe sunshine-git from AUR. The sunshine settings I have left completely as defaults, which probably makes it use wlroots for capture and nvenc for encoding.

Other stuff you probably need to do:

# no idea if all of these are needed
sudo usermod -a -G users,wheel,audio,video,input dregu
# start service on demand with WoL ping
sudo systemctl enable --now sway-sunshine.socket
# or if you don't want the WoL stuff
sudo systemctl enable --now sway-sunshine.service

Because the systemd service runs a login session in tty1 as a workaround to get input working without actually logging in, this is only meant for remote use. That said you can also log in to tty1 locally, run sway and connect to that session with moonlight, but only when the headless one is not running.

These silly input hacks are needed because there's currently no easier way to just assign the virtual input devices to a headless sway process. (Please correct me if I'm wrong, I'd really like to just use libseat or something properly.) Other hack that worked was setting up console autologin for the user running the sway service, or just running sway from tty1, but that's not very on demand and servery any more.

To top it off add a sunshine prep command /bin/bash /home/dregu/.config/sway/scripts/sway-sunshine.sh for Desktop to match the client resolution automatically.

Homework

• https://docs.lizardbyte.dev/projects/sunshine/latest/index.html
• https://wiki.archlinux.org/title/Sway
• https://wiki.archlinux.org/title/NVIDIA
• https://wiki.archlinux.org/title/Hardware_video_acceleration

sway-sunshine.conf

# /etc/sway/config.d/sway-sunshine.conf
# Of course you can also just add these to ~/.config/sway/config

# Minimizes latency when game has vsync off
output * allow_tearing yes max_render_time off

# Starts sunshine inside the sway session
exec sunshine

sway-sunshine.service

# /etc/systemd/system/sway-sunshine.service
# Runs sway in headless mode, unless sunshine is already running (maybe on a real display then)
[Unit]
Description=Sway Sunshine Headless

[Service]
User=dregu
Group=users
PAMName=login
TTYPath=/dev/tty1
WorkingDirectory=/home/dregu
Environment=WLR_RENDERER=vulkan
Environment=WLR_BACKENDS=libinput,headless
Environment=WLR_LIBINPUT_NO_DEVICES=1
Environment=XDG_CURRENT_DESKTOP=sway
Environment=XDG_SESSION_DESKTOP=sway
Environment=XDG_SESSION_CLASS=user
Environment=XDG_SESSION_TYPE=wayland
Environment=XDG_RUNTIME_DIR=/run/user/1000
ExecStartPre=/bin/sh -c '! "$@"' -- pidof -q sunshine
ExecStart=dbus-run-session sway --unsupported-gpu
#Restart=always

[Install]
WantedBy=multi-user.target

sway-sunshine.sh

#!/bin/bash
# /home/dregu/.config/sway/scripts/sway-sunshine.sh
# Sets output mode to match client settings
swaymsg "output HEADLESS-1 mode ${SUNSHINE_CLIENT_WIDTH}x${SUNSHINE_CLIENT_HEIGHT}@${SUNSHINE_CLIENT_FPS}Hz"

sway-sunshine.socket

# /etc/systemd/system/sway-sunshine.socket
# Listens for WakeOnLAN requests from Moonlight and starts the service if not running
[Socket]
ListenDatagram=9
Accept=no
FlushPending=yes

[Install]
WantedBy=sockets.target

First of all, thank you for your efforts to give me a good headless solution
When using a nvidia graphics card, set 'WLR_RENDERER' to 'vulkan'. There is a bug, sunshine may report the error 'Frame capture failed'。
Meanwhile, the sway log will show the following

Format XR24 can't be used with modifier BLOCK_LINEAR_2D,HEIGHT=4,KIND=8,GEN=2,SECTOR=1,COMPRESSION=1

The reason is that the NVIDIA Vulkan driver defaults to a compressed modifier buffer. The Screencopy implementation of the wlroots cannot correctly export/convert the modifier.
Try switching WLR_RENDERER to gles to fix this.

Current computer environment: I installed cuda(version: 13.2.0-1) in archlinux according to the requirements of sunshine guide, and nvidia uses the driver nvidia-open-driver-595.58.03. I don't clear it because I installed cuda caused sunshine to use nvidia dma-buf to capture the screen. The same problem occurs when I try to add wayland_capture_mode = pipewire to capture the screen using pipewire (maybe my use method is wrong).

NOTE
The above systemd script does not seem to output the sway log, and in the local session, login cannot test the headless because there is already a screen, in the ssh session, directly executing sway is blocked, it needs a prompt to ask for tty, and ssh uses ps/0 to simulate a terminal。
Try to circumvent it by using the following methods:

sudo machinectl shell <username>@ /bin/bash -c '
   export XDG_SESSION_TYPE=wayland
   export XDG_CURRENT_DESKTOP=sway
   export WLR_BACKENDS=headless,libinput
   export WLR_LIBINPUT_NO_DEVICES=1
   export WLR_RENDERER=vulkan
   export WLR_NO_HARDWARE_CURSORS=1
   export HOME=/home/philia093

  sway --unsupported-gpu -d 2>&1 | tee /tmp/sway-manual-test.log
'

This makes debugging easier.

I added the sway-sunshine.service environment variable WLR_RENDERER from vulkan to gless, and it showed the sway logs for some reason which I don't clear.):