Skip to content

Instantly share code, notes, and snippets.

@patrickjaja

patrickjaja/VSeeFace, OBs and MS Teams [Arch Linux Guide].md

Created October 27, 2023 11:46
Show Gist options
  • Save patrickjaja/45e3c955a80e4a3052db66fa219d6d69 to your computer and use it in GitHub Desktop.
Save patrickjaja/45e3c955a80e4a3052db66fa219d6d69 to your computer and use it in GitHub Desktop.
Download ZIP
Arch Linux: How to Use a Virtual 3D Avatar in MS Teams
Raw
VSeeFace, OBs and MS Teams [Arch Linux Guide].md

Arch Linux: How to Use a Virtual 3D Avatar in MS Teams

Archievement:

This tutorial will show you how to use a virtual 3D avatar with eye, mouth, and body tracking in MS Teams on Arch Linux.

Requirements:

  • Wine
  • VSeeFace (from Lutris)
  • OpenSeeFace
  • Iriun Webcam
  • Two v4l2loopback devices

Instructions:

  1. Install and start Wine. ( yay v4l2loopback-dkms v4l2loopback-utils lutris iriun obs winetricks wine)
  2. Install VSeeFace from Lutris. (https://lutris.net/games/vseeface/ , in lutris use the button start "Winetricks" and install arial font)
  3. Install OpenSeeFace:

cd ~ && git clone https://github.com/emilianavt/OpenSeeFace && cd OpenSeeFace && virtualenv -p python3 venv && source venv/bin/activate && pip install onnxruntime opencv-python pillow numpy

  1. Adjust /etc/modprobe.d/iriunwebcam-options.conf to ensure you have two virtual cams:
options v4l2loopback video_nr=8,9
options v4l2loopback exclusive_caps=1 card_label="OBS Virtualcam,Iriun Webcam"
  1. Reboot your computer.
  2. Connect your phone to your PC using Iriun Webcam.
  3. Start face and eye tracking using the OpenSeeFace tracking script:

cd ~/OpenSeeFace source venv/bin/active python facetracker.py -c 9 -W 1280 -H 720 --discard-after 0 --scan-every 0 --no-3d-adapt 1 --max-feature-updates 900 --ip 127.0.0.1 --port 11573

  1. In Lutris, start VSeeFace and select [OpenSeeFace tracking] as the camera. Load and start your .vrm avatar. (https://booth.pm/en/search/free%20vrm)
  2. Start OBS and its virtual camera by clicking Start Virtual Camera.
  3. To add your avatar to OBS, add a window capture and select VSeeFace Window.
  4. Add a Color Key Filter to the VSeeFace window to make the background transparent.

Troubleshooting:

Check if you setup v4l2 devices correctly:

v4l2-ctl --list-devices

Tips:

  • You can adjust the settings in the OpenSeeFace tracking script to improve the performance and accuracy of the tracking.
  • You can also adjust the settings in VSeeFace to customize the appearance of your avatar.
  • To use your virtual avatar in MS Teams, simply select the OBS Virtualcam as your camera when joining a meeting.

I hope this tutorial is helpful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment