Skip to content

Instantly share code, notes, and snippets.

@tovrstra

tovrstra/online_teaching_howto.md

Last active September 19, 2024 04:22
Show Gist options
  • Save tovrstra/cd1a631584d5619d4198d1625c0b085b to your computer and use it in GitHub Desktop.
Save tovrstra/cd1a631584d5619d4198d1625c0b085b to your computer and use it in GitHub Desktop.
Download ZIP
Share iPad/IPhone/Android screen in all sorts of ways...
Raw
online_teaching_howto.md

Turn any Linux laptop into a wireless access point

When your institution's WiFi is flaky or overloaded by students voting on your polls, you can set up your own WiFi access point as follows on your Linux laptop for your tablet (in case your tablet does not support USB tethering, i.e. you're stuck with an iPad):

nmcli d wifi hotspot ifname <wifi_device> ssid <YOURSSID> band a password <your_pass>

You need to fill in <wifi_device> with the correct device name, which can be obtained with iwconfig. This is usually a name starting with wlp. You are free to fill in any name and password for <YOURSSID> and <your_pass>, respectively. On modern Linux distributions, the access point can also be set-up through the graphical interface of NetworkManager.

When your hardware and software supports it, this will set up a 5GHz access point, which tends to interfere less with traditional WiFi routers than a more common 2.4GHz access point. Either way, both work. This also configures your laptop to function as a NAT router and a DHCP server for any device connecting to the access point.

To make your ngingx server available to the tablet connected to this AP, you need may need to configure your firewall. On fedora, the following needs to be entered (once):

firewall-cmd --add-port=1025-65535/tcp --permanent --zone=nm-shared
firewall-cmd --add-port=1025-65535/udp --permanent --zone=nm-shared
firewall-cmd --reload

See also bug for ipad: https://bugzilla.redhat.com/show_bug.cgi?id=2057352

Mirror mobile screen

RPiPlay & iOS

RPiPlay is an open-source AirPlay server program. Compile and start this program. The follow the standard method to share your screen on your iOS device. This should be fairly easy to set up. In case of trouble, (i) make sure both devices are on the same network and (ii) that the avahi daemon is running on your laptop.

Strengths

  • Stable connection, even when your mobile device goes into power save mode and back.
  • Practically no delay on the stream.

Limitations

  • No control over video quality. The settings seem to be hardwired by your iOS device.
  • No sound.
  • iOS only.

ScrCpy

I have not tested it yet: https://github.com/Genymobile/scrcpy

Strengths

  • Two-way communication between mobile and laptop
  • ...

Limitations

  • Android only
  • ...

Streamlabs + FFplay

This section explains how to show your iPad/IPhone/Android screen on a desktop/laptop by running FFplay as an RTMP server on your laptop and streaming to it with "Streamlabs: Live Streaming App" on your mobile device.

Strengths

  • Only cross-platform tools are needed, which should make this a universal solution. (I did not test it for all possible combinations.)
  • Fairly good control over audio and video quality.

Limitations

  • Streamlabs does not provide a stable connection on the iPad, i.e. regularly drops connection for no reason.
  • Small but noticeable delay on the stream.

Installation

On the desktop or laptop machine

  1. Use a fixed (local) IP address. This is not mandatory, but it makes life easier when you need to enter the address of your RTMP server later.
  2. Install FFmpeg:
    • Fedora: 
      # Activate RPM Fusion Repository, needed for the installation of ffmpeg
# See https://rpmfusion.org/
sudo dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf install ffmpeg
    • Ubuntu: 
      sudo apt install ffmpeg
    • Windows & macOS: download from https://ffmpeg.zeranoe.com/builds/ and run the installer.

On your mobile device:

Install the Streamlabs mobile app. This software can record your screen (or camera) and send the video feed to a custom RTMP server (or to other live streaming platforms).

Usage

  1. Run FFplay as an RTMP server on your laptop. As soon as it picks up a video+audio stream from your mobile device, it will appear.

    • Linux
      • To show the complete screen of your mobile device: 
        ffplay -fflags nobuffer rtmp://<ip_address>/live/test -listen 1
      • To show only a part of the screen: 
        ffplay -fflags nobuffer -vf "crop=1440:940:240:124" rtmp://<ip_address>/live/test -listen 1
    • Windows: TODO
    • macOS: TODO

    See FFmpeg documentation for more details:

  2. Configure the Streamlabs App with the following "Streaming platform": Custom RTMP server

    • Address: rtmp://<ip_address>/live where live matches the argument given to ffplay in the previous step.
    • Stream key: test, which also just matches the argument given to ffplay in the previous step.

  3. Start broadcasting

    • General: change the following audio & video settings in the Streamlabs App (tuned for sharing slides with annotations):
      • Audio: low quality (unless you need it)
      • Broadcast (video): maximal resolution and low framerate (15 fps).
    • iPad/iPhone:
    • Android:
      • Use the "Editor" feature to configure the layout of the video feed (camera, screencast, ...)
      • Press the red record button.

Streaming video and audio into an app (virtual microphone and webcam) on Linux

This is used by some sections below. Setting this up can be tricky. You've been warned.

Installation and configuration (video)

Install the v4l2loopback module: https://github.com/umlaeute/v4l2loopback. The installation from source works by simply following the steps in the README instructions. This module makes it possible to define virtual video devices. In short:

  • On Fedora, manual approach without DKMS (requires rebuild after every kernel update).

    sudo dnf install gcc kernel-devel git
git clone [email protected]:umlaeute/v4l2loopback.git
cd v4l2loopback
make && sudo make install && sudo depmod -a

  • On Ubuntu, just install the package

    sudo apt install v4l2loopback-dkms

  • On any Linux system with DKMS support, e.g. on Fedora, which auto-recompiles the module through kernel updates.

    sudo dnf install gcc kernel-devel git dkms
sudo su -
cd /usr/src/
# You can also use more recent versions
wget https://github.com/umlaeute/v4l2loopback/archive/v0.12.5.tar.gz
tar -xvzf v0.12.5.tar.gz && rm v0.12.5.tar.gz
dkms add -m v4l2loopback -v 0.12.5
dkms build -m v4l2loopback -v 0.12.5
dkms install -m v4l2loopback -v 0.12.5

Finally, on any Linux system, create the following files and afterwards rebuild the initrd image (on Fedora with sudo dracut -f. On ubuntu, use sudo update-initramfs), to make sure the module is loaded at boot time:

  • /etc/modules-load.d/v4l2loopback.conf to auto-load on boot: 
    v4l2loopback
  • /etc/modprobe.d/v4l2loopback.conf to set default options: 
    options v4l2loopback exclusive_caps=1
options v4l2loopback devices=2
options v4l2loopback video_nr=7,8
options v4l2loopback card_label="tablet,zoom"
    The option exclusive_caps=1 is needed for compatibility with some programs like Google chrome. The second line requests two virtual devices. This can be useful to pipe and incomming RTMP stream into a virtual webcam, which improve stability of the stream in OBS. The second virtual webcam can be used to pip the output of OBS into the presenter screen in Zoom. Remaining lines are used to control device numbers and names. Note that each option requires a new line in the config file. You cannot put two options on one line.

To avoid rebooting for first usage, load the module as follows:

sudo modprobe v4l2loopback

Verify that the module is loaded correctly:

cat /sys/module/v4l2loopback/parameters/exclusive_caps

This should print at least one Y

Installation and configuration (audio)

You must have the commands pactl and pacmd installed. These may already be present, if not:

  • Fedora

    sudo dnf install pulseaudio-utils

  • Ubuntu

    sudo apt install pulseaudio-utils

Put the following in the PulseAudio config file, ~/.config/pulse/default.pa. This gets loaded when you login to your laptop:

# include the default.pa pulseaudio config file
.include /etc/pulse/default.pa

# A virtual output device which can be piped to the virtual microphone.
.ifexists module-null-sink.so
load-module module-null-sink sink_name=VirtualSink sink_properties=device.description="VirtualSink"
.endif

# A virtual microphone, which "hears" the virtual sink defined above.
.ifexists module-virtual-source.so
load-module module-virtual-source source_name=VirtualMic master=VirtualSink.monitor
.endif

# Monitor the sound sent to VirtualSink on my audio output. This only makes
# sense with headphones.
# To find the right sink: pactl list short sinks
# To disable after login: pactl unload-module module-loopback
.ifexists module-loopback.so
load-module module-loopback source=VirtualSink.monitor sink=alsa_output.pci-0000_00_1f.3.analog-stereo latency_msec=1
.endif

Testing (video)

Install the package v4l-utils and ffmpeg with apt (Ubuntu) or dnf (Fedora).

To find out which device is the virtual webcam, run:

v4l2-ctl --list-devices

Alternatively, you can also check which directories are listed as virtual video devices:

ls /sys/devices/virtual/video4linux

Assuming it is /dev/video0 and you are already streaming to it (examples below), test the virtual webcam with ffplay as follows

ffplay /dev/video0

Remarks:

  • v4l2loopback cannot handle changes in stream resolution. It will assume the resolution remains the same as in the initial stream. When you change resolutions, the video vrom the virtual webcam looks like garbage.

  • Streaming into some apps like Zoom and MS Teams may not be ideal for slides because these tend to crop the video stream in rather unpredictable ways.

Testing (audio)

Try with Zoom, Skype, ... in comination with a video player to configure the correct virtual output (on the player) and input (on the video call). To understand what is going on, pavucontrol is quite useful.

Stream OBS output into a virtual webcam (on Linux)

You need to setup v4l2loopback first, see section above.

Installation

For now, the from source installation is the best way to go. This should become a lot easier with future release of OBS Studio.

First install dependencies:

  • Fedora 33:

    sudo dnf install \
    make gcc gcc-c++ gcc-objc cmake git libX11-devel mesa-libGL-devel \
    libv4l-devel pulseaudio-libs-devel speex-devel x264-devel \
    freetype-devel fontconfig-devel libXcomposite-devel libXinerama-devel \
    qt5-qtbase-devel qt5-qtx11extras-devel qt5-qtsvg-devel libcurl-devel \
    systemd-devel ffmpeg ffmpeg-devel luajit-devel python3-devel mbedtls \
    mbedtls-devel swig speexdsp-devel wayland-devel qt5-qtbase-private-devel

  • On Ubuntu: See https://github.com/obsproject/obs-studio/wiki/Install-Instructions#debian-based-build-directions

Then compile and install as follows:

cd ~/build
git clone --recursive https://github.com/obsproject/obs-studio.git
cd obs-studio
mkdir build && cd build
export OBS_VERSION=$(git describe)
export OBS_PREFIX=${HOME}/Software/install/obs-studio/${OBS_VERSION}
cmake -DUNIX_STRUCTURE=1 -DCMAKE_INSTALL_PREFIX=${OBS_PREFIX} ..
make -j4
make install

cd ~/build
git clone https://github.com/CatxFish/obs-v4l2sink.git
cd obs-v4l2sink
mkdir build && cd build
cmake -DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" -DCMAKE_INSTALL_PREFIX=${OBS_PREFIX} .. 
make -j
make install

# This is the script I use to start OBS Studio
cd ${OBS_PREFIX}
cat start-obs.sh
#!/usr/bin/env bash
SCRIPT_PATH=$(dirname $0)
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${SCRIPT_PATH}/lib:${SCRIPT_PATH}/lib64
${SCRIPT_PATH}/bin/obs

Usage

  • Start OBS Studio and configure your scene. In the Tools menu, select V4L2 Video Output. Enter the video device path and click Start.
  • Start a program which can make use of a webcam, e.g. MS Teams, Zoom, Google meet, etc. Go to device (or video) settings and select the Dummy video device (in case you did not use another name). You should see the scene from OBS in the preview.

(Streamlabs + nginx RTMP server + RTMP client)

This section describes a more advanced setup, which forwards your screencast to an RTMP server, to which multiple RTMP clients can be connected. Two useful RTMP clients are discussed: FFplay and OBS Studio. It is assumed you have OBS Installed, see section above.

I'm keeping this section in case it could be useful to anyone. I stopped using this approach because the Streamlabs app is not providing a stable connection on iOS.

Installation and configuration

This configuration runs in userspace and requires no root privileges to install or run the RTMP server on your Linux machine. The installation of dependencies is done through DNF, which does require root privileges for just that step.

  1. Configure your Linux to have fixed (local) IP address. This is not mandatory, but it makes life easier when you need to enter the address of your RTMP server later.

  2. Install dependencies on Linux:

    • Fedora 31: 
      sudo dnf groupinstall -y 'Development Tools'
sudo dnf install pcre-devel zlib-devel openssl-devel
    • Ubuntu: 
      # TODO

  3. Download and unpack the latest version of NGINX and its RTMP module in some work directory:

    wget https://nginx.org/download/nginx-1.19.0.tar.gz
tar -xvzf nginx-1.19.0.tar.gz
git clone [email protected]:arut/nginx-rtmp-module.git

  4. Select an installation path:

    export NGINX_PREFIX=/home/toon/Software/install/nginx/1.19.0

  5. Compile and install NGINX

    cd nginx-1.19.0
./configure  --add-module=../nginx-rtmp-module --prefix=${NGINX_PREFIX} --with-cc-opt="-Wno-error"
make -j
make install

  6. Put the following in your NGINX config file (${NGINX_PREFIX}/conf/nginx.conf):

    daemon off;
error_log /dev/stdout info;
worker_processes 1;
events {
    worker_connections  1024;
}
rtmp {
    server {
        listen 1935;
        application tablet {
            # Live mode must be on to allow ffmpeg to read the stream the video to a virtual webcam.
            # The virtual webcam is better supported in OBS.
            live on;
            record off;
            # You may have to correct the /dev/video* number.
            exec_push ffmpeg -i rtmp://localhost/$app/$name -fflags nobuffer -c:v rawvideo -pix_fmt yuv420p -f v4l2 /dev/video7 2>>/var/tmp/v4l2loopback-$name.log;
        }
    }
}

    This will tell NGINX to enable the RTMP module for live streaming and run NGINX in the foreground. The latter is convenient for temporary usage, e.g. during a lecture.

  7. Install the Streamlabs App on your iPad. This software can record your screen (or camera) and send the video feed to a custom RTMP server (or to other live streaming platforms).

Usage

  1. Start he NGINX server: 
    ${NGINX_PREFIX}/sbin/nginx
  2. Configure the Streamlabs App with the following settings (ideal for streaming annotations on slides in a tool like GoodNotes or Notability):
    • Streaming platform: Custom RTMP server
      • Address: rtmp://your_ip_or_host_name/tablet where tablet matches the name of the first application section in the NGINX config file above.
      • Stream key: test (or anything you like)
    • Audio: low quality (unless you need it)
    • Broadcast (video): maximal resolution (1080), low framerate (15 fps).
  3. Start broadcasting with the recording feature of your iPad, see https://support.apple.com/en-us/HT207935. Press the record button long enough and you will see the option to broadcast with Streamlabs.
  4. Show the video stream on your Linux client with ffplay. The following example crops away the black borders (added by Streamlabs unfortunately) and the header of the GoodNotes GUI. 
    ffplay -fflags nobuffer -vf "crop=1440:956:240:124" rtmp://localhost/tablet/test
    The same can be done with the virtual webcam (no sound): 
    ffplay -fflags nobuffer -vf "crop=1440:956:240:124" /dev/video7
    See FFmpeg documentation for more details:
  5. Stop ffplay with CTRL+C and start OBS Studio instead. Add a source and select "Media Source". Unselect "Local" file and enter the RTMP URL. Alternatively, you can also add a webcam and select the virtual webcam, which tends to be more robust (e.g. keeps working when network connections drop temporarily.)
  6. Stop ffplay with CTRL+C. Start a video-chat application in your browser (or a client like Zoom, skype) and select the virtual webcam to stream your tablet through such application. Zoom has a special feature to present from a second webcam, which can be useful. The video device can be either the one from your tablet, or the one with the output from OBS.

Take a snapshot from an RTMP stream

ffmpeg -i rtmp://localhost/tablet/les -vframes 1 output.jpg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment