Tutorial for turnutils_uclient and Coturn server

turnutils_uclient.md

Tutorial for turnutils_uclient and Coturn server

When beginning to work with Turn servers, it's really useful to use a another tool to generate traffic to understand if the server is working correctly, and what kind of loads it can handle. turnutils_uclient is a tool built for that purpose. In this tutorial you will learn how to use turnutils_uclient, and be ready to do basic functional testing, simple load testing, and possibly rig up some monitoring methods, also. turnutils_uclient is packaged as part of the Coturn distribution.

About Coturn

Coturn is one of the major open source implementations for building TURN and STUN servers. There is also a another TURN and STUN server called Restund, but the focus is on Coturn here. From here on, TURN and STUN will be written as proper nouns: Turn and Stun.

Why is TURN necessary

In the world of VoIP, and WebRTC, the goal is to have user applications talk directly to each other, or "peer to peer" at least for media like audio and video, when exchanging that media. Often VoIP and WebRTC applications are able to do this, and generally about 60-80% of the time this is possible. But, given certain conditions, endpoints are not able to negotiate direct peer to peer media connectivity, and for them to successfully exchange media, they need an intermediary outside their firewalls in order to exchange media and other data. So the function of a Turn server is to relay media between two endpoints, or peers that find themselves in this situation, and roughly 20-40% of VoIP and WebRTC calls need a Turn server.

Suggested environment: clean VM running Ubuntu 16.04, 17.04 or 18.04

In order to keep this tutorial simple, everything will be executed on a single VM or server. At the conclusion of this tutorial, you will be ready to testing using turnutils_uclient on multiple discrete servers or VMs. This tutorial has been tested against 16.04, 17.04, and 18.04. I would recommend using 2GB RAM, Ubuntu 18.04 system or VM for this tutorial, but 16.04 or 17.04 should be fine too. The nice thing about 18.04 Ubuntu is that it currently has the latest Coturn release available directly as an apt-get package. I recommend you do not start with a laying around server or VM, or trying a version of Ubuntu older than 16.04, or another brand of Linux. If you start a clean, new, Ubuntu 16.04-18.04 virtual machine, this tutorial should go smoothly.

Don't leave an open TURN server running too long

It is important to stress you don't want to leave a non-authenticating Turn server running for long on the public Internet, as eventually bad actors will find it, and use your Turn server for their Turn server while racking up a big bandwidth bill for you. It's your responsibility to make sure you protect your servers and your bandwidth. The most important step is not to leave Coturn running as an open daemon or open for long periods of time. The examples here run the server in a way that requires authenication for Turn. You should substitute in your own unique username and password in the command examples that follow.

Substitute your own username & password for 'user' and 'pass' in the samples

Okay, this is repetitive, but you really ought to use your own username & password in the examples that follow.

Installing Coturn on Ubuntu

Login into your fresh install of Ubuntu, and run the following commands one by one as root in order to install Coturn.

# commands to install coturn, you must run these as root
# 'sudo bash' if you are not logged in as root to get your bash shell
sudo apt-get clean
sudo apt-get update
sudo apt-get -f install
sudo apt-get -y install coturn

The first three commands prepare for the install, and the last command actually does the install.

Make sure Coturn is not running as a service now and after reboot

After installing Coturn, it won't successfully run as a daemon without tweaking some files, but to be safe we will make sure it won't run in the background automatically. We will run the Coturn server directly from the command line for our testing.

Run these commands as root to disable Coturn from starting in the background.

# commands to make sure coturn doesn't run as a service
systemctl stop coturn
systemctl disable coturn

Commands from this point do not need to be run as root, but can be for testing.

Confirm what version you got

Quickly check the version installed.

# bash command
turnserver -h|grep Version
# expected output:
# On 18.04 we get:  Version Coturn-4.5.0.7 'dan Eider'

Adding a user to the database for testing

This step will add a user so we can do authenticated Turn testing, and we don't need to leave the server open to do the testing. Coturn needs a domain name to be associated with a user, it doesn't need to be anything real at this point, but a domain name must be provided to proceed with testing. I randomly chose 'dummy.com', but it could be anything.

# -a add/update a long-term user in the database
# -u <username>  username of user to add/update
# -p <password>  password of user to add/update
# -r <domain>  this is the domain name of for the user
# no printing on success, prints out on errors

sudo turnadmin -a -u user -p pass -r dummy.com

If it works fine, there will be no output.

Starting the coturn server from the command line

The main binary in coturn for Turn/Stun is called 'turnserver'. Go ahead and start it as like this:

# -v verbose
# --no-stun disable stun for safety
# -n ignore config file on disk
# -a require authentication
# -r <domain>  default domain to use for authentication attempts

turnserver --no-stun -a -n -r dummy.com

Turnserver will blast out a bunch of debug info about what it's doing, and will not return to the command prompt. Perfect!, you now have a running Turn server ready for testing. Keep in mind that turnserver does NOT automatically go away when you close your terminal window, you must find it with ps and kill it yourself when you close your terminal without using ^C to exit turnserver. We are running turnserver directly, and not as a service so that: 1) you can see any error messages right in the terminal without searching around, 2) you are hopefully, less likely to accidentally leave the server running for long periods of time during testing and learning. But please use ps and kill to find and end turnserver after experimenting with it.

Exercise 1, Using turnutils_uclient as two, or both endpoints

In real world Turn, you have two endpoints talking to the Turn server. So, if we wanted to duplicate that for our testing, we would have the turnserver process, and two other processes acting as the endpoints, instead we will use turnutils_uclient as both endpoints as a simple way to get started testing.

Run the following command.

#  127.0.0.1 address of the turn server
# -DgX      suggested packet and behavior defaults.
# -u & -p   username, password
# -e <addr> tell the server where to forward my packets to. [other endpoint]
# -n 1      send one packet per "client"
# -c        do not do RTCP
# -y        act as both the originating endpoint and the receiving endpoint

turnutils_uclient 127.0.0.1 -DgX -u user -w pass -e 127.0.0.1 -n 1 -c -y

If it worked okay, you should have seen two packets sent, and two received, with zero loss: : start_mclient: tot_send_msgs=2, tot_recv_msgs=2

Optional step: if you use Wireshark you can download and view a sample pcap of this step

Exercise 2, turnutils_peer is required when running without: -y

If you drop the '-y' argument, turnutils_uclient won't act as both endpoints, but just one endpoint, and you will need another program to act as the 2nd endpoint. But before showing you how to do it correctly, let me show you how to do it wrong so you can recognize this issue when it occurs.

My first use of turnutils_uclient went like this exercise, and I didn't understand what was going wrong, and it was quite frustrating.

Normally, without the -j flag you would run turnutils_peer, but please proceed without starting turnutils_peer, and then run the following command:

# removed the -y, so uclient will only act as a single endpoint not both.
turnutils_uclient 127.0.0.1 -DgX -u user -w pass -e 127.0.0.1 -n 1 -c

In this case, uclient will send a single packet and it will be lost, because it is only acting as a single endpoint, not both endpoints: : start_mclient: tot_send_msgs=1, tot_recv_msgs=0

In my experience it is important to run through this example, as not understanding the need for the '-y' flag wasted a fair bit of my time before I understood that turnutils_uclient will not act as both endpoints without the: -y.

Optional step: if you use Wireshark you can download and view a sample pcap of this step

Exercise 3, Using turnutils_uclient in conjunction with turnutils_peer

In this exercise,turnutils_uclient will act as a single endpoint in a Turn session, and turnutils_peer will act as a second endpoint, like when you have two endpoints in a real-life Turn session. You need to open a new terminal session on your system, and start the following command.

# -v        verbose, add if you want
turnutils_peer

It will not exit or return until you kill it, so switch away from this terminal to the terminal you used for running turnutils_uclient commands.

Now run the following command:

# removed the -y, so uclient will only act as a single endpoint not both.
turnutils_uclient 127.0.0.1 -DgX -u user -w pass -e 127.0.0.1 -n 1 -c

Now, uclient has sent a single packet to turnserver, which forwarded it to turnutils_peer, and so you should have seen the output: : start_mclient: tot_send_msgs=1, tot_recv_msgs=1. One message sent, one received.

Optional step: if you use Wireshark you can download and view a sample pcap of this step

Summarizing the -y flag with turnutils_uclient

So, in summary, when you run with the -y flag, you don't want or need turnutils_peer running. But, if you drop the -y flag, you need turnutils_peer running to respond as the 2nd endpoint.

Some things to try from here

Hopefully you have a good fundamental feel of a couple ways to use turnutils_uclient.

Here are some suggested ways to explore it further:

1. Start using multiple computers, and start using non-loopback interfaces, not: 127.0.0.1, and start using two (with -y) or three (without -y) systems in order to do testing across a LAN or WAN.
2. Increase -n N to test more than sending a single message
3. Increase -m M to test emulating more than 2 virtual clients
4. Remove the -c flag so you are creating RTCP clients & traffic

Some questions that will be explored in coming blog posts...

1. What does msz= represent in the output from uclient? It represents the number of virtual clients or pseudo endpoints still sending or waiting for packets. On larger N, M, you will see it count down near the end of the test-run. msz is affected by these flags: -c, -m, -y
2. How are msz and the total number of media packets to be sent calculated? MSZ calculation is bit complicated and it will be covered in another post, but the total number of packets is just numpack = msz * N
3. As I start to increase M, I am seeing "error 437 (Mismatched allocation: wrong transaction ID)", from turnutils_uclient, what is that all about? I will write a post about this, please search my site for 437, or join my mailing list to get notified.
4. I have see messages like "2528: Trying to bind fd 1241 to <127.0.0.1:56149>: bind: Address already in use" coming out of turnserver during testing, why is that? I will write a post about this, please search my site for errno=98, or join my mailing list to get notified.

Is there now a new way to get the version through command line that is reliable?

I observe that --version works with version 4.6.1:

$ turnserver --version
4.6.1

In "Exercise 1":

-u & -p username, password

The password short option in turnutils_uclient is -w.

This is all not really helpful because if your two peers are running on the same machine, you don't need TURN.

What is really needed is to: run two clients on two machines behind different NAT-ed subnets, and make the TURN server relay information between them.

But, as far as I understand, there is not such a testing tool.