Test setsockopt with SO_PRIORITY and IP_TOS. It shows that order matters.

test_setsockopt.c

#include <assert.h>
#include <netinet/ip.h>
#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>


void test_setsockopt(int priority_first)
{
  int priority = 6;
  int iptos = IPTOS_CLASS_CS6;

  int fd = socket(AF_INET, SOCK_STREAM, 0);

  if (priority_first){
    if(setsockopt(fd, SOL_SOCKET, SO_PRIORITY, &priority,
                  sizeof(priority)) < 0){
      printf("Oh no\n");
    }
  }

  if(setsockopt(fd, IPPROTO_IP, IP_TOS, &iptos, sizeof(iptos)) < 0){
    printf("Oh no\n");
  }

  if (!priority_first){
    if(setsockopt(fd, SOL_SOCKET, SO_PRIORITY, &priority,
                  sizeof(priority)) < 0){
      printf("Oh no\n");
    }
  }

 int new_priority;
 int optlen = sizeof(new_priority);
 if (getsockopt(fd, SOL_SOCKET, SO_PRIORITY, &new_priority, &optlen) < 0){
   printf("Oh no\n");
 }
 if (priority_first){
   printf("Priority is %d if setsockopt(SO_PRIORITY) *before* setsockopt(IP_TOS)\n",
     new_priority);
 } else {
   printf("Priority is %d if setsockopt(SO_PRIORITY) *after* setsockopt(IP_TOS)\n",
     new_priority);
 }

 close(fd);
}

int main(void)
{
  // background:  http://lists.openwall.net/netdev/2009/12/21/59

  int priority_first = 0;
  test_setsockopt(priority_first);

  priority_first = 1;
  test_setsockopt(priority_first);

  return 0;
}

I found this specific example a bit misleading. The code above results in:

Priority is 6 if setsockopt(SO_PRIORITY) *after* setsockopt(IP_TOS)
Priority is 0 if setsockopt(SO_PRIORITY) *before* setsockopt(IP_TOS)

The zero priority in the "after" case implies that IP_TOS erases SO_PRIORITY but that isn't correct.

Setting IP_TOS puts both the IPv4 header TOS value into inet_sk(sk)->tos and the corresponding TC priority value into sk->sk_priority.

On Linux, IPTOS_CLASS_CS6 maps to TC_PRIO_BESTEFFORT which is priority 0, which is why the "after" case returns prio 0.

If you set IP_TOS to something else like IPTOS_DSCP_AF11 then the prio is changed accordingly, eg:

Priority is 6 if setsockopt(SO_PRIORITY) *after* setsockopt(IP_TOS)
Priority is 2 if setsockopt(SO_PRIORITY) *before* setsockopt(IP_TOS)

I think it would be more accurate to say:

• IP_TOS controls both the IPv4 TOS header and the Linux Traffic Control priority
• SO_PRIORITY just controls the TC priority

Setting SO_PRIORITY after also appears to allow setting the IPv4 header contents and TC behaviour separately, see ip_build_and_send_pkt() in current kernel.

The zero priority in the "after" case implies that IP_TOS erases SO_PRIORITY...

That's was my assumption.

I've got your point, thanks for the detailed explanation.

No worries! :) If you look into the kernel setsockopt for IP_TOS you'll see where it copies the values to the inet_sk(sk)->tos and sk->sk_priority fields. The corresponding TOS/TC values are stored in an array const __u8 ip_tos2prio[16].

So, I should set socket priority AFTER IP_TOS value ? That is good to know, thanks