Skip to content

Instantly share code, notes, and snippets.

@surki

surki/gist:4b1a3d8d40328712b6586e151a0813c5

Created September 26, 2017 12:19
Show Gist options
  • Save surki/4b1a3d8d40328712b6586e151a0813c5 to your computer and use it in GitHub Desktop.
Save surki/4b1a3d8d40328712b6586e151a0813c5 to your computer and use it in GitHub Desktop.
Download ZIP
Getting a socket's 'struct sock' information
Raw
gistfile1.txt
You can get even more detailed information about a socket from kernel's internal socket struct.
For example, we will try to get nginx listening (on port 81) socket's backlog length
NOTE: You may need to install kernel debug info if not already installed
# yum-config-manager --enable "amzn-main-debuginfo" --enable "amzn-updates-debuginfo"
# yum -y install kernel-debuginfo kernel-devel
Or get socket info for listening socket on port 81
# ss -len | grep :81
tcp LISTEN 0 511 *:81 *:* ino:29842919 sk:55 <->
Get its sk buff address
# grep 29842919 /proc/net/tcp
8: 00000000:0051 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 29842919 1 ffff8800e9d1da00 100 0 0 10 0
Now get the details, for example, the backlog length
# gdb /usr/lib/debug/lib/modules/`uname -r`/vmlinux /proc/kcore
................
................
Reading symbols from /usr/lib/debug/lib/modules/4.4.51-40.69.amzn1.x86_64/vmlinux...done.
[New process 1]
Core was generated by `root=LABEL=/ console=tty1 console=ttyS0 selinux=0 LANG=en_US.UTF-8 KEYTABLE=us'.
................
(gdb) set print pretty on
(gdb) p *(struct sock *)0xffff8800e9d1da00
................
................
sk_ack_backlog = 0,
sk_max_ack_backlog = 511,
................
................
@lafolle
Copy link

lafolle commented Sep 27, 2017

It seems kafka (and some other processes) are listening on ipv6 addresses (on ports 9092,9093 and 9094) (as you had noted), as shown by

lafolle@yoss:~/Downloads/confluent-3.3.0$ ss -lnt
State       Recv-Q Send-Q                                                        Local Address:Port                                                                       Peer Address:Port
LISTEN      0      128                                                                       *:25672                                                                                 *:*
LISTEN      0      80                                                                127.0.0.1:3306                                                                                  *:*
LISTEN      0      128                                                                       *:5355                                                                                  *:*
LISTEN      0      1024                                                              127.0.0.1:11211                                                                                 *:*
LISTEN      0      128                                                                       *:4369                                                                                  *:*
LISTEN      0      128                                                                       *:22                                                                                    *:*
LISTEN      0      5                                                                 127.0.0.1:631                                                                                   *:*
LISTEN      0      1024                                                                      *:15672                                                                                 *:*
LISTEN      0      100                                                               127.0.0.1:25                                                                                    *:*
LISTEN      0      50                                                                       :::9092                                                                                 :::*
LISTEN      0      50                                                                       :::9093                                                                                 :::*
LISTEN      0      50                                                                       :::2181                                                                                 :::*
LISTEN      0      50                                                                       :::9094                                                                                 :::*
LISTEN      0      50                                                                       :::26471                                                                                :::*
LISTEN      0      50                                                                       :::17639                                                                                :::*
LISTEN      0      128                                                                      :::5672                                                                                 :::*
LISTEN      0      128                                                                      :::5355                                                                                 :::*
LISTEN      0      511                                                                      :::80                                                                                   :::*
LISTEN      0      128                                                                      :::4369                                                                                 :::*
LISTEN      0      50                                                                       :::11027                                                                                :::*
LISTEN      0      128                                                                      :::22                                                                                   :::*
LISTEN      0      5                                                                       ::1:631                                                                                  :::*
LISTEN      0      100                                                                     ::1:25                                                                                   :::*
LISTEN      0      50                                                                       :::12447                                                                                :::*
LISTEN      0      128                                                                      :::12865                                                                                :::*

inode info for sockets on ipv6 are found in /proc/net/tcp6 as opposed to /proc/net/tcp.

lafolle@yoss:~/Downloads/confluent-3.3.0$ cat /proc/net/tcp6
  sl  local_address                         remote_address                        st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000000000000000000000000000:2384 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 133817 1 0000000000000000 100 0 0 10 0
   1: 00000000000000000000000000000000:2385 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 134614 1 0000000000000000 100 0 0 10 0
   2: 00000000000000000000000000000000:0885 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 134159 1 0000000000000000 100 0 0 10 0
   3: 00000000000000000000000000000000:2386 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 133980 1 0000000000000000 100 0 0 10 0
   4: 00000000000000000000000000000000:6767 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 133974 1 0000000000000000 100 0 0 10 0
   5: 00000000000000000000000000000000:44E7 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 134489 1 0000000000000000 100 0 0 10 0
   6: 00000000000000000000000000000000:1628 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000   132        0 27118 1 0000000000000000 100 0 0 10 0
   7: 00000000000000000000000000000000:14EB 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000   102        0 26182 1 0000000000000000 100 0 0 10 0
   8: 00000000000000000000000000000000:0050 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 24507 1 0000000000000000 100 0 0 10 0
   9: 00000000000000000000000000000000:1111 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000   132        0 29266 1 0000000000000000 100 0 0 10 0
  10: 00000000000000000000000000000000:2B13 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 134154 1 0000000000000000 100 0 0 10 0
  11: 00000000000000000000000000000000:0016 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 48605 1 0000000000000000 100 0 0 10 0
  12: 00000000000000000000000001000000:0277 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 68754 1 0000000000000000 100 0 0 10 0
  13: 00000000000000000000000001000000:0019 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 27378 1 0000000000000000 100 0 0 10 0
  14: 00000000000000000000000000000000:309F 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 133843 1 0000000000000000 100 0 0 10 0
  15: 00000000000000000000000000000000:3241 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 33243 1 0000000000000000 100 0 0 10 0
  16: 0000000000000000FFFF00000100007F:7158 0000000000000000FFFF00000101007F:2386 08 00000000:00000001 02:00022FB2 00000000  1000        0 133982 2 0000000000000000 20 4 28 10 -1
  17: 0000000000000000FFFF00000100007F:0885 0000000000000000FFFF00000100007F:7790 01 00000000:00000000 00:00000000 00000000  1000        0 135623 1 0000000000000000 20 4 31 10 -1
  18: 0000000000000000FFFF00000100007F:0885 0000000000000000FFFF00000100007F:7788 01 00000000:00000000 00:00000000 00000000  1000        0 130845 1 0000000000000000 20 4 31 10 -1
  19: 0000000000000000FFFF00000100007F:7788 0000000000000000FFFF00000100007F:0885 01 00000000:00000000 00:00000000 00000000  1000        0 132599 1 0000000000000000 20 4 30 10 -1
  20: 0000000000000000FFFF00000100007F:7790 0000000000000000FFFF00000100007F:0885 01 00000000:00000000 00:00000000 00000000  1000        0 130016 1 0000000000000000 20 4 30 10 -1
  21: 0000000000000000FFFF00000100007F:81D6 0000000000000000FFFF00000101007F:2385 08 00000000:00000001 02:00022C8A 00000000  1000        0 132624 2 0000000000000000 20 4 28 10 -1
  22: 0000000000000000FFFF00000100007F:573A 0000000000000000FFFF00000101007F:2384 08 00000000:00000001 02:00022B2E 00000000  1000        0 132603 2 0000000000000000 20 4 28 10 -1
  23: 0000000000000000FFFF00000100007F:0885 0000000000000000FFFF00000100007F:778C 01 00000000:00000000 00:00000000 00000000  1000        0 129993 1 0000000000000000 20 4 31 10 -1
  24: 0000000000000000FFFF00000100007F:778C 0000000000000000FFFF00000100007F:0885 01 00000000:00000000 00:00000000 00000000  1000        0 136261 1 0000000000000000 20 4 30 10 -1

Here the inode of socket in which we're interested in is present, but corresponding address (?) of socket is 0 (and hence socket can't be accessed using gdb.)

So, the problem, I think, boils down to "why socket address (?) is 0 for all active sockets in /proc/net/ipv6?".
But interestingly socket address is 0 also for all sockets in /proc/net/tcp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment