TCP-notes

$TCP-notes.md

TCP notes

This gist contains all my note related to TCP.

But this file itself is only a placeholer. Nothing here.

FStack-YaStack.md

Fstack-YaStack

Comparison between FStack and YaStack.

A brief comparison

	Fstack	YAStack
Layers	DPDK + FreeBSD + POSIX api + Nginx //+ Redis	FStack - Nginx + Envoy
Event loop	Nginx's	libevent
Multi-core flow distribution	Hardware RSS (NIC's)	Software RSS--a layer of vswitch ( not much detail collected)
Lock	Removed locks of FreeBSD stack	same
Scheduling	Removed kernel, interupt and timer thread of FreeBSD stack	same
Other works	Ported netgraph, sysctl, ifconfig, netstat, IPfw, and pf	Adapted build system to cmake

Some details about FStack

• Developers of FStack had tested SeaStar. Their comments are following:

  1. It performed well in private network while bad in public internet.
  2. It can work with POSIX stack to solve the problem in 1), which means giving up DPDK and leaving NIC spare capcity.
  3. Althrough SeaStar has a simpler stack and more efficient memory management, its performance is similiar as FStack's in L7 app.

• In my opinion, the major contributions of FStack are

  1. porting FreeBSD stack to userspace and gluing it with DPDK.
  2. good usability. For nginx app, developers can directly port it to FStack by configing nginx. For other app, developer can port it to FStack by switching to POXIS api The other features are mainly from DPDK.

• A list of major modification of FreeBSD:

  • Scheduling：removed kernel, interupt and clock thread, sched, and sleep of FreeBSD stack
  • Lock：Removed locks in FreeBSD stack, including mtx, rw, rm, sx, and cond
  • Memory：Implemented phymem, uma_page_slab_hash, uma, kmem_malloc, and malloc
  • Global variable：pcpu, curthread, proc0, and thread
  • Env variable：Implemented setenv and getenv
  • SYS_INIT：mi_startup
  • Clock：Implemented timecounter, ticks, hz, and timer.
  • Other：
    • Switch errno between Linux and FreeBSD
    • Ported netgraph, ifconfig, sysctl, netstat for traffic monitoring
    • Ported IPfw and pf for firewall/security.
    • Removed unnecessary modules

Detail about YaStack

//TODO

---

Reference

• YaStack repo on github. https://github.com/saaras-io/yastack
• YaStack blog. https://medium.com/@chintan_
• FStack repo on github. https://github.com/F-Stack/f-stack
• A post from FStack developer(Chinese). https://www.zhihu.com/question/59779624
• A blog from FStack(Chinese). https://cloud.tencent.com/developer/article/1005179
• A blog analyzing FStack(Chinese). https://blog.csdn.net/ygm_linux/article/details/81589536

TCP-FastOpen.md

TCP Fast Open

This gist is about everything useful I found during researching TFO

Academic paper and Standards

//TODO

Use TFO in native linux

1. Check linux kernel version. It has to be higher than 3.7.x. uname -r can return the kernel version.

2. vim /etc/sysctl.conf to add TFO parameter to override default kernel parameter values set by sysctl.

       # add following line
       net.ipv4.tcp_fastopen = 3

3. sysctl -p to bring the modified config into effect

4. (Optional) If want to keep TFO option enabled after reboot. vim /etc/rc.local to add following line

       echo 3 > /proc/sys/net/ipv4/tcp_fastopen

5. Check TFO connection stats.

       # list all tcp connection
       ip tcp_metrics
   
       # monitor tfo stats
       grep '^TcpExt:' /proc/net/netstat | cut -d ' ' -f 87-92 | column -t

Use TFO in a userspace FreeBSD stack implementation--Fstack

//TODO