TCP connection states and netstat output

tcp_connection_state.gist

TCP Connection States

Following is a brief explanation of this handshake. In this context the "client" is the peer requesting a connection and the "server" is the peer accepting a connection. Note that this notation does not reflect Client/Server relationships as an architectural principal. 

Connection Establishment

The client sends a SYN message which contains the server's port and the client's Initial Sequence Number (ISN) to the server (active open).
The server sends back its own SYN and ACK (which consists of the client's ISN + 1).
The Client sends an ACK (which consists of the server's ISN + 1).

Connection Tear-down (modified three way handshake).

The client sends a FIN (active close). This is a now a half-closed connection. The client no longer sends data, but is still able to receive data from the server. Upon receiving this FIN, the server enters a passive close state.
The server sends an ACK (which is the clients FIN sequence + 1)
The server sends its own FIN.
The client sends an ACK (which is server's FIN sequence + 1). Upon receiving this ACK, the server closes the connection.

A half-closed connection can be used to terminate sending data while sill receiving data. Socket applications can call shutdown with the second argument set to 1 to enter this state.


State explanations as shown in Netstat:
State Explanation
------------ --------------------------------------------------------

SYN_SEND Indicates active open.

SYN_RECEIVED Server just received SYN from the client.

ESTABLISHED Client received server's SYN and session is established.

LISTEN Server is ready to accept connection.

FIN_WAIT_1 Indicates active close.

TIMED_WAIT Client enters this state after active close.

CLOSE_WAIT Indicates passive close. Server just received first FIN from a client.

FIN_WAIT_2 Client just received acknowledgment of its first FIN from the server.

LAST_ACK Server is in this state when it sends its own FIN.

CLOSED Server received ACK from client and connection is closed.
As an example, consider the following scenario: 

A socket application has been terminated, but Netstat reports the socket in a CLOSE_WAIT state. This could indicate that the client properly closed the connection (FIN has been sent), but the server still has its socket open. This could be the result of one instance (among all threads or processes) of the socket not being closed. 

NOTE: It is normal to have a socket in the TIME_WAIT state for a long period of time. The time is specified in RFC793 as twice the Maximum Segment Lifetime (MSL). MSL is specified to be 2 minutes. So, a socket could be in a TIME_WAIT state for as long as 4 minutes. Some systems implement different values (less than 2 minutes) for the MSL. 

Difference between close_wait and time_wait
If we take snapshot of netstat (netstat -nP tcp) the common states we see would be   ESTABLISHED,  TIME_WAIT,  CLOSE_WAIT. 
I will try explaining what they mean. 

Before we go into TIME_WAIT and   CLOSE_WAIT, lets take close look at sequence of steps for socket closing. 
Socket connection is essentially between two peers (Browser to webserver, a java client to webserver, webserver to DB server, a webserver to another webserver etc ) 

Say there is a socket connection established between webserver1 and webserver2. This would be the closing sequence, once the data transfer is done: (From TCP sequence diagram) 

Here I am assuming webserver1 initiates the close of connection. 
1)   Socket on webserver1 sends a TCP segment with FIN bit (in TCP header) and the socket goes into FIN_WAIT_1 state. 

2)  Socket on webserver2 receives the FIN and responds back with ACK to acknowledge the FIN and the socket goes to CLOSE_WAIT state. 
Now until the application calls the close() on this socket this is going to be in CLOSE_WAIT state. 

3)  Socket on webserver1 receives the ACK and changes to FIN_WAIT_2 

4)  Socket on webserver2 closes the connection(once the application calls close()) and sends back FIN to its peer to close the connection and changes its state to Last Ack 

5)  Socket on webserver1 receives the FIN and sends back ACK. 
At this point the socket implementation on webserver1 would start a timer (TIME_WAIT) to handle the scenario where last ACK has been lost and server resends FIN. 
Now the socket would wait for 2* MSL (Maximum segment lifetime- default is 4mins for solaris & windows) 

6)  Socket on webserver2 receives the ACK and it moves the connection to closed state 

7)  After TIME_WAIT is elapsed socket/connection will be closed on webserver1. 

These multiple levels of acknowledgments & retransmits are needed since TCP is a reliable protocol unlike basic UDP 
Here is what the three states mean: 

ESTABLISHED: 
This is pretty explanatory which basically means the two ends are in a state where data transfer can occur or occurring in both directions. (tcp socket is full duplex, i.e data can be received and responded to on same channel) 

CLOSE_WAIT: 
This is a state where socket is waiting for the application to execute close() 
CLOSE_WAIT is not something that can be configured where as TIME_WAIT can be set through tcp_time_wait_interval (The attribute tcp_close_wait_interval has nothing to do with close_wait state and this was renamed to tcp_time_wait_interval starting from Solaris 7)
A socket can be in CLOSE_WAIT state indefinitely until the application closes it. 
Faulty scenarios would be like filedescriptor leak, server not being execute close() on socket leading to pile up of close_wait sockets. (At java level, this manifests as "Too many open files" error)