VMware ESXi and vSphere Cluster Management
Transmission Control Protocol (TCP)
Learn how TCP provides reliable, ordered, connection-oriented communication with handshakes, sequence numbers, windows, flags, termination, and troubleshooting.
Transmission Control Protocol (TCP) is a transport-layer protocol used when applications need a reliable, ordered connection between two endpoints. Web browsing, secure shell sessions, email transfer, file transfer, and many database connections commonly use TCP.
This lesson explains how TCP fits into networking, establishes connections, transfers data, controls flow and congestion, closes connections, and can be observed with operating-system tools and packet captures.
TCP's purpose and position in networking
TCP is a core protocol in the TCP/IP suite, the family of protocols used to communicate across interconnected networks. In the OSI model, TCP operates at Layer 4, the Transport layer.
The Transport layer provides communication between application processes. An IP address identifies a host, while a port number identifies an application endpoint on that host. TCP therefore extends host-to-host IP delivery into process-to-process delivery.
- IP: Delivers packets between hosts identified by IP addresses.
- TCP: Delivers a reliable byte stream between application endpoints identified by IP addresses and ports.
- Application protocol: Defines the meaning of the data, such as HTTP requests or SSH messages.
TCP data is encapsulated as it moves down the protocol stack. Application data is placed in a TCP segment. The segment is carried inside an IP packet, and the IP packet is carried inside a link-layer frame such as an Ethernet or Wi-Fi frame.
Application data
└── TCP header + application data = TCP segment
└── IP header + TCP segment = IP packet
└── Link-layer header/trailer + IP packet = frameTCP does not choose routes between networks; IP and lower layers perform that work. TCP manages the conversation between the applications at the two ends.
Connection-oriented communication
TCP is connection-oriented. Before normal application data is exchanged, both endpoints create and synchronize protocol state. This state includes sequence-number information, receive-window information, and negotiated options.
Usually, a server performs a passive open by listening on a port. A client performs an active open by requesting a connection to that listening service. After establishment, the distinction is mainly about how the connection began: both endpoints can send and receive data.
Sockets and the TCP four-tuple
A socket is an endpoint commonly identified by an IP address and port. An established TCP connection is identified by a four-tuple:
source IP address, source port, destination IP address, destination portFor example:
Client: 192.0.2.10:49152
Server: 192.0.2.20:443
Four-tuple:
192.0.2.10, 49152, 192.0.2.20, 443The client normally selects an ephemeral port, a temporary port used for an outbound connection. The server commonly listens on a well-known or configured service port.
A listening socket is not the same thing as an established connection. A listening service might be represented as 192.0.2.20:443, without a particular client peer. Each accepted client creates a separate connection with its own four-tuple.
The three-way handshake
The normal TCP setup sequence is called the three-way handshake:
- The client sends a segment with the SYN flag set. It proposes an initial sequence number.
- The server replies with SYN and ACK. It proposes its own initial sequence number and acknowledges the client's SYN.
- The client sends an ACK acknowledging the server's SYN. The connection can now carry application data.
Client Server
| |
| --- SYN, sequence = client_isn --------> |
| |
| <--- SYN-ACK, sequence = server_isn, ----- |
| acknowledgment = client_isn + 1 |
| |
| --- ACK, acknowledgment = server_isn + 1 ->|
| |
| ESTABLISHED on both endpoints |An initial sequence number (ISN) begins the sequence-number space for one direction. The client and server have independent sequence spaces. A SYN consumes one sequence-number position, so the acknowledgment for a SYN is its sequence number plus one.
Important setup states
- LISTEN: A server is waiting for incoming connection requests.
- SYN-SENT: An endpoint has sent a SYN and is waiting for a response.
- SYN-RECEIVED: An endpoint has received a SYN, sent a SYN-ACK, and is waiting for the final ACK.
- ESTABLISHED: Both sides have synchronized state and can exchange application data.
The handshake confirms that packets can travel in both directions and synchronizes sequence-number spaces. It also allows endpoints to negotiate options such as maximum segment size and window scaling.
Common handshake outcomes
- Successful establishment: SYN, SYN-ACK, and ACK are exchanged.
- Closed port: A reachable host commonly responds to a SYN with RST, indicating that no connection is accepted at that endpoint.
- Timeout: A firewall may silently drop the SYN or reply, or packets may be lost. The sender retransmits the SYN and eventually gives up.
Reliable and ordered byte-stream delivery
TCP numbers bytes, not application messages. The sequence number identifies the position of the first payload byte in a segment. The acknowledgment number identifies the next byte the receiver expects.
For example, if a receiver has received every byte through byte 4,999, it can send acknowledgment number 5,000. This is a cumulative acknowledgment: it confirms all earlier bytes in sequence, not just one isolated segment.
Out-of-order data
IP packets can take different paths or experience different delays. A later TCP segment can arrive before an earlier one. TCP retains out-of-order data and reorders it before presenting the continuous byte stream to the application.
Suppose a sender transmits these byte ranges:
Segment 1: bytes 0-999
Segment 2: bytes 1000-1999 (lost)
Segment 3: bytes 2000-2999 (arrives)
Receiver expects byte 1000, so it acknowledges 1000.
The sender retransmits bytes 1000-1999.
After the missing bytes arrive, the receiver can deliver bytes 0-2999 in order.Depending on the implementation and timing, repeated acknowledgments for the same next-expected byte can provide a duplicate acknowledgment signal. TCP may retransmit before its retransmission timer expires when enough duplicate acknowledgments suggest a missing segment. Otherwise, a retransmission timeout causes unacknowledged data to be sent again.
TCP presents a reliable, ordered byte stream. It does not preserve application message boundaries. If an application writes two blocks, the receiver might read them as one block, several smaller blocks, or a different grouping. Applications that need message boundaries must add framing, such as a length field or delimiter.
TCP reliability means that TCP attempts to deliver bytes correctly and in order, or reports a connection failure. It does not guarantee that the receiving application successfully parsed, stored, or acted on those bytes. Application-level acknowledgments may be needed when processing matters.
Flow control and receive windows
Flow control protects a receiver from being overwhelmed. The receiver advertises a receive window, which represents how much additional data it can accept beyond the data already received but not yet acknowledged as available to the application.
The sender uses the advertised window to limit the amount of unacknowledged data in flight. If the receiving application reads slowly, the operating system's receive buffer can fill. The receiver then advertises a smaller window.
A zero window means the receiver currently has no buffer space for additional payload. The sender pauses normal data transmission, although TCP may send window probes or control traffic. When the application consumes data and space becomes available, the receiver sends a window update.
Window Scale is a TCP option negotiated during the handshake. The ordinary window field is limited in size, so window scaling applies a multiplier that permits a much larger effective receive window. Large windows are useful on paths with high bandwidth and noticeable latency.
Flow control and congestion control solve different problems:
- Receive-window flow control: Protects the receiving host and its buffers.
- Congestion control: Protects the network from excessive traffic.
Congestion control overview
A sender must not transmit as quickly as its local connection permits if the network between the endpoints is congested. TCP therefore maintains a congestion window, often called cwnd, that limits in-flight data based on observed network conditions.
The usable sending amount is constrained by both receiver capacity and network conditions. Conceptually, the sender is limited by the smaller of the advertised receive window and the congestion window:
effective sending window = minimum(receive window, congestion window)- Slow start: Begins with a relatively small congestion window and increases it rapidly as acknowledgments arrive.
- Congestion avoidance: Increases sending pressure more cautiously after the connection reaches a threshold.
- Loss signaling: Retransmission timeouts, duplicate acknowledgments, or explicit congestion signals can indicate trouble.
- Recovery: The sender reduces its congestion window and then cautiously increases it again according to the TCP implementation's algorithm.
Congestion control is a network-stability mechanism, not an error-correction feature. Retransmission addresses missing data; congestion control reduces the likelihood that sending behavior will worsen network overload.
TCP segment and header structure
A TCP segment contains a TCP header followed by optional payload data. The fixed header is 20 bytes. Options can extend the header, up to a maximum of 60 bytes. Padding aligns the options to a suitable boundary.
Checksum and pseudo-header
The TCP checksum covers the TCP header and payload. Its calculation also includes a pseudo-header derived from IP information, such as source and destination addresses, the IP protocol number, and the TCP length. The pseudo-header helps detect delivery of a segment to the wrong network endpoint or protocol context.
Common TCP options
- MSS: Maximum Segment Size, indicating the largest TCP payload an endpoint is willing to receive in one segment.
- Window Scale: Extends the effective receive-window size and must be negotiated during setup.
- SACK: Selective acknowledgment information can identify noncontiguous blocks received successfully, improving recovery from multiple losses.
- Timestamps: Can assist round-trip measurement and protect against sequence-number ambiguity.
The MTU, or Maximum Transmission Unit, is the largest packet payload supported by a link. MTU and IP headers influence the MSS that can be used without fragmentation. Incorrect MTU or path-MTU behavior can cause connections to stall even when the handshake succeeds.
TCP flags
Packet-analysis tools display these flags so you can identify connection phases. A SYN commonly starts setup, a SYN-ACK is the server's normal response, FIN begins an orderly close, and RST indicates an abrupt reset or rejection.
Connection termination
TCP is full-duplex: data can travel independently in both directions. Consequently, closing a TCP connection is normally directional. One endpoint can finish sending while it continues receiving data from the peer.
A typical graceful close proceeds as follows:
- Endpoint A sends FIN, indicating that it will send no more data in its direction.
- Endpoint B sends ACK. B may continue sending data to A.
- When B finishes, B sends its own FIN.
- Endpoint A sends the final ACK and enters TIME-WAIT.
Endpoint A Endpoint B
| |
| ------------- FIN ---------------------> |
| <------------ ACK ---------------------- |
| |
| <------------ FIN ---------------------- |
| ------------- ACK ---------------------> |
| |
TIME-WAIT CLOSEDBecause each direction closes separately, a graceful close may require four control segments. Relevant states include:
- FIN-WAIT: The endpoint that sent FIN is waiting for acknowledgment and possibly the peer's FIN.
- CLOSE-WAIT: An endpoint has received the peer's FIN and acknowledged it, but its local application has not yet closed its sending side.
- LAST-ACK: An endpoint has sent its FIN after receiving the peer's FIN and is waiting for the final acknowledgment.
- TIME-WAIT: The active closer keeps state temporarily after sending the final ACK.
- CLOSED: No connection state remains.
TIME-WAIT helps prevent delayed segments from an old connection from being confused with traffic from a later connection using the same four-tuple. It also allows the endpoint that sent the final ACK to retransmit that ACK if the peer retransmits its FIN. TIME-WAIT is expected protocol behavior and should not be removed casually.
An RST is different from FIN. It rejects or abruptly terminates a connection, discarding the normal graceful-close exchange. It can result from a closed port, an invalid segment for the current state, an application abort, or an active firewall rejection.
TCP application examples and ports
A port identifies a service endpoint, but a port number does not guarantee which protocol is actually running there. Administrators can configure services to use nonstandard ports, and an untrusted service can listen on a port normally associated with something else.
Database services, directory services, APIs, and custom applications also frequently use TCP. In an HTTPS example, a client may choose source port 49152 and connect to server port 443. The TCP handshake completes first; then TLS negotiation and HTTP data travel inside the established TCP byte stream.
TCP is useful when an application needs complete, ordered delivery and can tolerate connection setup, retransmission delay, and stream semantics. Applications such as interactive media, fast request protocols, or multiplexed systems may prefer UDP or another transport when lower latency, independent messages, or application-specific recovery is more important.
TCP compared with UDP
UDP is not automatically useless or incapable of reliability. An application can add sequence numbers, acknowledgments, retransmissions, encryption, or congestion behavior above UDP. The difference is that these features are not supplied by UDP itself.
TCP can also be undesirable for some workloads. Its ordered stream creates head-of-line blocking: if an early segment is missing, later received bytes cannot be delivered to the application as a continuous stream until the missing bytes are recovered. This can add delay to real-time or multiplexed workloads.
Observing TCP on a host
Use diagnostic commands only on systems and networks you own or are explicitly authorized to assess.
Inspect listening and established sockets
ss -ltn
ss -tnThe first command lists listening TCP sockets without resolving names. The second lists TCP connections. Typical columns include local address and port, peer address and port, and TCP state. A wildcard local address means the service may be listening on multiple local interfaces.
To include process ownership, use elevated privileges on a host you administer:
sudo ss -ltnpThis helps connect a listening port to the process that owns it. It is useful when a service appears to be missing, is bound to the wrong address, or conflicts with another listener.
Capture a service conversation
sudo tcpdump -ni any 'tcp port 443'This captures TCP traffic involving port 443. In a controlled test, look for SYN, SYN-ACK, and ACK during setup, then FIN or RST during termination. A packet analyzer can also show sequence and acknowledgment progress, advertised windows, negotiated options, duplicate acknowledgments, and retransmissions.
To restrict a capture to two documented placeholder addresses:
sudo tcpdump -ni any 'tcp and host 192.0.2.10 and host 192.0.2.20'Adjust the interface and addresses for an authorized lab. A capture can answer questions that socket listings cannot, such as whether a SYN left the host, whether a reply returned, and whether a firewall or peer sent an RST.
Test a controlled TCP service
nc -vz 127.0.0.1 22A successful TCP connect indicates that the endpoint is reachable and accepted a TCP connection. It does not prove that authentication will succeed or that the application request is authorized.
Observe authorized Nmap TCP scan behavior
nmap -sS -p 22,80,443 127.0.0.1Run scans only with explicit authorization. In a SYN scan, common TCP-level evidence includes:
- Open: The target responds to the SYN with SYN-ACK. Nmap then commonly sends a reset rather than completing a normal application connection.
- Closed: The target is reachable but responds with RST, indicating that no service is accepting that port.
- Filtered: A firewall or filter prevents a conclusive response, often producing no response and a timeout.
These classifications are observations about packets and responses, not absolute claims about the application behind a port.
Diagnosing common TCP problems
A client connection attempt times out
- A firewall or security group may drop the SYN or the return traffic.
- The host may be unavailable, incorrectly routed, or using an asymmetric return path.
- The service may listen only on another interface or address.
Check the listener with ss -ltn, capture the attempt with tcpdump, review firewall policy, and verify routing in both directions. A capture showing an outgoing SYN but no reply differs from one showing a returned RST or a SYN-ACK that never reaches the client.
A connection is immediately refused
- No service is listening on the destination port.
- A host firewall is actively rejecting the connection.
- The service is bound to a different address or has stopped.
Look for an RST in a packet capture and verify the listener's local address, port, and owning process. An immediate refusal generally differs from a silent filtering timeout.
A TCP application is slow or stalls
- Packet loss may be causing retransmissions.
- High latency may limit throughput when the effective window is too small.
- The receiving application may not consume data quickly, causing a reduced or zero receive window.
- Network congestion or traffic shaping may reduce the congestion window.
- Incorrect MTU or path-MTU behavior may prevent some packets from progressing.
Inspect retransmission indicators, duplicate acknowledgments, receive-window changes, and zero-window events. Measure latency and loss in an authorized environment and validate MTU and path-MTU configuration.
Many connections remain in TIME-WAIT
A high rate of short-lived outbound connections or a lack of connection reuse can create many TIME-WAIT sockets. Use ss to inspect them, then review application keep-alive and connection-pooling behavior. TIME-WAIT is part of correct TCP operation; deleting or shortening it without understanding the traffic can create new problems.
The server has many CLOSE-WAIT sockets
CLOSE-WAIT means the peer has closed its sending direction and the local TCP stack acknowledged that FIN, but the local application has not closed its socket. Identify the owning process and review its socket-close handling. CLOSE-WAIT commonly indicates an application resource-management problem rather than a basic network connectivity failure.
TCP connection lifecycle summary
Key points to remember
- TCP is an OSI Layer 4, connection-oriented transport protocol in the TCP/IP suite.
- IP identifies hosts; TCP ports identify application endpoints on those hosts.
- An established connection is identified by a source and destination IP address and port: the TCP four-tuple.
- The normal setup is SYN, SYN-ACK, ACK, with independent sequence-number spaces in each direction.
- TCP numbers bytes, uses cumulative acknowledgments, retransmits lost data, and delivers an ordered byte stream.
- TCP does not preserve application message boundaries and cannot guarantee that an application processed received data.
- The receive window provides flow control; the congestion window provides network congestion control.
- The TCP header is at least 20 bytes and can reach 60 bytes with options.
- FIN provides graceful, directional closure; RST abruptly resets or rejects a connection.
- Packet captures and socket listings reveal different parts of TCP behavior, so use both when troubleshooting.
For a closely related transport-layer comparison, continue with Transmission Control Protocol (TCP) as a reference while practicing the command and packet-analysis examples in an authorized lab.