Maximum transmission unit (MTU)
A maximum transmission unit (MTU) is the largest packet or frame size, specified in octets (eight-bit bytes) that can be sent in a packet- or frame-based network such as the internet. The internet’s transmission control protocol (TCP) uses the MTU to determine the maximum size of each packet in any transmission. MTU is usually associated with the Ethernet protocol, where a 1500-byte packet is the largest allowed in it (and hence over most of the internet).
One of the most common problems related to MTU is that sometimes higher-level protocols may create packets larger than a particular link supports, and you’ll need to make adjustments to make it work.
To get around this issue, IPv4 allows fragmentation which divides the datagram into pieces. Each piece is small enough to pass over the single link that it is being fragmented for, using the MTU parameter configured for that interface. This fragmentation process takes place at the IP layer (OSI layer 3) and marks the packets it fragments as such. This ensures the IP layer of the destination host knows it should reassemble the packets into the original datagram.
Fragmentation is sometimes not supported by applications, and is something we should avoid if possible. The best way to avoid fragmentation is to adjust the maximum segment size or TCP MSS so the segment will adjust its size before reaching the data link layer.
Before we look at TCP MSS, it helps to understand the build of the “unit” that’s being sent over the internet.
Command to test jumbo frame packets (9000 Bytes) in an Esxi 7.0
What Is Maximum Transmission Unit (MTU)?
The maximum MTU value is 1500, let me explain to you why :
1. MTU is normal referred to as IP MTU. 2. Jumbo frame is Ethernet MAC layer concept to send the packet in a large size in once, because the Switch has no capability to do the IP fragment. 3. 1500 is the Ethernet packet size.
Jumbo frame means that if the switch receives an Ethernet packet with packet size larger than the jumbo frame size configured on the switch interface, then the Switch will discard it. However the IP MTU is normally performed by the router (by powerful CPU), if the received IP packet size is larger than the MTU number configured on the interface, the Router will NOT discard it but fragment it. IP fragmentation consumes a large amount of CPU resources because it has to be processed packet by packet, so it is normally processed by a Router and some high end Switches. It cannot be done by the mid or low end Switches because they rely on the hardware chip.
So in our case, setting the jumbo frame to the size that will allow the packet to go through is enough, don't worry about the IP MTU.
As mentioned, the common value of MTU in the internet is 1500 bytes.
As you can see in the figure above, the MTU is built from payload (also referred as data) and the TCP and the IP header, 20 bytes each. The total value of the IP and the TCP header is 40 bytes and mandatory for each packet, which leaves us 1460 bytes for our data.
Now, imagine that we are using the GRE protocol in our network, encapsulating the original packet and adding 24 bytes for the GRE header.
The total size of this kind of packet will be 1524 bytes, exceeding the 1500 bytes MTU value. The “data” size in this packet is 1460, but we can and should decrease it in order to make sure the total size will be 1500 bytes or less. And this is where TCP MSS comes into the picture.
TCP MSS, the maximum segment size, is a parameter of the options field of the TCP header that specifies the largest amount of data, specified in bytes, that a computer or communications device can receive in a single TCP segment. It does not include the TCP header or the IP header. This value will dictate the maximum size of the “data” part of the packet. In the following case for the GRE tunnel, we will set the tcp mss value to be 1436 or lower, while the default size is 1460.
The MSS announcement (often mistakenly called a negotiation) is sent during the three-way handshake by both sides, saying: “I can accept TCP segments up to size x”. The size (x) may be larger or smaller than the default. The MSS can be used completely independently in each direction of data flow.
Since the end device will not always know about high level protocols that will be added to this packet along the way, like GRE packets for example, it won’t usually adjust the TCP MSS value. As a result the network devices have the option to rewrite the value of TCP MSS packets that are processed through them. For example, in a Cisco Router the command “ip tcp mss-adjust 1436” in the interface level will rewrite the value of the TCP MSS of any SYN packet that will go via this interface.
Background Information
This section describes the basic terminology that this document uses. This section also explains the background theory for the configurations in this document.
Term Definitions
MTU: MTU is short for Maximum Transmission Unit, the largest physical packet size, measured in bytes, that a network can transmit. Any messages larger than the MTU are divided into smaller packets before transmission.
Jumbo: Jumbo frames are frames that are bigger than the standard Ethernet frame size, which is 1518 bytes (including Layer 2 (L2) header and FCS). The definition of frame size is vendor-dependent, as these are not part of the IEEE standard.
Baby giants: The baby giants feature allows a switch to pass or forward packets that are slightly larger than the IEEE Ethernet MTU. Otherwise, the switch declares big frames as oversize and discards them.
Background Theory In order to transport traffic across switched-networks, ensure that transmitted traffic MTU does not exceed the MTU that the switch platforms support. Here are the reasons why the MTU size of certain frames is truncated:
Vendor-specific requirements: Applications and some Network Interface Cards (NICs) can specify an MTU size outside of the standard 1500 bytes. Much of this drive has been due to studies undertaken, which prove that an increase in the size of an Ethernet frame can increase average throughput.
Trunking: In order to carry VLAN-ID information between switches or other network devices, trunking has been employed to augment the standard Ethernet frame. Today, the two most common forms of trunking are Cisco's proprietary InterSwitch Link (ISL) encapsulation and IEEE 802.1q. Refer to these documents for more information on trunking:
InterSwitch Link Frame Format
Basic Characteristics of 802.q Trunking
MultiProtocol Label Switching (MPLS): When you enable MPLS on an interface, MPLS can also augment the frame size of a packet, depending on the number of labels in the Label stack for an MPLS-tagged packet. The total size of a label is four bytes. The total size of a label stack is n x 4 bytes. If a label stack is formed, the frames can exceed the MTU.
Inter-Switch Link and IEEE 802.1Q Frame Format
Configuring TCP MSS for Incapsula Infrastructure Protection
In Incapsula Infrastructure Protection, we usually use GRE tunnels. Therefore, we need to address the possibility of fragmentation if we are not adjusting the TCP MSS. Our solution, which is asymmetric by definition (meaning in-bound traffic via Incapsula, outbound traffic via the original ISP) requires us to rewrite the TCP MSS value of SYN packets that are being sent by our protected customer to their end clients. We can achieve this in a Cisco Router, for example by configuring “ip tcp mss-adjust 1436” on the WAN interface or the interface that faces the provider.
Let’s look at the following diagram.
As you can see, the customer server sends the packet with the MSS value of 1460, but in the router’s interface we will adjust it to 1420. It’s important to understand that this value will affect the size of the traffic coming from the end client toward the customer’s server. This is the traffic that will go via the tunnel and therefore be encapsulated with 24 bytes of GRE. The outbound traffic, from the customer’s server toward the end client, goes directly via the original ISP, therefore we don’t need to adjust the SYN packets the end user sends as well.
The diagram above shows how the SYN packets in the three-way handshake travel. After the three-way handshake is completed and the connection established, the end user will send packets whose “data” won’t exceed the 1420 bytes size. In addition the customer’s server will send packets whose “data” won’t exceed the default 1460 bytes.
A common mistake is to configure the “ip tcp mss-adjust 1420” command on the tunnel interface of the customer. The reason it doesn’t work is because the SYN packets that are sent from the server toward the end user are not going via the GRE tunnel interface, but via the original ISP’s interface.