Disabling ICMP unreachable messages will break Path MTU discovery with legacy IP, that is IPv4.

Operators often disable generating ICMP unreachable messages in order to protect the router’s CPU. This technique to protect the router’s control plane is rather obsolete and dangerous. Namely, IPv4 router will use ICMP message type 3, code 4 (The datagram is too big. Packet fragmentation is required but the ‘don’t fragment’ (DF) flag is on.) to signal the sender that the packet he is sending is too big to forward and has to be fragmented. This is a crucial message in Path MTU discovery process for IPv4. By the way, in IPv6 this is not the case – the Packet too big ICMPv6 message is not of the “unreachable” kind.

Let us look into an interesting example where the “no ip unreachables” Cisco IOS directive can lead to a network operator’s headache. We have a simple ring topology as depicted below:

topology

All the links except the one between routers R3 and R4 have MTU of 4000 bytes and link R3 – R4 is a bottleneck with 1500 bytes MTU. Let us focus on a iBGP session between R1 and R2. Everything works fine – BGP session is established via the shortest path R1 – R2 with Path MTU of 4000 bytes. Let us now asume that the link R1 – R2 fails. No problem, you’d say – BGP packets are simply being rerouted via R1 – R4 – R3 – R2. But the link R3 – R4 is not capable of carrying 4000-byte packets. When BGP updates get larger that 1500 bytes, packets are being dropped by R3 or R4. With “no ip unreachables” configured on those two routers, they are dropped silently! BGP session suddenly timeouts and closes, then re-establishes again. The cycle repeats – BGP is flapping. Great stuff for a NOC engineer on duty to debug!

In our case, enabling ICMP unreachables on R3, gi1/0 and R4, gi0/0 will solve the problem. Once again, I have to point to an excellent Ivan’s post with a little comment: I’ve seen this happened live in a real network before Googling to ipSpace. Should read that sooner :-).

Advertisements