ping6 can be a useful tool in troubleshooting MTU-related issues. Being IPv6 network operators we must known how it actually works with IPv6 in relation to fragmentation. Take you time and inspect the ping behaviour on your system and find a way how to examine your IPv6 destination cache. The know-how you will gain might come handy sometimes.

Troubleshooting MTU-related issues is a common task in IP network operations, IPv6 being no exception. A simple tool like ping is often used to discover MTU on the path between two hosts. When it comes to IPv6, fragmentation no longer happens in the network but only at the hosts. Hence, host (a sender) must be informed about the MTU for a certain destination. This is, of course, done by the routers on the path, which use ICMPv6 packet-too-big messages to inform the sender that packet is too big to dispatch. Hopefully these messages, which include the reduced MTU value, are received by the sender. This allows the sender to store the reduced MTU values in its IPv6 destination cache for a certain period of time.

In IPv4, ping is used with the “Don’t Fragment Flag set” (use option -M do in Linux, -f in Windows and -D in BSD-like OS X to set the DF-bit in IPv4 header). An operater can vary the ping payload size and establish the correct Path MTU (here we assume that ping packets are not being blocked somewhere on the path). However, in IPv6 there is no DF-bit and the trick which worked fine with ping can not be used anymore.

Ping for IPv6, let’s call it ping6, behaves like any other IPv6 application on Windows and OS X – if the packet which is about to be sent is too big, it gets fragmented. If the sender is not already aware of the Path MTU for the “pinged” destination, then packets are first sent oversized and then, after ICMPv6 packet-too-big message is received, fragmented accordingly.
Linux is a bit different. ping6 still has the -M option, however -M do does not set the DF-bit (remember, there is no DF-bit in IPv6) but instructs ping6 not to send any fragments. To accomplish this on Linux ping6 needs raw access to the socket and therefore runs with root privileges (it is often used with set-uid root).

Let’s look into a few examples for Linux (Ubuntu 13), Windows 8.1 and OS X 10.8. Our ping target (shown as <target>, 2001:x:e813:a00::d25) sits behind a link with MTU of 1400 bytes.


Linux

First we ping successfully with 1500 bytes IPv6 packets. We capture this with tcpdump which shows that our packets are sent as fragments. Obviously our system already knows the Path MTU for 2001:x:e813:a00::d25. Fragment size is 1352 bytes, which leed us to a conclusion that Path MTU is 1352 (ping payload) + 8 (ICMPv6 echo-request) + 40 (IPv6) = 1400 bytes.

janez@ubuntu13:~$ ping6 -c 3 -s 1452 <target>
PING <target>(2001:x:e813:a00::d25) 1452 data bytes
1460 bytes from 2001:x:e813:a00::d25: icmp_seq=1 ttl=63 time=0.502 ms
1460 bytes from 2001:x:e813:a00::d25: icmp_seq=2 ttl=63 time=0.825 ms
1460 bytes from 2001:x:e813:a00::d25: icmp_seq=3 ttl=63 time=0.698 ms

--- <target> ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.502/0.675/0.825/0.132 ms

14:01:57.955920 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: frag (0|1352) ICMP6, echo request, seq 1, length 1352
14:01:57.956004 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: frag (1352|108)
14:01:57.956376 IP6 2001:x:e813:a00::d25 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, echo reply, seq 1, length 1460
14:01:58.957698 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: frag (0|1352) ICMP6, echo request, seq 2, length 1352
14:01:58.957862 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: frag (1352|108)
14:01:58.958462 IP6 2001:x:e813:a00::d25 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, echo reply, seq 2, length 1460
14:01:59.959481 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: frag (0|1352) ICMP6, echo request, seq 3, length 1352
14:01:59.959556 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: frag (1352|108)
14:01:59.960121 IP6 2001:x:e813:a00::d25 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, echo reply, seq 3, length 1460

Now we use the -M do option. First, packet is too big:

janez@ubuntu13:~$ ping6 -c 3 -s 1452 -M do <target>
PING <target>(2001:x:e813:a00::d25) 1452 data bytes
From 2001:x:e811:b00::1 icmp_seq=1 Packet too big: mtu=1400
From 2001:x:e811:b00:a853:36a6:be30:898 icmp_seq=2 Packet too big: mtu=1400
From 2001:x:e811:b00:a853:36a6:be30:898 icmp_seq=2 Packet too big: mtu=1400

--- <target> ping statistics ---
1 packets transmitted, 0 received, +3 errors, 100% packet loss, time 1013ms

14:03:11.552104 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: ICMP6, echo request, seq 1, length 1460
14:03:11.552597 IP6 2001:x:e811:b00::1 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, packet too big, mtu 1400, length 1240

…but when we reduce the payload size, ping works fine:

janez@ubuntu13:~$ ping6 -c 3 -s 1352 -M do <target>
PING <target>(2001:x:e813:a00::d25) 1352 data bytes
1360 bytes from 2001:x:e813:a00::d25: icmp_seq=1 ttl=63 time=0.601 ms
1360 bytes from 2001:x:e813:a00::d25: icmp_seq=2 ttl=63 time=0.691 ms
1360 bytes from 2001:x:e813:a00::d25: icmp_seq=3 ttl=63 time=0.669 ms

--- <target> ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.601/0.653/0.691/0.048 ms

14:03:59.267633 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: ICMP6, echo request, seq 1, length 1360
14:03:59.268191 IP6 2001:x:e813:a00::d25 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, echo reply, seq 1, length 1360
14:04:00.269105 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: ICMP6, echo request, seq 2, length 1360
14:04:00.269752 IP6 2001:x:e813:a00::d25 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, echo reply, seq 2, length 1360
14:04:01.270513 IP6 2001:x:e811:b00:a853:36a6:be30:898 > 2001:x:e813:a00::d25: ICMP6, echo request, seq 3, length 1360
14:04:01.271141 IP6 2001:x:e813:a00::d25 > 2001:x:e811:b00:a853:36a6:be30:898: ICMP6, echo reply, seq 3, length 1360

We can now examine the IPv6 destination cache for the 2001:x:e813:a00::d25. This eventually confirms that Path MTU for our target is 1400 bytes:

janez@ubuntu13:~$ ip -6 route get 2001:x:e813:a00::d25
2001:x:e813:a00::d25 from :: via fe80::669e:f3ff:fe68:2ba0 dev eth0  src 2001:x:e811:b00:a853:36a6:be30:898  metric 0
    cache  expires 375sec mtu 1400

QED.


Windows

Let’s move to Windows 8.1. First we clear the IPv6 destination cache and use the standard ping. It works fine. When we examine the destination cache, our test target is there with MTU 1500. Obviously, because only small packets were sent to 2001:x:e813:a00::d25 and Path MTU discovery did not occur (no packet-too-big messages were received) our Windows system is still not aware of the reduced MTU.

C:\>netsh interface ipv6 delete destinationcache
Ok.

C:\>ping -n 3 <target>

Pinging <target> [2001:x:e813:a00::d25] with 32 bytes of data:
Reply from 2001:x:e813:a00::d25: time=6ms
Reply from 2001:x:e813:a00::d25: time=1ms
Reply from 2001:x:e813:a00::d25: time<1ms

Ping statistics for 2001:x:e813:a00::d25:
    Packets: Sent = 3, Received = 3, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 6ms, Average = 2ms

C:\>netsh interface ipv6 show destinationcache interface="Ethernet"

Interface 3: Ethernet

PMTU Destination Address                           Next Hop Address
---- --------------------------------------------- ---
1500 2001:x:e813:a00::d25                          fe80::669e:f3ff:fe68:2ba0
1500 fe80::390d:4faf:5b8d:f112                     fe80::390d:4faf:5b8d:f112

We increase our ping payload now to reach 1500 bytes including IPv6 header. Ping still works (well, our first packet got lost). With wireshark we can confirm that our packets are now fragmented. Finally, IPv6 destination cache shows the MTU of 1400 bytes for the “pinged” target:

C:\>ping -n 3 -l 1452 <target>

Pinging <target> [2001:x:e813:a00::d25] with 1452 bytes of data:
General failure.
Reply from 2001:x:e813:a00::d25: time=1ms
Reply from 2001:x:e813:a00::d25: time<1ms

Ping statistics for 2001:x:e813:a00::d25:
    Packets: Sent = 3, Received = 2, Lost = 1 (33% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 1ms, Average = 0ms

C:\>netsh interface ipv6 show destinationcache interface="Ethernet"

Interface 3: Ethernet

PMTU Destination Address                           Next Hop Address
---- --------------------------------------------- ---
1400 2001:x:e813:a00::d25                          fe80::669e:f3ff:fe68:2ba0
1500 ff02::c                                       ff02::c
1500 fe80::390d:4faf:5b8d:f112                     fe80::390d:4faf:5b8d:f112


OS X

Apple OS X as well as some BSD-like machines can be tricky with ping6. By default they use the minimum MTU for IPv6 echo-requests which is 1280 bytes. It looks like this:

matjaz@macbook:$ ping6 -c 3 -s 1452 <target>
PING6(1500=40+8+1452 bytes) 2001:x::d9e1:432d:f9ae:2af4 --> 2001:x:e813:a00::d25
1460 bytes from 2001:x:e813:a00::d25, icmp_seq=0 hlim=62 time=6.049 ms
1460 bytes from 2001:x:e813:a00::d25, icmp_seq=1 hlim=62 time=6.268 ms
1460 bytes from 2001:x:e813:a00::d25, icmp_seq=2 hlim=62 time=5.876 ms

--- <target> ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.876/6.064/6.268/0.160 ms

14:11:37.376318 IP6 (hlim 64, next-header Fragment (44) payload length: 1240) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x26a17197:0|1232) ICMP6, echo request, length 1232, seq 0
14:11:37.376320 IP6 (hlim 64, next-header Fragment (44) payload length: 236) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x26a17197:1232|228)
14:11:37.382070 IP6 (hlim 62, next-header ICMPv6 (58) payload length: 1460) 2001:x:e813:a00::d25 > 2001:x::d9e1:432d:f9ae:2af4: [icmp6 sum ok] ICMP6, echo reply, length 1460, seq 0
14:11:38.377832 IP6 (hlim 64, next-header Fragment (44) payload length: 1240) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x6d910500:0|1232) ICMP6, echo request, length 1232, seq 1
14:11:38.377835 IP6 (hlim 64, next-header Fragment (44) payload length: 236) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x6d910500:1232|228)
14:11:38.383413 IP6 (hlim 62, next-header ICMPv6 (58) payload length: 1460) 2001:x:e813:a00::d25 > 2001:x::d9e1:432d:f9ae:2af4: [icmp6 sum ok] ICMP6, echo reply, length 1460, seq 1
14:11:39.376884 IP6 (hlim 64, next-header Fragment (44) payload length: 1240) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x522ea4fa:0|1232) ICMP6, echo request, length 1232, seq 2
14:11:39.376887 IP6 (hlim 64, next-header Fragment (44) payload length: 236) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x522ea4fa:1232|228)
14:11:39.382462 IP6 (hlim 62, next-header ICMPv6 (58) payload length: 1460) 2001:x:e813:a00::d25 > 2001:x::d9e1:432d:f9ae:2af4: [icmp6 sum ok] ICMP6, echo reply, length 1460, seq 2

We see that despite a 1500-bytes probes were requested, fragments were sent as our system MTU was 1280 bytes only. Manual page for ping6 comes to the rescue. With an optional -m, which requires root privileges, bigger packets are sent (as requested) and Path MTU discovery occurs. MTU is reported as 1400 bytes – see the tcpdump snippet below:

matjaz@macbook:$ sudo ping6 -c 3 -m -s 1452 <target>
Password:
PING6(1500=40+8+1452 bytes) 2001:x::d9e1:432d:f9ae:2af4 --> 2001:x:e813:a00::d25
1460 bytes from 2001:x:e813:a00::d25, icmp_seq=1 hlim=62 time=7.213 ms
1460 bytes from 2001:x:e813:a00::d25, icmp_seq=2 hlim=62 time=7.784 ms

--- <target> ping6 statistics ---
3 packets transmitted, 2 packets received, 33.3% packet loss
round-trip min/avg/max/std-dev = 7.213/7.498/7.784/0.285 ms

14:12:43.471937 IP6 (hlim 64, next-header ICMPv6 (58) payload length: 1460) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: [icmp6 sum ok] ICMP6, echo request, length 1460, seq 0
14:12:43.477081 IP6 (hlim 63, next-header ICMPv6 (58) payload length: 1240) 2001:x:209f:ff::2 > 2001:x::d9e1:432d:f9ae:2af4: [icmp6 sum ok] ICMP6, packet too big, length 1240, mtu 1400
14:12:44.472607 IP6 (hlim 64, next-header Fragment (44) payload length: 1360) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x32cee980:0|1352) ICMP6, echo request, length 1352, seq 1
14:12:44.472610 IP6 (hlim 64, next-header Fragment (44) payload length: 116) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x32cee980:1352|108)
14:12:44.479467 IP6 (hlim 62, next-header ICMPv6 (58) payload length: 1460) 2001:x:e813:a00::d25 > 2001:x::d9e1:432d:f9ae:2af4: [icmp6 sum ok] ICMP6, echo reply, length 1460, seq 1
14:12:45.472058 IP6 (hlim 64, next-header Fragment (44) payload length: 1360) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x4001bbcc:0|1352) ICMP6, echo request, length 1352, seq 2
14:12:45.472061 IP6 (hlim 64, next-header Fragment (44) payload length: 116) 2001:x::d9e1:432d:f9ae:2af4 > 2001:x:e813:a00::d25: frag (0x4001bbcc:1352|108)
14:12:45.479537 IP6 (hlim 62, next-header ICMPv6 (58) payload length: 1460) 2001:x:e813:a00::d25 > 2001:x::d9e1:432d:f9ae:2af4: [icmp6 sum ok] ICMP6, echo reply, length 1460, seq 2

On BSD we check MTU with the “hardcore” netstat. Use options -narWl to get the proper result and refer to “man netstat” for more information. For example:

matjaz@macbook:~$ netstat -narWl -f inet6 | head
Routing tables

Internet6:
Destination               Gateway                         Flags        Refs      Use    Mtu    Netif Expire
default                   fe80::222:56ff:feba:21bf%en1    UGc            21        0   1500      en1
...

matjaz@macbook:~$ netstat -narWl -f inet6 | grep 2001:x:e813:
2001:x:e813:a00::d25      fe80::222:56ff:feba:21bf%en1    UGHW3Ii         0        8   1400      en1   1378


To conclude (please adjust your packet size according to your needs, these examples are for 1500 bytes which gives you 1500-40-8 = 1452 bytes for ping payload) – here are the basics:

  • Linux
    ping6 -M do -s 1452 <target>
    ip -6 route get <target>
  • Windows
    ping -l 1452 <target>
    netsh interface ipv6 show destinationcache interface=…
  • BSD, OS X
    ping6 -m -s 1452 <target>
    netstat -narWl -f inet6

Happy troubleshooting!


References:
Debugging IPv6 MTU issues in Windows
RFC 4443 – Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification

Advertisements