IPv6 Connectivity

It's probably about time that I figured out how to enable IPv6 on my RouterPi and network! At first, configuring dhcpcd was fairly straightforward and IPv6 connectivity worked almost right away. However, it later became intermittent after rebooting the router and checking that everything was still working. For some reason my ISP's (Hyperoptic) upstream router (not the one in my home) had decided to stop responding to Router Solicitation (RS) packets sent by my router.

Router Solicitations (RS) are part of the IPv6 Neighbour Discovery Protocol (NDP) and are how IPv6-enabled devices locate routers on the link, such as the default gateway. When an RS packet is transmitted, IPv6-enabled routers should respond with a Router Advertisement (RA) packet advertising their presence. Routers also transmit RAs at periodic intervals; these are called unsolicited router advertisements.

While Hyperoptic's upstream router did not respond to RS packets, it did send unsolicited RA packets roughly every 15 - 30 minutes. In fact, it would send two identical RA packets at the same time, what's going on there?

This meant that after re-plugging the WAN cable or restarting the router, it would:

• Successfully obtain a DHCPv6 prefix delegation,
• ...then take up to 30 minutes before receiving an unsolicited RA,
• ...leaving the network with valid IPv6 addresses but no default route.

This resulted in the network seeming slow and strange, as devices would attempt to connect to websites using IPv6 before giving up and sometimes falling back to IPv4. The same thing also happened with the official home router provided by Hyperoptic.

After some experimentation I found that changing the MAC address of the WAN interface to any other valid address would trigger the ISP's upstream router into sending an unsolicited RA immediately after a new DHCPv6 prefix delegation had been assigned. This only happened once per MAC address change. I verified this by swapping between two routers - the RouterPi and the home router supplied by Hyperoptic. Since they have different MAC addresses, an RA would be sent quickly after DHCPv6 completed, and IPv6 connectivity would work right away. However, re-plugging the same router would once again result in the network appearing broken for a while due to the lack of a router advertisement and missing default IPv6 route.

So, if you're running into this problem while using the Hyperoptic home router, there's not much you can do about it. But if you're running your own custom Linux router, you can use macchanger as a quick workaround:

sudo macchanger -e eth1
sudo systemctl restart dhcpcd

The WAN cable may have to be unplugged and plugged back in after running the commands, as it seems Hyperoptic only allows one MAC address change per cable plug-in.

Alternatively, since the default gateway address does not seem to change, it's possible to just add the gateway address manually:

sudo ip -6 route replace default via (gateway IPv6 address) dev eth1 metric 2000

This can be automated by creating a dhcpcd hook script that adds the default gateway on the RENEW6 event.

Hyperoptic also does not assign non-temporary addresses (ia_na), only prefix delegations (ia_pd). Remove ia_na from dhcpcd.conf to stop messages like eth1: DHCPv6 REPLY: No addresses have been assigned from spamming logs.

But we're not finished yet!

Out-of-Order Packets

Another small but annoying problem I noticed on the network was random out of order (OOO) packets. There are many reasons why OOO packets can occur, such as network congestion, but these events were happening frequently - even when streaming a 192 kbps MP3 over the gigabit internet connection.

After a bit of Googling, I came across this Reddit thread:

RFC4448 section 4.6

Packet reordering can happen if a frame has a leading '4' or '6' Destination MAC address, going over a L2VPN PW traversing a LAG (RFC4448 states it's the source MAC, but I have yet to see this be the case).

The first nibble of the Ethernet header is the first character of the destination MAC. Also the first nibble of the IP header is the version. The router incorrectly assumes that if the MAC starts with a '4' it must be an IPv4 packet. If it starts with a '6' it must be an IPv6 packet.

Adding the control word to the PW fixes this because it forces the router to see a '0' rather than '4' or '6' after the MPLS label.

I believe this happens because the MPLS label has no field to indicate the upper layer. For instance IP has the protocol field, Ethernet has the type field, TCP/UDP have port numbers. With MPLS there is no such field, so the router just assumes an IPv4/IPv6 header comes next, but it's really an ethernet header when using PW/L2VPN.

https://tools.ietf.org/html/rfc4448#section-4.6

As it turned out, the MAC address of my RouterPi's WAN interface started with 4. Changing it to a0:de:ad:bb:ee:ff instantly fixed the out of order packets, hooray!

To make the MAC address permanent, create a file at /etc/systemd/network/01-wan.link containing:

[Match]
MACAddress=(original WAN MAC address)

[Link]
Name=eth1
MACAddress=a0:de:ad:bb:ee:ff

I do wonder how many people could be affected by out of order packets simply because their router's WAN MAC address starts with 4 or 6, which could be especially troublesome for online gaming. D: