Simplifying Should Present Less Issues, Right?
Recently we had an interesting routing conundrum with a client when we consolidated their networking infrastructure. They have two redundant circuits and an entire /24 block of IP addresses (so, 256 of them, to be exact). They originally had two Juniper routers for border gateway protocol (BGP), which in turn handed off that entire block of IP addresses to their internal network, passing through SonicWall firewalls.
This is the logical diagram of how their edge networking worked. As you can see, the Juniper routers were involved in the BGP routing to the circuits and the SonicWall firewalls shared the same IP address out of that /24 block.
When hardware performance became an issue for them, we ripped and replaced all their edge networking equipment, including those Juniper routers and SonicWall firewalls in the diagram above. In doing so, we had their two circuits directly terminating onto their new FortiGates, which handle both the routing and firewall services. However, that /24 block of IP addresses was not on an interface on the FortiGate, so they weren’t being redistributed into BGP. Public services could not be reached and VPNs could not terminate since the IP address they were configured to terminate on was no longer there.
Solving the BGP Routing Problem
There were a couple of different solutions we considered. We thought about using a secondary IP on one of the circuits, and that would’ve resolved access to the public services, but that wouldn’t have given the redundancy of the second circuit. We also considered a black hole route, which would address both the public services and the redundancy issue, but not fix the VPN connectivity.
With all that in mind, we decided to set up a loopback address. Instead of using a /32 IP block, however, which only offers one IP address, we used the entire aforementioned /24 block of IP addresses, effectively creating a subnet. This allowed the VPNs to terminate on the IP of the loopback and BGP to redistribute that /24 subnet. By that point, we didn’t need to use the black hole route since BGP was working appropriately and the issues were resolved for a single FortiGate.
A Wildcard: How FortiGates Function in HA Pairs
However, we had two FortiGates in an HA pair, in a master/slave configuration. By default, during a failover, the slave firewall will have to reestablish BGP, which causes a blip in service. We want seamless failover, so we had to set up graceful restart on our neighbors (our redundant ISP connections, in this case) in BGP and also set the routes to survive longer in the HA configuration. The reason for that is because the routes on a slave device expire sooner and have a much lower priority. This is rather unique to Fortinet devices, so if you have different routing/firewall equipment, your mileage may vary.
Above is their current setup. As you can see, we have the loopback on the FortiGate set up with that IP address that the VPNs need to terminate on. Now they have a simplified edge network and huge performance gains to boot.