SOL9487: BIG-IP support for neighboring VRRP/HSRP routers
Summary
VRRP and last hop compatibility
Disabling the auto_lasthop setting
Using the Last Hop Pools setting with HSRP
VRRP / HSRP router groups
Virtual router addressing
Summary
Virtual Router Redundancy Protocol (VRRP) and Hot Standby Router Protocol (HSRP) are router redundancy protocols designed to increase the availability of the default gateway servicing hosts on the same subnet. This increased reliability is achieved by sharing a virtual router IP address among multiple physical routers (virtual router group) to provide a fault-tolerant gateway and seamless failover in the event of a physical router failure.
When the BIG-IP system is in a routing path adjacent to VRRP/HSRP routers, the BIG-IP system's default stateful response traffic management may interfere with seamless VRRP/HSRP failover. To support the intended failover mechanics, the BIG-IP system may be configured to perform less stateful IP forwarding.
The BIG-IP system's default behavior is to return response traffic along the same path by which the request arrived, even if a routing table entry might dictate otherwise. The BIG-IP system records the MAC address from which the frame was forwarded (the last hop) and uses the same MAC address to forward response traffic instead of referring to the routing table. This last hop management behavior may interfere with the flow of traffic when a neighboring VRRP or HSRP router group experiences a failover of the master router hosting the virtual gateway IP addresses.
For example, the connection flow established for the following SSH connection shows that the BIG-IP system has recorded the MAC address 00:0f:f8:11:11:11 as its last hop address (the L2 address from which the request was sent):
VIRTUAL any:any <-> NODE 10.10.10.10:ssh
CLIENTSIDE 172.16.111.111:60710 <-> 192.168.10.1:ssh
(pkts,bits) in = (113, 76048), out = (91, 199136)
SERVERSIDE 172.16.111.111:60710 <-> 192.168.10.1:ssh
(pkts,bits) in = (91, 199136), out = (113, 76016)
PROTOCOL tcp UNIT 1 IDLE 5 (300) LASTHOP 4095 00:0f:f8:11:11:11
This last hop MAC address will be used in lieu of any routing table information to return the response to the client via the same path it arrived.
The BIG-IP system’s last hop functionality reduces both latency and network congestion by eliminating a large percentage of routing table lookups and ARP requests. The last hop functionality is efficient for most load balancing scenarios in a static L2 / L3 network (where the hops surrounding the BIG-IP system do not share or change MAC or IP addresses). However, it may interfere with protocols like HSRP/VRRP, which do share virtual MAC and IP addresses. In those cases, F5 Networks recommends modifying the default last hop configuration so response traffic may be handled more flexibly upon failure of the master router. The BIG-IP system can manage the last hop functionality with the auto_lasthop setting, which is enabled by default, or the Last Hop Pools setting.
Because VRRP and HSRP differ with regard to the use of the virtual IP address, a different BIG-IP configuration is recommended in each case:
- To support neighboring VRRP router groups, the default global auto_lasthop setting must be disabled, and the Last Hop Pools setting may not be used.
- To support neighboring HSRP router groups, the Last Hop Pools setting is the preferred solution, but may not work in all cases. If last hop pools are not configured, the auto_lasthop setting must be disabled.
Important: As vendor implementations may vary, F5 Networks recommends testing thoroughly to confirm actual behavior with the intended configuration.
VRRP and last hop compatibility
The BIG-IP system’s last hop functionality is not compatible with VRRP; the BIG-IP system must be able to detect a master router failure, and is unable to do so when monitoring VRRP router groups.
VRRP uses the IP address of the configured master router as the virtual router group IP address. Upon failure of the configured master router, the virtual IP address floats to the new master router. When the configured master router recovers, it again resumes hosting the virtual router group address. Since the virtual IP address is the only address available for monitoring the configured master router, and that address floats to the backup router when the configured master router fails, the IP address corresponding to the configured master router always appears available, preventing the BIG-IP system from detecting the master router failure.
For example, the following VRRP configuration using the Last Hop Pool setting will result in traffic being sent to the configured master router even after it fails:
Router 1 (master) ROUTER 2 (backup)
IP=10.10.10.1 IP=10.10.10.2
MAC=00:60:cf:11:11:11 MAC=00:60:cf:22:22:22
VIP=10.10.10.1
VMAC=00:00:5e:00:01:01
pool my_vrrp_lasthop_pool {
action on svcdown reselect
monitor all gateway_icmp_transparent_short_timeout
members
10.10.10.1:any ## Real IP of router
10.10.10.2:any ## Real IP of router
}
To monitor the pool members, the BIG-IP system first sends an ARP request to learn the MAC address of each. The master router will respond to the ARP who-has 10.10.10.1? request with the virtual MAC (VMAC) address 00:00:5e:00:01:01. The backup router 10.10.10.2 will respond to the ARP who-has 10.10.10.2? request with its unique MAC address 00:60:cf:22:22:22. The BIG-IP system will send the monitor for pool member 10.10.10.1 to the VMAC.
If the configured master router fails, the backup router will assume the virtual IP and MAC addresses 10.10.10.1 and 00:00:5e:00:01:01. When the BIG-IP system again sends the monitor request for pool member 10.10.10.1 to the virtual MAC address, and receives the expected response from the backup router. Thus, the pool member corresponding to the virtual IP address will always appear healthy to the BIG-IP system, and a new pool member will not be reselected. Any traffic that originated from the failed pool member's unique source MAC address will be impacted.
With the auto_lasthop setting enabled, a similar issue arises. The BIG-IP system cannot detect a master router failure, and will continue to send response traffic to its unique MAC address. Any traffic that originated from the failed pool member's unique source MAC address will be impacted.
Therefore, when using VRRP, F5 Networks recommends disabling auto_lasthop globally and avoiding the use of last hop pools, instead depending on IP routing to forward response traffic to its next hop.
Disabling the auto_lasthop setting
To disable the auto_lasthop setting from the BIG-IP GUI, clear the auto_lasthop check box on the System->General Properties->Local Traffic screen.
To disable the auto_lasthop setting from the command line, type the following commands:
# bigpipe db Connection.AutoLasthop disable
# bigpipe save
Note: Disabling the auto_lasthop setting may affect system performance. Without it, the BIG-IP system must instead ARP for MAC addresses and perform routing table lookups rather than using last hop data from the connection table. In addition, Packet Velocity Accelerator (PVA) will also be disabled globally when the auto_lasthop setting is disabled on platforms with a PVA.
F5 Networks is currently tracking enhancement request CR100152 to enable or disable the auto_lasthop setting on a per virtual server basis in order to avoid this performance penalty for unrelated virtual servers.
Using the Last Hop Pools setting with HSRP
The Last Hop Pools setting provides both high availability and security by allowing the BIG-IP system to select an alternate L2 return path from the remaining healthy members of the last hop pool when an active member of the pool fails. Last hop pools are configured and monitored similar to firewall or router load balancing pools, but instead of containing destination addresses, they contain the IP addresses of devices from which traffic is expected.
Note: For more information, refer to SOL2211: Using auto lasthop and lasthop pools for firewall load balancing.
As with the auto_lasthop setting, the source MAC address is recorded in the connection table. When response traffic is seen, the BIG-IP system checks the status of the pool member IP address associated with the last hop address, and if the monitor status reflects that the pool member is available, the response traffic is forwarded to that pool member. If the associated pool member is not available, another member of the last hop pool is chosen, and the response traffic is forwarded to that pool member instead.
For example, the following configuration allows the BIG-IP system to use last hop information in preference to the routing table if the master router fails:
Router 1 (Active) ROUTER 2 (Standby)
IP=10.10.10.2 IP=10.10.10.3
MAC=00:0f:f8:11:11:11 MAC=00:0f:f8:22:22:22
VIP=10.10.10.1
VMAC=00:00:0C:07:AC:01
pool my_hsrp_lasthop_pool {
action on svcdown reselect
monitor all gateway_icmp_transparent_short_timeout
members
10.10.10.2:any ## Real IP of router
10.10.10.3:any ## Real IP of router
}
When Router 1 fails, it is marked down by the pool monitor. The BIG-IP system then receives response traffic on a flow with a last hop address of 00:0f:f8:11:11:11. Since the pool member to which that MAC address corresponds is no longer available to pass traffic, the BIG-IP system instead selects the remaining healthy pool member and forwards the traffic to the new MAC address 00:0f:f8:22:22:22.
By default, if traffic arrives on a virtual server from a source other than those defined in the last hop pool (from a MAC address not corresponding to an IP address in the configured last hop pool), the BIG-IP system uses the routing table to forward the response traffic back to the client. This default behavior is appropriate regardless of whether traffic is expected to originate from a last hop pool member. If all traffic is expected to originate from a last hop pool member, network security may optionally be tightened by configuring the BIG-IP system to limit the MAC addresses to which a virtual server will return traffic, or even to reject the offending response traffic. To modify the BIG-IP behavior for response traffic not corresponding to a last hop pool member, you can change the value of the db variable TM.LHPNoMemberAction.
Note: For more information, refer to SOL8290: Change in Behavior: TM.LHPNoMemberAction variable allows control over routing behavior when last hop pools are used.
In order for BIG-IP to detect a master router failure and direct traffic to the new master router, last hop pools must be monitored using a transparent health monitor that ensures that each pool member can not only respond to ICMP on the local interface, but can successfully route traffic to the next hop. F5 Networks recommends a shorter timeout than the default 30 seconds to limit the amount of time a pool member appears healthy to the BIG-IP system after it has failed. Since the monitor timeout must expire before the pool member is marked down, the timeout represents the maximum number of seconds it may take the BIG-IP system to recognize the failure and reselect another pool member when the active pool member becomes unresponsive. For TCP connections, this delay is potentially recoverable, but it may be unacceptable for other protocols.
Note: A health monitor must mark the pool member down to prevent the BIG-IP system from using it and to force reselection of another member. A last hop pool member that is administratively down may still be marked up if the pool member is able to route the health monitor request through its unique IP address.
Note: For more information, refer to SOL350: Recommended health monitor frequency and timeout values and the Transparent gateway ICMP monitors section of SOL8971: Creating transparent ICMP health monitors.
Note: On platforms with a PVA, virtual servers configured to use last hop pools are not eligible for PVA Acceleration.
VRRP / HSRP router groups
Note: The following information specific to VRRP / HSRP is intended to provide context for understanding the flow of L2 and L3 traffic upon router failover, and to help you determine the most appropriate BIG-IP configuration supporting your VRRP/HSRP redundant routing infrastructure. For more information, refer to the following specifications documents:
RFC 3768: Virtual Router Redundancy Protocol (VRRP)
RFC 2281: Cisco Hot Standby Router Protocol (HSRP)
Note: VRRP is non-proprietary, while HSRP is Cisco-proprietary.
HSRP and VRRP are router redundancy protocols; they are not routing protocols and they do not advertise IP routes or affect the routing table in any manner. The members of a virtual router group use a router redundancy protocol for intergroup communications to ensure the virtual router presence always resides on only one physical router (the master router).
In both VRRP and HSRP, a set of routers is configured as a virtual router group. One router is configured as the master router, and hosts the virtual MAC address and virtual IP address shared by the virtual router group (virtual router presence) until it fails, then another router in the group is elected as master and assumes responsibility for the virtual MAC address and virtual IP address.
Members of a virtual router group exchange multicast HELLO packets to communicate the priority and state of the master virtual router. The master router transmits packets using the virtual MAC address as the source MAC address, allowing learning bridges to determine the network segment on which the virtual router currently resides.
Virtual Router Addressing
Virtual Router Group Multicast Addresses
In order to ensure the goal of only one virtual router presence at any time, the virtual router group exchanges status information using the chosen redundancy routing protocol (VRRP or HSRP). Efficient communication to all members of the group is accomplished by multicasting to the multicast group address on the local subnet.
VRRP uses multicast IP address 224.0.0.18 and IP protocol number 112. Only the master router sends multicast HELLO packets to advertise its presence on the network. Absence of these advertisements causes a new master router to be elected.
HSRP uses multicast IP address 224.0.0.2 and UDP port 1985. All router group members exchange advertisements to communicate their priority and state, and coordinate the choice of the master router based on the information received from other group members.
Unique and Virtual Router IP Addresses
The master virtual router is also known as the IP address owner. VRRP and HSRP each use a slightly different approach in managing the virtual router IP address.
VRRP uses the IP address of the configured master router as the virtual router group IP address. Upon failover, the new master router assumes the virtual router group IP address. When the configured master router recovers, it again resumes hosting the virtual router group address.
Router #1 (configured master router) 10.10.10.1
Router #2 (configured backup router) 10.10.10.2
The 10.10.10.1 address floats to the active master router, and thus is always available. The 10.10.10.2 address will become unavailable if Router 2 fails.
HSRP uses a unique IP address as the virtual router group IP address. Upon failover, the new master router assumes the virtual router group IP address. When the configured master router recovers, it again resumes hosting the virtual router group address.
Router #1 (configured master router) 10.10.10.1
Router #2 (configured backup router) 10.10.10.2
Virtual Router 10.10.10.3
The 10.10.10.3 address floats to the active master router, and is always available. The 10.10.10.1 address will become unavailable if Router 1 fails. The 10.10.10.2 address will become unavailable if Router 2 fails.
Each member of a virtual router group uses its actual IP address as the source address for VRRP/HSRP multicast packets so they can identify each other.
The master router must accept IP datagrams addressed to the virtual router IP address. Backup routers must not perform this function.
Each physical router in the virtual router group must forward packets received on its unique assigned IP address regardless of Master/Backup state.
Virtual MAC Addresses
Both VRRP and HSRP use a virtual MAC address to minimize the impact of router failover on active traffic flows. The virtual MAC address floats to a new master router when multicast advertisements from the previous master router instance are no longer seen on the network.
VRRP virtual routers use Media Access Control (MAC) addresses in the form of 00-00-5E-00-01-XX. The last byte of the address (XX) is the Virtual Router Identifier (VRID), which must be unique for each virtual router in the network.
HSRP virtual routers use MAC addresses in the form of 00-00-0C-07-AC-XX. The last byte of the address (XX) is the configured HSRP group number in hexadecimal notation, which must be unique for each virtual router in the network.
The master router must forward packets addressed to the virtual router MAC address, and also must respond with the virtual MAC address to ARP requests for the virtual IP address. Backup routers must not perform these functions.
VRRP/HSRP multicast frames originating from the master router use the virtual router MAC address as the source MAC address. Frames are forwarded to the virtual router IP address by using the virtual router MAC address as the destination MAC address.
Unique MAC Addresses
Regardless of whether it is currently hosting the virtual MAC address, each physical router also hosts its own unique MAC address. Routers must forward packets received on their unique assigned MAC address regardless of Master/Backup state.
Traffic originating from a specific member of a virtual router group (other than multicast virtual router group communications) may use that router's unique physical MAC address as the source MAC address. Frames are forwarded to a router group member's real IP address by using the physical router's unique MAC address as the destination MAC address. Frames forwarded by a router group member use the physical router's unique MAC address as the source MAC address for outbound frames.
Note: For information about configuring the equivalent of IP forwarding, refer to SOL7595: Overview of IP forwarding virtual servers.
