VXLAN EVPN Multihoming

March 9th, 2018
VXLAN EVPN Multihoming

Introduction 

Cisco Nexus platforms support a feature called VPC (virtual port channel) which a pair of switches act as a single switch for redundancy and both of switches will function as an active switch.

For Cisco Nexus 9000 in VXLAN EVPN environments, two solutions are supported:

• Traditional VPC

• BGP EVPN 

Traditional VPC use consistency checking, which is a mechanism used by both switches as VPC pair to exchange configuration information and verify compatibility. BGP EVPN lacks this consistency check. BGP EVPN relies on LACP protocol to detect any miss-configuration.

This approach eliminates the MCT link used by traditional VPC and offers more flexibility.

BGP EVPN Multihoming:

Terminology:

• EVI: The EVPN instance (EVI) is represented by the virtual network identifier (VNI).

• MAC-VRF: The MAC Virtual Routing and Forwarding (MAC-VRF) instance is a container for housing the virtual forwarding table for MAC addresses.

• ES: The Ethernet segment (ES) is a set of bundled links.

• ESI: The Ethernet segment identifier (ESI) represents each Ethernet segment uniquely across the network.

Guidelines and limitations

• EVPN multihoming is supported on the Cisco Nexus 9300 Series switches only and it is not supported on the Cisco Nexus 9300-EX and 9500 Series switches. The Cisco Nexus 9500 Series switches can be used as Spine switches but they cannot be used as VTEPs.

• EVPN multihoming is not supported on FEX.

• EVPN multihoming is supported with multihoming to two switches only.

• Cisco recommends enabling LACP on ES PO.

EVPN Multihoming redundancy group

The following figure shows a dual-homed topology in which VTEPs Leaf1 and Leaf 2 are distributed anycast VXLAN gateways performing IRB (integrated Routing and Bridging). Host 2 is connected to a switch which is dual-homed to both Leaf1 and Leaf2.

EVPN Multihoming

The switch is not aware of that the bundle is configured on two different switches. But the Leaf 1 and Leaf 2 must be aware of that they are part of the same bundle.

To make the VTEPs aware that they belong to the same bundle link, Cisco NXOS uses the ESI and system MAC address configured on the port-channel interface.

Ethernet Segment Identifier 

It is a 10-byte value for bundled link that they share with multihomed neighbor. 

LACP Bundling

LCAP can detect ESI misconfiguration and the port-channel bundle. The LACP on the leaf switches will send the ESI configured MAC address in the hello message sent to the access switch. LACP is not mandatory with ESI. A given ESI interface shares the same ESI across the VTEPs in the group.

EVPN Multihoming with LACP

It is recommended to run LACP between VTEPs and access devices because LCAP BPDs have a mechanism to detect and act on the ESI misconfiguration.

Layer 2 Gateway STP

L2G-STP builds a loop-free topology. The STP root always must be the VXLAN Fabric. A bridge ID for STP consists of a MAC and bridge priority. For VXLAN fabric the system automatically assigns the VTEPs with a MAC address from the reserved range (c84c.75fa.6000), so each switch uses the same MAC address for the bridge ID emulating a single logical pseudo root. The L2G-STP is disabled by default.

All L2G-STP vlans should be set to a lower priority than the fabric edge switches. Or you can set the leaf switches L2G-STP priority to 0.

BGP EVPN Multihoming Configuration steps:
  • Enable multihoming globally: evpn esi multihoming
  • Enable BGP maximum paths. This setting enables equal-cost multipath (ECMP) for host routes. Otherwise, host routes will have only 1 VTEP as the next hop. 
    • maximum-paths ibgp x
    • maximum-paths x
  • Enable core links. This setting tracks uplink interfaces to the core. If all uplinks are down, local Ethernet segment–based ports will be shut down or suspended. This setting is used mainly to avoid black-holing south-to-north traffic when no uplinks are available.
    • evpn multihoming core-tracking
  • Configure the Ethernet segment.
    • interface port-channel x
    • ethernet-segment <es-id>
    • system-mac <es-system-mac>
  • Configure TCAM.
    • hardware access-list tcam region vpc-convergence 256
    • hardware access-list tcam region arp-ether 256 

 

Sample Configuration

 

Leaf 1 configuration:

evpn esi multihoming ⇒enable the feature

Interface X/Y ⇒  leaf switch connection to core 

evpn multihoming core-tracking ⇒ enable core link tracking

interface port-channel XXX  ⇒ port channel link to downstream switch

ethernet-segment 2011 ⇒ ESI

    system-mac 0000.0000.2011 ⇒ system MAC

Leaf 2 configuration:

evpn esi multihoming ⇒ enable the feature

Interface X/Y ⇒  leaf switch connection to core 

evpn multihoming core-tracking ⇒ enable core link tracking

interface port-channel XXX  ⇒ port channel link to downstream switch

ethernet-segment 2011 ⇒ ESI

    system-mac 0000.0000.2011 ⇒ system MAC

References:

• Cisco VXLAN EVPN Multihoming white paper

• Cisco Nexus 9000 series NX-OS VXLAN Configuration

 

Join the High Availability, Inc. Mailing List

Subscribe