In this post, I will outline the configuration for a VXLAN fabric using BGP EVPN and OSPF as the underlay. This is similar to my previous Arista project and VXLAN project.
The topology of this post will be four switches and six clients.
- Nexus 9000v version 9.3(1)
- OSPF will be running as point to point between the routed ports Eth1/1 and Eth1/2
- BGP will be between loopback0
- Loopback1 on the Spine switches is the multicast rendezvous point
- Loopback1 on the Leaf switches is for the NVE (VTEP)
For the configuration, I will go section by section for each device to get the full working config.
Interfaces
All physical interfaces that are part of the underlay fabric will have the MTU set to 9216 that will support jumbo frames for VXLAN. The ports will also be routed.
In this section, I am also configuring PIM sparse mode on the interfaces.
#######ALL#######
interface Eth1/1 - 2
no switchport
mtu 9216
no shut#######Spine1 INTERFACES#######
feature bgp
feature nv overlay
feature ospf
feature pim
nv overlay evpn
interface loopback0
ip address 10.10.10.11/32
ip pim sparse-mode
interface loopback1
ip address 1.1.1.1/32
ip pim sparse-mode
interface Eth1/1
ip address 10.1.1.0/31
ip pim sparse-mode
interface Eth1/2
ip address 10.1.1.2/31
ip pim sparse-mode#######Spine2 INTERFACES#######
feature bgp
feature nv overlay
feature ospf
feature pim
nv overlay evpn
interface loopback0
ip address 10.10.10.12/32
ip pim sparse-mode
interface loopback1
ip address 1.1.1.1/32
ip pim sparse-mode
interface Eth1/1
ip address 10.1.2.0/31
ip pim sparse-mode
interface Eth1/2
ip address 10.1.2.2/31
ip pim sparse-mode#######Leaf1 INTERFACES#######
feature ospf
feature bgp
feature pim
feature interface-vlan
feature nv overlay
feature vn-segment-vlan-based
nv overlay evpn
interface loopback0
ip address 10.10.10.21/32
ip pim sparse-mode
interface loopback1
ip address 2.2.2.1/32
ip pim sparse-mode
interface Eth1/1
ip address 10.1.1.1/31
ip pim sparse-mode
interface Eth1/2
ip address 10.1.2.1/31
ip pim sparse-mode#######Leaf2 INTERFACES#######
feature ospf
feature bgp
feature pim
feature interface-vlan
feature nv overlay
feature vn-segment-vlan-based
nv overlay evpn
interface loopback0
ip address 10.10.10.22/32
ip pim sparse-mode
interface loopback1
ip address 2.2.2.2/32
ip pim sparse-mode
interface Eth1/1
ip address 10.1.1.3/31
ip pim sparse-mode
no shut
interface Eth1/2
ip address 10.1.2.3/31
ip pim sparse-mode
no shutOSPF
OSPF will be used as the underlay protocol. All the loopbacks and physical Ethernet ports needs to be taking part in OSPF for all switches to heave learnt the complete fabric that will be used later for the overlay. The Ethernet ports are configured as point-to-point which reduces the amount of LSAs required.
#######Spine1 OSPF#######
router ospf UNDERLAY-NET
router-id 10.10.10.11
interface loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface loopback1
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/1
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/2
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/3
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/4
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0#######Spine2 OSPF#######
router ospf UNDERLAY-NET
router-id 10.10.10.12
interface loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface loopback1
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/1
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/2
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/3
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/4
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0#######Leaf1 OSPF#######
router ospf UNDERLAY-NET
router-id 10.10.10.21
interface loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface loopback1
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/1
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/2
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/3
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/4
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0#######Leaf2 OSPF#######
router ospf UNDERLAY-NET
router-id 10.10.10.22
interface loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface loopback1
ip ospf network point-to-point
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/1
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/2
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/3
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0
interface Ethernet1/4
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf UNDERLAY-NET area 0.0.0.0PIM
PIM as always is for the multicast. Each spine has the rendezvous point Anycast IP of 1.1.1.1 for redundancy.
#######Spine1 PIM#######
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8#######Spine2 PIM#######
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8#######Leaf1 PIM#######
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8#######Leaf2 PIM#######
ip pim rp-address 1.1.1.1 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8VLANs
These are the VLANs that will be used in the single tenant. The VLANs will be able to have traffic routed between them via VLAN 10, VNI 10000. This is also mapping the VLANs to the VNIs.
#######Leaf1 VLANs#######
vlan 1,10-15
vlan 10
vn-segment 10000
vlan 11
vn-segment 10011
vlan 12
vn-segment 10012
vlan 13
vn-segment 10013
vlan 14
vn-segment 10014
vlan 15
vn-segment 10015#######Leaf2 VLANs#######
vlan 1,10-15
vlan 10
vn-segment 10000
vlan 11
vn-segment 10011
vlan 12
vn-segment 10012
vlan 13
vn-segment 10013
vlan 14
vn-segment 10014
vlan 15
vn-segment 10015
VLAN Interfaces
These are the VLAN interfaces that contain the Anycast gateways and are mapped to the VRF for Tenant-1. VLAN 10 is here as a way to forward intervals traffic.
#######Leaf1 VLAN INTERFACES#######
fabric forwarding anycast-gateway-mac 1234.5678.9000
interface Vlan10
no shutdown
vrf member Tenant-1
ip forward
interface Vlan11
no shutdown
vrf member Tenant-1
ip address 10.0.11.1/24
fabric forwarding mode anycast-gateway
interface Vlan12
no shutdown
vrf member Tenant-1
ip address 10.0.12.1/24
fabric forwarding mode anycast-gateway
interface Vlan13
no shutdown
vrf member Tenant-1
ip address 10.0.13.1/24
fabric forwarding mode anycast-gateway
interface Vlan14
no shutdown
vrf member Tenant-1
ip address 10.0.14.1/24
fabric forwarding mode anycast-gateway
interface Vlan15
no shutdown
vrf member Tenant-1
ip address 10.0.15.1/24
fabric forwarding mode anycast-gateway
#######Leaf2 VLAN INTERFACES#######
fabric forwarding anycast-gateway-mac 1234.5678.9000
interface Vlan10
no shutdown
vrf member Tenant-1
ip forward
interface Vlan11
no shutdown
vrf member Tenant-1
ip address 10.0.11.1/24
fabric forwarding mode anycast-gateway
interface Vlan12
no shutdown
vrf member Tenant-1
ip address 10.0.12.1/24
fabric forwarding mode anycast-gateway
interface Vlan13
no shutdown
vrf member Tenant-1
ip address 10.0.13.1/24
fabric forwarding mode anycast-gateway
interface Vlan14
no shutdown
vrf member Tenant-1
ip address 10.0.14.1/24
fabric forwarding mode anycast-gateway
interface Vlan15
no shutdown
vrf member Tenant-1
ip address 10.0.15.1/24
fabric forwarding mode anycast-gatewayTenant
This is the single tenant on the fabric. Tenants are the customers or separate business unit. The tenants should not communicate with one another. There can be a shared services tenant for things like; DHCP, DNS, NTP, etc. But generally tenants do not communicate and can contain overlapping network addresses.
#######Leaf1 TENANT-1#######
interface Vlan10
vrf member Tenant-1
interface Vlan11
vrf member Tenant-1
interface Vlan12
vrf member Tenant-1
interface Vlan13
vrf member Tenant-1
interface Vlan14
vrf member Tenant-1
interface Vlan15
vrf member Tenant-1
vrf context Tenant-1
vni 10000
rd auto
address-family ipv4 unicast
route-target both auto evpn
#######Leaf2 TENANT-1#######
interface Vlan10
vrf member Tenant-1
interface Vlan11
vrf member Tenant-1
interface Vlan12
vrf member Tenant-1
interface Vlan13
vrf member Tenant-1
interface Vlan14
vrf member Tenant-1
interface Vlan15
vrf member Tenant-1
vrf context Tenant-1
vni 10000
rd auto
address-family ipv4 unicast
route-target both auto evpn
BGP
This is the MP-BGP EVPN overlay. EVPN is used as the control plane for the VXLAN data plane. EVPN will distribute the IP/MAC addresses of the endpoint hosts between the two leaf switches. By default, VXLAN operates on a flood and learn model. This is an enhancement to that behaviour, reducing the flood and learn.
The spines are acting as a route reflector as this is forming iBGP adjacencies.
#######Spine1 BGP#######
feature bgp
router bgp 65001
router-id 10.10.10.11
neighbor 10.10.10.21
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
route-reflector-client
neighbor 10.10.10.22
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
route-reflector-client#######Spine2 BGP#######
feature bgp
router bgp 65001
router-id 10.10.10.12
neighbor 10.10.10.21
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
route-reflector-client
neighbor 10.10.10.22
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
route-reflector-client#######Leaf1 BGP#######
feature bgp
router bgp 65001
router-id 10.10.10.21
neighbor 10.10.10.11
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
neighbor 10.10.10.12
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
vrf Tenant-1
address-family ipv4 unicast
advertise l2vpn evpn
evpn
vni 10011 l2
rd auto
route-target import auto
route-target export auto
vni 10012 l2
rd auto
route-target import auto
route-target export auto
vni 10013 l2
rd auto
route-target import auto
route-target export auto
vni 10014 l2
rd auto
route-target import auto
route-target export auto
vni 10015 l2
rd auto
route-target import auto
route-target export auto
vrf context Tenant-1
rd auto
address-family ipv4 unicast
route-target both auto evpn#######Leaf2 BGP#######
feature bgp
router bgp 65001
router-id 10.10.10.22
neighbor 10.10.10.11
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
neighbor 10.10.10.12
remote-as 65001
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
vrf Tenant-1
address-family ipv4 unicast
advertise l2vpn evpn
evpn
vni 10011 l2
rd auto
route-target import auto
route-target export auto
vni 10012 l2
rd auto
route-target import auto
route-target export auto
vni 10013 l2
rd auto
route-target import auto
route-target export auto
vni 10014 l2
rd auto
route-target import auto
route-target export auto
vni 10015 l2
rd auto
route-target import auto
route-target export auto
vrf context Tenant-1
rd auto
address-family ipv4 unicast
route-target both auto evpnNVE Interface
This is the Network Virtual Endpoint, also known as the VTEP (Virtual Tunnel Endpoint). It is loopback1 in this topology that will form the VXLAN tunnel. The NVE is responsible for VXLAN encapsulation/decapsulation.
#######Leaf1 NVE INTERFACE#######
interface nve1
no shutdown
host-reachability protocol bgp
source-interface loopback1
member vni 10000 associate-vrf
member vni 10011 mcast-group 239.0.0.11
member vni 10012 mcast-group 239.0.0.12
member vni 10013 mcast-group 239.0.0.13
member vni 10014 mcast-group 239.0.0.14
member vni 10015 mcast-group 239.0.0.15#######Leaf2 NVE INTERFACE#######
interface nve1
no shutdown
host-reachability protocol bgp
source-interface loopback1
member vni 10000 associate-vrf
member vni 10011 mcast-group 239.0.0.11
member vni 10012 mcast-group 239.0.0.12
member vni 10013 mcast-group 239.0.0.13
member vni 10014 mcast-group 239.0.0.14
member vni 10015 mcast-group 239.0.0.15Show Commands
As there are multiple stages to this configuration, there are lots of show commands. Some of them overlap by showing the same information.
show int status
sh ip ospf neighbors
show ip route ospf-UNDERLAY
show ip pim interface brief
show ip pim neighbor
show ip pim rp
show vrf Tenant-1 detail
show run vrf tenant-1
show interface nve 1
show nve vrf
show nve vni
show nve peers
show vxlan
show l2route evpn mac all
show l2route evpn mac-ip all
show bgp l2vpn evpn
show bgp l2vpn evpn summary
show system internal l2fwder macOSPF
OSPF is configured as point to point networks between each physical interface. All Ethernet and loopback interfaces are taking part in OSPF.
Spine1
sh ip ospf neighbors
OSPF Process ID UNDERLAY VRF default
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
10.10.10.21 1 FULL/BDR 18:04:18 10.1.1.1 Eth1/1
10.10.10.22 1 FULL/BDR 18:04:17 10.1.1.3 Eth1/2
show ip route ospf-UNDERLAY
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
2.2.2.1/32, ubest/mbest: 1/0
*via 10.1.1.1, Eth1/1, [110/41], 18:04:31, ospf-UNDERLAY, intra
2.2.2.2/32, ubest/mbest: 1/0
*via 10.1.1.3, Eth1/2, [110/41], 18:04:31, ospf-UNDERLAY, intra
10.1.2.0/31, ubest/mbest: 1/0
*via 10.1.1.1, Eth1/1, [110/80], 18:04:31, ospf-UNDERLAY, intra
10.1.2.2/31, ubest/mbest: 1/0
*via 10.1.1.3, Eth1/2, [110/80], 18:04:31, ospf-UNDERLAY, intra
10.10.10.12/32, ubest/mbest: 2/0
*via 10.1.1.1, Eth1/1, [110/81], 18:04:25, ospf-UNDERLAY, intra
*via 10.1.1.3, Eth1/2, [110/81], 18:04:29, ospf-UNDERLAY, intra
10.10.10.21/32, ubest/mbest: 1/0
*via 10.1.1.1, Eth1/1, [110/41], 18:04:31, ospf-UNDERLAY, intra
10.10.10.22/32, ubest/mbest: 1/0
*via 10.1.1.3, Eth1/2, [110/41], 18:04:31, ospf-UNDERLAY, intra
Leaf1
sh ip ospf neighbors
OSPF Process ID UNDERLAY VRF default
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
10.10.10.11 1 FULL/DR 18:05:26 10.1.1.0 Eth1/1
10.10.10.12 1 FULL/DR 18:05:21 10.1.2.0 Eth1/2
show ip route ospf-UNDERLAY
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
1.1.1.1/32, ubest/mbest: 2/0
*via 10.1.1.0, Eth1/1, [110/41], 18:05:46, ospf-UNDERLAY, intra
*via 10.1.2.0, Eth1/2, [110/41], 18:05:43, ospf-UNDERLAY, intra
2.2.2.2/32, ubest/mbest: 2/0
*via 10.1.1.0, Eth1/1, [110/81], 18:05:46, ospf-UNDERLAY, intra
*via 10.1.2.0, Eth1/2, [110/81], 18:05:43, ospf-UNDERLAY, intra
10.1.1.2/31, ubest/mbest: 1/0
*via 10.1.1.0, Eth1/1, [110/80], 18:05:46, ospf-UNDERLAY, intra
10.1.2.2/31, ubest/mbest: 1/0
*via 10.1.2.0, Eth1/2, [110/80], 18:05:43, ospf-UNDERLAY, intra
10.10.10.11/32, ubest/mbest: 1/0
*via 10.1.1.0, Eth1/1, [110/41], 18:05:46, ospf-UNDERLAY, intra
10.10.10.12/32, ubest/mbest: 1/0
*via 10.1.2.0, Eth1/2, [110/41], 18:05:43, ospf-UNDERLAY, intra
10.10.10.22/32, ubest/mbest: 2/0
*via 10.1.1.0, Eth1/1, [110/81], 18:05:46, ospf-UNDERLAY, intra
*via 10.1.2.0, Eth1/2, [110/81], 18:05:43, ospf-UNDERLAY, intra
PIM
PIM is active on all interfaces on the fabric, and each switch will have two PIM neighbours.
Spine1
sh ip pim interface brief
PIM Interface Status for VRF "default"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Ethernet1/1 10.1.1.0 10.1.1.1 1 no
Ethernet1/2 10.1.1.2 10.1.1.3 1 no
loopback0 10.10.10.11 10.10.10.11 0 no
loopback1 1.1.1.1 1.1.1.1 0 no
sh ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP Redirect
Priority Capable State Capable
10.1.1.1 Ethernet1/1 15:05:36 00:01:38 1 yes n/a no
10.1.1.3 Ethernet1/2 15:05:37 00:01:43 1 yes n/a no
show ip pim rp
PIM RP Status Information for VRF "default"
BSR disabled
Auto-RP disabled
BSR RP Candidate policy: None
BSR RP policy: None
Auto-RP Announce policy: None
Auto-RP Discovery policy: None
Anycast-RP 1.1.1.1 members:
10.10.10.11* 10.10.10.12
RP: 1.1.1.1*, (0),
uptime: 15:09:18 priority: 255,
RP-source: (local),
group ranges:
224.0.0.0/4
Leaf1
show ip pim interface brief
PIM Interface Status for VRF "default"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Ethernet1/1 10.1.1.1 10.1.1.1 1 no
Ethernet1/2 10.1.2.1 10.1.2.1 1 no
loopback0 10.10.10.21 10.10.10.21 0 no
loopback1 2.2.2.1 2.2.2.1 0 no
show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP Redirect
Priority Capable State Capable
10.1.1.0 Ethernet1/1 18:01:44 00:01:40 1 yes n/a no
10.1.2.0 Ethernet1/2 18:01:42 00:01:35 1 yes n/a no
show ip pim rp
PIM RP Status Information for VRF "default"
BSR disabled
Auto-RP disabled
BSR RP Candidate policy: None
BSR RP policy: None
Auto-RP Announce policy: None
Auto-RP Discovery policy: None
RP: 1.1.1.1, (0),
uptime: 18:01:52 priority: 255,
RP-source: (local),
group ranges:
224.0.0.0/4
Tenant
The tenant is the single business unit, organisation or customer that will reside in this single overlay. There can be multiple overlay networks for multiple tenants. All of them can have overlapping networks, but no communication between them.
The tenants will only be on the leaf switches.
Leaf1
show vrf Tenant-1 detail
VRF-Name: Tenant-1, VRF-ID: 3, State: Up
VPNID: unknown
RD: 10.10.10.21:3
VNI: 10000
Max Routes: 0 Mid-Threshold: 0
Table-ID: 0x80000003, AF: IPv6, Fwd-ID: 0x80000003, State: Up
Table-ID: 0x00000003, AF: IPv4, Fwd-ID: 0x00000003, State: Up
show run vrf tenant-1
interface Vlan10
vrf member Tenant-1
interface Vlan11
vrf member Tenant-1
interface Vlan12
vrf member Tenant-1
interface Vlan13
vrf member Tenant-1
interface Vlan14
vrf member Tenant-1
interface Vlan15
vrf member Tenant-1
vrf context Tenant-1
vni 10000
rd auto
address-family ipv4 unicast
route-target both auto evpn
router bgp 65001
vrf Tenant-1
address-family ipv4 unicast
advertise l2vpn evpn
NVE
The NVE will only be on the leaf switches. It is the virtual interface that is responsible for the VXLAN encaps/decaps. The idea of BGP EVPN is that when the leaf switch that is connected to the endpoints learns of the client, it will forward that over to the other leaf switches, Leaf2 in this case.
show interface nve 1
nve1 is up
admin state is up, Hardware: NVE
MTU 9216 bytes
Encapsulation VXLAN
Auto-mdix is turned off
RX
ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
TX
ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
show nve vrf
VRF-Name VNI Interface Gateway-MAC
------------ ---------- --------- -----------------
Tenant-1 10000 nve1 0c02.30d4.9d07
show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 10000 n/a Up CP L3 [Tenant-1]
nve1 10011 239.0.0.11 Up CP L2 [11]
nve1 10012 239.0.0.12 Up CP L2 [12]
nve1 10013 239.0.0.13 Up CP L2 [13]
nve1 10014 239.0.0.14 Up CP L2 [14]
nve1 10015 239.0.1.2 Up CP L2 [15]
show nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- -------------------------------------- ----- --------- -------- -----------------
nve1 2.2.2.2 Up CP 18:12:45 0c02.3039.6b07
The output below shows the MAC addresses for the endpoint that have been learnt on the network. This will only work if traffic has been received by the leaf switch the endpoints are connected to.
show l2route evpn mac all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending
(S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override
(Pf):Permanently-Frozen, (Orp): Orphan
Topology Mac Address Prod Flags Seq No Next-Hops
----------- -------------- ------ ------------- ---------- ---------------------------------------
10 0c02.3039.6b07 VXLAN Rmac 0 2.2.2.2
11 0050.7966.6800 Local L, 0 Eth1/7
11 0050.7966.6801 Local L, 0 Eth1/6
11 0050.7966.6802 BGP SplRcv 0 2.2.2.2
14 0c02.300c.4c00 BGP SplRcv 0 2.2.2.2
15 0c02.3069.4500 BGP SplRcv 0 2.2.2.2
15 0c02.30ad.7d00 Local L, 0 Eth1/5
The next command is similar to the previous, but contains the IP and MAC of the endpoints.
show l2route evpn mac-ip all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (Ro):Re-Originated (Orp):Orphan
Topology Mac Address Host IP Prod Flags Seq No Next-Hops
----------- -------------- --------------------------------------- ------ ---------- ---------- ---------------------------------------
11 0050.7966.6801 10.0.11.11 HMM L, 0 Local
11 0050.7966.6800 10.0.11.12 HMM L, 0 Local
11 0050.7966.6802 10.0.11.13 BGP -- 0 2.2.2.2
14 0c02.300c.4c00 10.0.14.11 BGP -- 0 2.2.2.2
15 0c02.30ad.7d00 10.0.15.11 HMM L, 0 Local
15 0c02.3069.4500 10.0.15.12 BGP -- 0 2.2.2.2
show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 2198, Local Router ID is 10.10.10.21
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 10.10.10.21:32778 (L2VNI 10011)
*>l[2]:[0]:[0]:[48]:[0050.7966.6800]:[0]:[0.0.0.0]/216
2.2.2.1 100 32768 i
*>l[2]:[0]:[0]:[48]:[0050.7966.6801]:[0]:[0.0.0.0]/216
2.2.2.1 100 32768 i
*>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[0]:[0.0.0.0]/216
2.2.2.2 100 0 i
*>l[2]:[0]:[0]:[48]:[0050.7966.6800]:[32]:[10.0.11.12]/272
2.2.2.1 100 32768 i
*>l[2]:[0]:[0]:[48]:[0050.7966.6801]:[32]:[10.0.11.11]/272
2.2.2.1 100 32768 i
*>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[32]:[10.0.11.13]/272
2.2.2.2 100 0 i
Route Distinguisher: 10.10.10.21:32781 (L2VNI 10014)
*>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[0]:[0.0.0.0]/216
2.2.2.2 100 0 i
*>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[32]:[10.0.14.11]/272
2.2.2.2 100 0 i
Route Distinguisher: 10.10.10.21:32782 (L2VNI 10015)
*>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[0]:[0.0.0.0]/216
2.2.2.2 100 0 i
*>l[2]:[0]:[0]:[48]:[0c02.30ad.7d00]:[0]:[0.0.0.0]/216
2.2.2.1 100 32768 i
*>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[32]:[10.0.15.12]/272
2.2.2.2 100 0 i
*>l[2]:[0]:[0]:[48]:[0c02.30ad.7d00]:[32]:[10.0.15.11]/272
2.2.2.1 100 32768 i
Route Distinguisher: 10.10.10.22:32778
*>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[0]:[0.0.0.0]/216
2.2.2.2 100 0 i
*>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[32]:[10.0.11.13]/272
2.2.2.2 100 0 i
Route Distinguisher: 10.10.10.22:32781
*>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[0]:[0.0.0.0]/216
2.2.2.2 100 0 i
*>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[32]:[10.0.14.11]/272
2.2.2.2 100 0 i
Route Distinguisher: 10.10.10.22:32782
*>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[0]:[0.0.0.0]/216
2.2.2.2 100 0 i
*>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[32]:[10.0.15.12]/272
2.2.2.2 100 0 i
Route Distinguisher: 10.10.10.21:3 (L3VNI 10000)
*>i[2]:[0]:[0]:[48]:[0050.7966.6802]:[32]:[10.0.11.13]/272
2.2.2.2 100 0 i
*>i[2]:[0]:[0]:[48]:[0c02.300c.4c00]:[32]:[10.0.14.11]/272
2.2.2.2 100 0 i
*>i[2]:[0]:[0]:[48]:[0c02.3069.4500]:[32]:[10.0.15.12]/272
2.2.2.2 100 0 i
BGP
BGP or iBGP to be precise is the overlay network. It is responsible for the leaf switches to be able to form the VXLAN tunnels between one another, which allows the endpoints learnt on each leaf switch to be shared.
show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.10.10.21, local AS number 65001
BGP table version is 2198, L2VPN EVPN config peers 1, capable peers 1
21 network entries and 21 paths using 3960 bytes of memory
BGP attribute entries [19/3116], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [1/4]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.10.10.11 4 65001 1991 1329 2198 0 0 18:32:14 6
show system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 15 0c02.3069.4500 static - F F nve-peer1 2.2.2.2
* 11 0050.7966.6800 dynamic 00:07:39 F F Eth1/7
G 11 0c02.30d4.9d07 static - F F sup-eth1(R)
G 10 0c02.30d4.9d07 static - F F sup-eth1(R)
G 13 0c02.30d4.9d07 static - F F sup-eth1(R)
G 12 0c02.30d4.9d07 static - F F sup-eth1(R)
G 15 0c02.30d4.9d07 static - F F sup-eth1(R)
G 14 0c02.30d4.9d07 static - F F sup-eth1(R)
* 11 0050.7966.6801 dynamic 00:07:51 F F Eth1/6
* 11 0050.7966.6802 static - F F nve-peer1 2.2.2.2
* 15 0c02.30ad.7d00 dynamic 00:01:54 F F Eth1/5
* 14 0c02.300c.4c00 static - F F nve-peer1 2.2.2.2
1 1 -12:34:56:78:90:00 - 1