Cisco device configuration tutorials and networking fundamentals

Cisco configuration tutorials

HOME
INTRO
ABOUT
HOMEINTROABOUT
Archive

VXLAN Introduction to Basic Concepts and Configurations

«    2024 Dec 3    »

Table of Contents

Jump to section:

Top of page

What is VXLAN vs VLAN?

VXLAN and VLAN enable network segmentation at different levels of the OSI model. More specifically, a 4-byte VLAN tag is added to an Ethernet frame to segment traffic at Layer-2. On the other hand, an 8-byte VXLAN header is inserted as the payload of a UDP packet, and encapsulates an Ethernet frame (MAC-in-UDP encapsulation). A VXLAN header tunnels an Ethernet frame over an IP routed network.

VLAN vs VXLAN
VLAN VXLAN

4-byte header, provides 4096 theoretical VLAN ID values (there are some reserved VIDs)

8-byte header, provides 16 million VXLAN Virtual Network IDs (VNI)

Within a VLAN Spanning Tree Protocol (STP) is used to avoid network loops

VXLAN tunnels traffic across an IP routed infrastructure which is also called a VXLAN underlay network, STP is not used

Network virtualization with VLAN is less scalable, for improved scalability Q-in-Q can be used which creates double-tagged frames

VXLAN is more scalable than VLAN or Q-in-Q, and VXLAN provides additional functionality when combined with the EVPN control plane. Added benefits of EVPN include dynamic MAC learning (Route Type 2), dynamic ingress replication flood lists (Route Type 3), VXLAN IRB (Route Type 5), ARP suppression, and Anycast Gateway. Furthermore, Q-in-VNI could also be used for added scalability
VXLAN and VLAN comparison

A customer can have a single VXLAN VNI assigned, or a VNI range. For example, Customer A could have the VNI range 5000 - 5010 assigned.

VXLAN configuration - static ingress replication with flood list (IOS XE)

In the following example, the hosts in VLAN 100 are connected over a VXLAN tunnel stretched between VTEP devices R1 and R3. This means, R1 and R2 carry out VXLAN encapsulation and decapsulation to ensure Layer-2 frames are tunneled over the Layer-3 IP network.

In this example, VXLAN static ingress replication is configured without EVPN (BGP). Since ingress replication is used, there is no multicast configuration needed. Each VTEP device is statically configured with one (or optionally more) VXLAN peers that belong to the same virtual network (VNI). The static peer configuration defines where BUM traffic (broadcast, unknown-unicast and multicast traffic) should be flooded, for example ARP request is a type of BUM traffic. This deployment method is also called VXLAN with a static flood list.

When VTEP R1 receives an ARP frame from Host1, it encapsulates the ARP frame in a VXLAN/UDP header, and sends (floods) it using a unicast packet to each VTEP peer that is identified in the flood list.

VXLAN static ingress replication configuration of a flood list on the VTEP

Configuration:

R1 (VTEP) 
    R1#show run int Gi1 | sec int
interface GigabitEthernet1
 description ** to Host1 **
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 10 ethernet
  encapsulation dot1q 100
 !

R1#show run int Gi2 | sec int
interface GigabitEthernet2
 description ** to R2 **
 ip address 10.1.0.1 255.255.255.252
 negotiation auto
 no mop enabled
 no mop sysid

R1#show run int Lo10 | sec int
interface Loopback10
 ip address 1.1.1.1 255.255.255.255

R1#show run int nve10 | sec int
interface nve10
 no ip address
 source-interface Loopback10
 member vni 5000
  ingress-replication 3.3.3.3
 !
 no mop enabled
 no mop sysid

R1#show run | sec ^bridge-domain
bridge-domain 10 
 member vni 5000
 member GigabitEthernet1 service-instance 10

R1#show run | sec ^router      
router ospf 10
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 10.1.0.0 0.0.0.3 area 0
R2 
    R2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to VTEP R1 **
 ip address 10.1.0.2 255.255.255.252
 duplex auto
 speed auto
 media-type rj45

R2#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to VTEP R3 **
 ip address 10.2.0.2 255.255.255.252
 duplex auto
 speed auto
 media-type rj45

R2#show run | sec ^router     
router ospf 10
 router-id 2.2.2.2
 network 10.1.0.0 0.0.0.3 area 0
 network 10.2.0.0 0.0.0.3 area 0
R3 (VTEP) 
    R3#show run int Gi1 | sec int
interface GigabitEthernet1
 description ** to Host2 **
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 10 ethernet
  encapsulation dot1q 100
 !

R3#show run int Gi2 | sec int
interface GigabitEthernet2
 description ** to R2 **
 ip address 10.2.0.1 255.255.255.252
 negotiation auto
 no mop enabled
 no mop sysid

R3#show run int Lo10 | sec int
interface Loopback10
 ip address 3.3.3.3 255.255.255.255

R3#show run int nve10 | sec int
interface nve10
 no ip address
 source-interface Loopback10
 member vni 5000
  ingress-replication 1.1.1.1
 !
 no mop enabled
 no mop sysid

R3#show run  | sec ^bridge-domain
bridge-domain 10 
 member vni 5000
 member GigabitEthernet1 service-instance 10

R3#show run  | sec ^router      
router ospf 10
 router-id 3.3.3.3
 network 3.3.3.3 0.0.0.0 area 0
 network 10.2.0.0 0.0.0.3 area 0
SW1 
    SW1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host1 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW1#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to VTEP R1 **
 switchport trunk allowed vlan 100
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
SW2 
    SW2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host2 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW2#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to VTEP R3 **
 switchport trunk allowed vlan 100
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
Host1 
    Host1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW1 **
 mac-address 0000.0000.0001
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45

R1#show nve peers
Interface  VNI      Type Peer-IP          RMAC/Num_RTs   eVNI     state flags UP time
nve10      5000     L2DP 3.3.3.3              ----       -          --   -/-              « VXLAN static peer is R3




R1#show bridge-domain 10
Bridge-domain 10 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
Maximum address limit: 65536
    GigabitEthernet1 service instance 10
    vni 5000
   AED MAC address    Policy  Tag       Age  Pseudoport
   0   0000.0000.0002 forward dynamic   297  nve10.VNI5000, VxLAN            « MAC address learned through VXLAN tunnel
                                             src: 1.1.1.1 dst: 3.3.3.3
   0   0000.0000.0001 forward dynamic   297  GigabitEthernet1.EFP10




R1#show nve interface nve 10
Interface: nve10, State: Admin Up, Oper Up, Encapsulation: Vxlan,            « VXLAN virtual interface is operational
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2CP 0 L2DP 1
source-interface: Loopback10 (primary:1.1.1.1 vrf:0)
tunnel interface: Tunnel0




Host1#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/6/7 ms         « Host1 can reach Host2 through VXLAN tunnel




Host1#show arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.1.1             -   0000.0000.0001  ARPA   GigabitEthernet0/0
Internet  192.168.1.2           107   0000.0000.0002  ARPA   GigabitEthernet0/0          « Host1 learns MAC address of Host2

VXLAN configuration - static ingress replication with flood list (NX-OS)

In the following example scenario, VXLAN is configured between the VTEP devices SW1 and SW3 using static ingress replication. The same principles apply as in the previous section (VXLAN static ingress replication on IOS XE).

With NX-OS the required features need to be enabled before they can be applied. Accordingly, on the VTEP devices the following configuration is added to enable OSPF and VXLAN.

SW1# 
SW1# configure
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)# 
SW1(config)# feature ospf
SW1(config)# feature vn-segment-vlan-based
SW1(config)# feature nv overlay
SW1(config)# 
SW1(config)# exit
SW1#
VXLAN flood list configuration static ingress replication

Configuration:

SW1 (VTEP) 
    SW1# show run | sec feature
feature ospf
feature vn-segment-vlan-based
feature nv overlay

SW1# show run vlan | sec vlan
!Command: show running-config vlan
vlan 1,100
vlan 100
  name VLAN100
  vn-segment 1000

SW1# show run int Eth1/1 | sec int
!Command: show running-config interface Ethernet1/1
interface Ethernet1/1
  description ** to Host1 **
  switchport
  switchport access vlan 100
  no shutdown

SW1# show run int Eth1/2 | sec int
!Command: show running-config interface Ethernet1/2
interface Ethernet1/2
  description ** to SW2 **
  ip address 10.1.0.1/30
  ip router ospf 10 area 0.0.0.0
  no shutdown

SW1# show run int Lo10 | sec int
!Command: show running-config interface loopback10
interface loopback10
  ip address 1.1.1.1/32
  ip router ospf 10 area 0.0.0.0

SW1# show run int nve1 | sec int
!Command: show running-config interface nve1
interface nve1
  no shutdown
  source-interface loopback10
  member vni 1000
    ingress-replication protocol static
      peer-ip 3.3.3.3

SW1# show run ospf | sec ^router
router ospf 10
  router-id 1.1.1.1
SW2 
    SW2# show run | sec feature
feature ospf

SW2# show run int Eth1/1 | sec int
!Command: show running-config interface Ethernet1/1
interface Ethernet1/1
  description ** to VTEP SW1 **
  ip address 10.1.0.2/30
  ip router ospf 10 area 0.0.0.0
  no shutdown

SW2# show run int Eth1/2 | sec int
!Command: show running-config interface Ethernet1/2
interface Ethernet1/2
  description ** to VTEP SW3 **
  ip address 10.2.0.2/30
  ip router ospf 10 area 0.0.0.0
  no shutdown

SW2# show run ospf | sec ^router
router ospf 10
  router-id 2.2.2.2
SW3 (VTEP) 
    SW3# show run | sec feature
feature ospf
feature vn-segment-vlan-based
feature nv overlay

SW3# show run vlan | sec vlan
!Command: show running-config vlan
vlan 1,100
vlan 100
  vn-segment 1000

SW3# show run int Eth1/1 | sec int
!Command: show running-config interface Ethernet1/1
interface Ethernet1/1
  description ** to Host2 **
  switchport
  switchport access vlan 100
  no shutdown

SW3# show run int Eth1/2 | sec int
!Command: show running-config interface Ethernet1/2
interface Ethernet1/2
  description ** to SW2 **
  ip address 10.2.0.1/30
  ip router ospf 10 area 0.0.0.0
  no shutdown

SW3# show run int Lo10 | sec int
!Command: show running-config interface loopback10
interface loopback10
  ip address 3.3.3.3/32
  ip router ospf 10 area 0.0.0.0

SW3# show run int nve1 | sec int
!Command: show running-config interface nve1
interface nve1
  no shutdown
  source-interface loopback10
  member vni 1000
    ingress-replication protocol static
      peer-ip 1.1.1.1

SW3# show run ospf | sec ^router
router ospf 10
  router-id 3.3.3.3
Host1 
    Host1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to VTEP SW1 **
 mac-address 0000.0000.0001
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45

SW1# show nve vni 1000
Codes: CP - Control Plane        DP - Data Plane          
       UC - Unconfigured         SA - Suppress ARP        
       SU - Suppress Unknown Unicast 
       Xconn - Crossconnect      
       MS-IR - Multisite Ingress Replication
 
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      1000     UnicastStatic     Up    DP   L2 [100]                            « Static ingress replication is configured




SW1# show nve peers 
Interface Peer-IP                                 State LearnType Uptime   Route
r-Mac       
--------- --------------------------------------  ----- --------- -------- -----
------------
nve1      3.3.3.3                                 Up    DP        02:11:10 n/a      « Peer address is that of SW3




SW1# show nve interface nve1
Interface: nve1, State: Up, encapsulation: VXLAN                                    « VXLAN virtual interface is operational
 VPC Capability: VPC-VIP-Only [not-notified]
 Local Router MAC: 5204.796c.1b08
 Host Learning Mode: Data-Plane
 Source-Interface: loopback10 (primary: 1.1.1.1, secondary: 0.0.0.0)




SW1# show mac address-table 
Legend: 
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*  100     0000.0000.0001   dynamic  0         F      F    Eth1/1
*  100     0000.0000.0002   dynamic  0         F      F    nve1(3.3.3.3)         « MAC address of Host2 learned through VXLAN tunnel
G    -     5204.796c.1b08   static   -         F      F    sup-eth1(R)




Host1#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/9/19 ms            « Host1 can reach Host2 in VLAN 100 through the VXLAN tunnel

VXLAN configuration - multicast core PIM Sparse Mode with BSR (IOS XE)

In the following example, a VXLAN overlay tunnel is configured between VTEP R1 and R5. The VTEP devices use multicast with PIM Sparse Mode to join the group address 239.1.1.1, so they can encapsulate and forward VXLAN BUM traffic (broadcast, unknown-unicast and multicast traffic).

This means, Host1 sends a broadcast ARP request if it wants to ping Host2. VTEP R1 receives the ARP request from Host1, and encapsulates the ARP request with a VXLAN header, and then sends it using a PIM Register message to Rendezvous Point (RP) R6. Since VTEP R5 is listening to the multicast group address 239.1.1.1 (served by the RP R6), it receives the ARP request. R5 decapsulates the ARP request and forwards it to Host2. When Host2 replies, R5 sends an encapsulated ARP reply directly to R1 in a unicast IP packet.

When configuring VXLAN on IOS XE the default UDP port 4789 is used, however this can be changed in global configuration mode.

VXLAN configuration multicast mode PIM SM with Bootstrap Router Rendezvous Point

Configuration:

R1 (VTEP) 
    R1#show run int Gi1 | sec int
interface GigabitEthernet1
 description ** to Host1 **
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 10 ethernet
  encapsulation dot1q 100
 !

R1#show run int Gi2 | sec int
interface GigabitEthernet2
 description ** to R2 **
 ip address 10.1.0.1 255.255.255.252
 ip pim sparse-mode
 negotiation auto
 no mop enabled
 no mop sysid

R1#show run int Lo10 | sec int
interface Loopback10
 ip address 1.1.1.1 255.255.255.255
 ip pim sparse-mode

R1#show run int nve 10 | sec int
interface nve10
 no ip address
 source-interface Loopback10
 member vni 5000 mcast-group 239.1.1.1
 no mop enabled
 no mop sysid

R1#show run | sec ^bridge
bridge-domain 10 
 member vni 5000
 member GigabitEthernet1 service-instance 10

R1#show run | sec ^ip multicast
ip multicast-routing distributed

R1#show run | sec ^router 
router ospf 10
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 10.1.0.0 0.0.0.3 area 0
R2 
    R2#show run | sec ^ip multi
ip multicast-routing

R2#show run int Gi0/0 | sec int 
interface GigabitEthernet0/0
 description ** to VTEP R1 **
 ip address 10.1.0.2 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R2#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to R3 **
 ip address 10.2.0.1 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R2#show run | sec ^router  
router ospf 10
 router-id 2.2.2.2
 network 10.1.0.0 0.0.0.3 area 0
 network 10.2.0.0 0.0.0.3 area 0
R3 
    R3#show run | sec ^ip multi
ip multicast-routing

R3#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to R2 **
 ip address 10.2.0.2 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R3#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to R4 **
 ip address 10.4.0.1 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R3#show run int Gi0/2 | sec int
interface GigabitEthernet0/2
 description ** to BSR/RP R6 **
 ip address 10.3.0.1 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R3#show run | sec ^router
router ospf 10
 router-id 3.3.3.3
 network 10.2.0.0 0.0.0.3 area 0
 network 10.3.0.0 0.0.0.3 area 0
 network 10.4.0.0 0.0.0.3 area 0
R4 
    R4#show run | sec ^ip multi
ip multicast-routing

R4#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to VTEP R5 **
 ip address 10.5.0.2 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R4#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to R3 **
 ip address 10.4.0.2 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R4#show run | sec ^router
router ospf 10
 router-id 4.4.4.4
 network 10.4.0.0 0.0.0.3 area 0
 network 10.5.0.0 0.0.0.3 area 0
R5 (VTEP) 
    R5#show run int Gi1 | sec int
interface GigabitEthernet1
 description ** to Host2 **
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 10 ethernet
  encapsulation dot1q 100
 !

R5#show run int Gi2 | sec int
interface GigabitEthernet2
 description ** to R4 **
 ip address 10.5.0.1 255.255.255.252
 ip pim sparse-mode
 negotiation auto
 no mop enabled
 no mop sysid

R5#show run int Lo10 | sec int
interface Loopback10
 ip address 5.5.5.5 255.255.255.255
 ip pim sparse-mode

R5#show run int nve 10 | sec int
interface nve10
 no ip address
 source-interface Loopback10
 member vni 5000 mcast-group 239.1.1.1
 no mop enabled
 no mop sysid

R5#show run | sec ^bridge
bridge-domain 10 
 member vni 5000
 member GigabitEthernet1 service-instance 10

R5#show run | sec ^ip multicast 
ip multicast-routing distributed

R5#show run | sec ^router
router ospf 10
 router-id 5.5.5.5
 network 5.5.5.5 0.0.0.0 area 0
 network 10.5.0.0 0.0.0.3 area 0
R6 (BSR/RP) 
    R6#show run | sec ^ip multi
ip multicast-routing

R6#show run | sec ^ip pim
ip pim bsr-candidate Loopback10 0
ip pim rp-candidate Loopback10

R6#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to R3 **
 ip address 10.3.0.2 255.255.255.252
 ip pim sparse-mode
 duplex auto
 speed auto
 media-type rj45

R6#show run int Lo10 | sec int
interface Loopback10
 ip address 6.6.6.6 255.255.255.255
 ip pim sparse-mode

R6#show run | sec ^router
router ospf 10
 router-id 6.6.6.6
 network 6.6.6.6 0.0.0.0 area 0
 network 10.3.0.0 0.0.0.3 area 0
SW1 
    SW1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host1 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW1#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to VTEP R1 **
 switchport trunk allowed vlan 100
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
SW2 
    SW2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host2 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW2#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to VTEP R5 **
 switchport trunk allowed vlan 100
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
Host1 
    Host1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW1 **
 mac-address 0000.0000.0001
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
Host2 
    Host2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW2 **
 mac-address 0000.0000.0002
 ip address 192.168.1.2 255.255.255.0
 duplex auto
 speed auto
 media-type rj45

R1#show nve vni 5000
Interface  VNI        Multicast-group VNI state  Mode  BD    cfg vrf                      
nve10      5000       239.1.1.1       Up         L2DP  10    CLI N/A         « VXLAN VNI 5000 is configured




R1#show nve vni interface nve 10 detail
Interface  VNI        Multicast-group VNI state  Mode  BD    cfg vrf                      
nve10      5000       239.1.1.1       Up         L2DP  10    CLI N/A         « VXLAN multicast core with group address 239.1.1.1 is configured
VNI Detailed statistics:
   Pkts In   Bytes In   Pkts Out  Bytes Out
        16       1703         22       2135




R1#show bridge-domain 10                   
Bridge-domain 10 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
Maximum address limit: 65536
    GigabitEthernet1 service instance 10
    vni 5000
   AED MAC address    Policy  Tag       Age  Pseudoport
   0   0000.0000.0002 forward dynamic   212  nve10.VNI5000, VxLAN            « MAC address of Host2 learned through VXLAN tunnel
                                             src: 1.1.1.1 dst: 5.5.5.5
   0   0000.0000.0001 forward dynamic   212  GigabitEthernet1.EFP10




R1#show ip mroute | beg Int    
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.1.1.1), 01:07:43/00:02:07, RP 6.6.6.6, flags: SJCx         « PIM Sparse Mode is used with RP R6
  Incoming interface: GigabitEthernet2, RPF nbr 10.1.0.2
  Outgoing interface list:
    Tunnel0, Forward/Sparse-Dense, 01:07:43/00:02:20

(*, 224.0.1.40), 01:07:44/00:02:22, RP 0.0.0.0, flags: DCL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    Loopback10, Forward/Sparse, 01:07:42/00:02:18




R1#show ip pim rp mapping
PIM Group-to-RP Mappings

Group(s) 224.0.0.0/4
  RP 6.6.6.6 (?), v2
    Info source: 6.6.6.6 (?), via bootstrap, priority 0, holdtime 150       « RP address is learned through BSR
         Uptime: 01:07:00, expires: 00:01:41




Host1#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/8/9 ms        « Host1 can reach Host2 in VLAN 100 over VXLAN




Host1#show arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.1.1             -   0000.0000.0001  ARPA   GigabitEthernet0/0
Internet  192.168.1.2            67   0000.0000.0002  ARPA   GigabitEthernet0/0    « Host1 receives Host2's MAC address with ARP through VXLAN

VXLAN configuration - multicast core PIM Sparse Mode static RP (NX-OS)

In the following scenario, a VXLAN overlay network transports traffic for VLAN 100 (VNI 1000) and VLAN 200 (VNI 2000). The VTEP devices SW2 and SW4 use PIM Sparse Mode with a static RP address configuration. The RP is SW3.

Required features need to be specifically enabled on the devices running NX-OS. This means, the VTEP SW2 and SW4 have the features for PIM (multicast), OSPF, VLAN-based VN-Segment, and for VXLAN enabled in global configuration mode. If these features are not enabled, the necessary technology-specific configuration commands are not available.

Only the VTEP devices SW2 and SW4 learn end-host MAC addresses. Due to the tunneling mechanism enabled by the VXLAN/UDP encapsulation, SW3 just passes traffic between the VTEPs. The core device SW3 does not need to support VXLAN encapsulation/decapsulation. This is in contrast to Q-in-Q, where each core switch also learns end-host MAC adddresses. Further, notice this example uses the same IP address and MAC addresses in both VLANs, however they are treated independently due to per-VLAN MAC learning (IVL).

VXLAN multicast core configuration with static RP

Configuration:

SW1 
    SW1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host1 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW1#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to Host2 **
 switchport access vlan 200
 switchport mode access
 negotiation auto

SW1#show run int Gi0/2 | sec int
interface GigabitEthernet0/2
 description ** to VTEP SW2 **
 switchport trunk allowed vlan 100,200
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
SW2 (VTEP) 
    SW2# show run | sec feature
feature ospf
feature pim
feature vn-segment-vlan-based
feature nv overlay

SW2# show run | grep rp-address
ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4
  
SW2# show run vlan | sec vlan
!Command: show running-config vlan
vlan 1,100,200
vlan 100
  name VLAN100
  vn-segment 1000
vlan 200
  name VLAN200
  vn-segment 2000

SW2# show run int Eth1/1 | sec int
!Command: show running-config interface Ethernet1/1
interface Ethernet1/1
  description ** trunk to SW1 **
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 100,200
  no shutdown

SW2# show run int Eth1/2 | sec int
!Command: show running-config interface Ethernet1/2
interface Ethernet1/2
  description ** to RP R3 **
  ip address 10.1.0.1/30
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

SW2# show run int Lo10 | sec int
!Command: show running-config interface loopback10
interface loopback10
  ip address 2.2.2.2/32
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode

SW2# show run ospf | sec ^router
router ospf 10
  router-id 2.2.2.2

SW2# show run int nve1 | sec int
!Command: show running-config interface nve1
interface nve1
  no shutdown
  source-interface loopback10
  member vni 1000
    mcast-group 239.1.1.1
  member vni 2000
    mcast-group 239.2.2.2
SW3 
    SW3# show run | sec feature
feature ospf
feature pim

SW3# show run | grep rp-address
ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4

SW3# show run ospf | sec ^router
router ospf 10
  router-id 3.3.3.3

SW3# show run int Eth1/1 | sec int
!Command: show running-config interface Ethernet1/1
interface Ethernet1/1
  description ** to VTEP SW2 **
  ip address 10.1.0.2/30
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

SW3# show run int Eth1/2 | sec int
!Command: show running-config interface Ethernet1/2
interface Ethernet1/2
  description ** to VTEP SW4 **
  ip address 10.2.0.2/30
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

SW3# show run int Lo10 | sec int
!Command: show running-config interface loopback10
interface loopback10
  ip address 3.3.3.3/32
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode
SW4 (VTEP) 
    SW4# show run | sec feature
feature ospf
feature pim
feature vn-segment-vlan-based
feature nv overlay

SW4# show run | grep rp-address
ip pim rp-address 3.3.3.3 group-list 224.0.0.0/4

SW4# show run vlan | sec vlan
!Command: show running-config vlan
vlan 1,100,200
vlan 100
  name VLAN100
  vn-segment 1000
vlan 200
  name VLAN200
  vn-segment 2000

SW4# show run int Eth1/1 | sec int
!Command: show running-config interface Ethernet1/1
interface Ethernet1/1
  description ** trunk to SW5 **
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 100,200
  no shutdown

SW4# show run int Eth1/2 | sec int
!Command: show running-config interface Ethernet1/2
interface Ethernet1/2
  description ** to RP SW3 **
  ip address 10.2.0.1/30
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

SW4# show run int Lo10 | sec int
!Command: show running-config interface loopback10
interface loopback10
  ip address 4.4.4.4/32
  ip router ospf 10 area 0.0.0.0
  ip pim sparse-mode

SW4# show run ospf | sec ^router
router ospf 10
  router-id 4.4.4.4

SW4# show run int nve1 | sec int
!Command: show running-config interface nve1
interface nve1
  no shutdown
  source-interface loopback10
  member vni 1000
    mcast-group 239.1.1.1
  member vni 2000
    mcast-group 239.2.2.2
SW5 
    SW5#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host3 **
 switchport access vlan 200
 switchport mode access
 negotiation auto

SW5#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to Host4 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW5#show run int Gi0/2 | sec int
interface GigabitEthernet0/2
 description ** to VTEP SW4 **
 switchport trunk allowed vlan 100,200
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
Host1 
    Host1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW1 **
 mac-address 0000.0000.0001
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
Host2 
    Host2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW1 **
 mac-address 0000.0000.0001
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45

SW2# show nve vni
Codes: CP - Control Plane        DP - Data Plane          
       UC - Unconfigured         SA - Suppress ARP        
       SU - Suppress Unknown Unicast 
       Xconn - Crossconnect      
       MS-IR - Multisite Ingress Replication
 
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      1000     239.1.1.1         Up    DP   L2 [100]                       « VNI 1000 uses the multicast group 239.1.1.1       
nve1      2000     239.2.2.2         Up    DP   L2 [200]




SW2# show nve interface
Interface: nve1, State: Up, encapsulation: VXLAN                               « VXLAN NVE (network virtual interface) state is Up
 VPC Capability: VPC-VIP-Only [not-notified]
 Local Router MAC: 520f.fd09.1b08
 Host Learning Mode: Data-Plane
 Source-Interface: loopback10 (primary: 2.2.2.2, secondary: 0.0.0.0)




SW2# show nve peer
Interface Peer-IP                                 State LearnType Uptime   Route
r-Mac       
--------- --------------------------------------  ----- --------- -------- -----
------------
nve1      4.4.4.4                                 Up    DP        00:40:19 n/a           « VXLAN peer is SW4




SW2# show nve internal pim-cache | section RP
   RP DB         : 0x1
   RP Grange DB  : 0x1
 RP Database for VRF : default
     RP : 3.3.3.3                           « PIM Sparse Mode Rendezvous Point (RP) is SW3
        DF Bits 
          None 
        Peer DF Bits 
          None 
 RP Grange Database for VRF : default
       RP: 3.3.3.3  static                 « PIM RP is statically configured (not BSR/Bootstrap)




Host1#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/10 ms             « Host1 in VLAN 100 can reach Host2 in same VLAN 100




Host1#show arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.1.1             -   0000.0000.0001  ARPA   GigabitEthernet0/0
Internet  192.168.1.2           158   0000.0000.0002  ARPA   GigabitEthernet0/0           « Host1 learns MAC address of Host2




SW2# show mac address-table 
Legend: 
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*  100     0000.0000.0001   dynamic  0         F      F    Eth1/1
*  100     0000.0000.0002   dynamic  0         F      F    nve1(4.4.4.4)                  « per-VLAN MAC address learning on VTEP
*  200     0000.0000.0001   dynamic  0         F      F    Eth1/1
*  200     0000.0000.0002   dynamic  0         F      F    nve1(4.4.4.4)
G    -     520f.fd09.1b08   static   -         F      F    sup-eth1(R)

Q-in-Q with VXLAN configuration, Q-in-VNI (IOS XE)

In the following example scenario, Q-in-Q is combined with VXLAN (without EVPN) for improved scalability. With this method, a single VNI can carry traffic for overlapping Customer-VLANs (C-VLAN). Multiple C-VLANs are mapped to a single Service-VLAN (S-VLAN) using Q-in-Q double-tagged frames. Importantly, the VTEP maps the S-VLAN to the VXLAN VNI. Since an S-VLAN carries traffic for multiple C-VLANs, the solution improves scalability.

Specifically in this example, C-VLAN 100 and C-VLAN 200 are mapped to S-VLAN 300 on SW1. So between SW1 and the VTEP R1 there are two VLAN tags inserted into frames. VTEP R1 encapsulates the double-tagged frame in a VXLAN header (MAC-in-UDP) and tunnels it to VTEP R3. The double-tagged frame loses its VXLAN header (decapsulation) on VTEP R3, and is forwarded to SW2.

VXLAN with Q-in-Q configuration Q-in-VNI

Configuration:

R1 (VTEP) 
    R1#show run int Gi1 | sec int
interface GigabitEthernet1
 description ** to SW1 **
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 10 ethernet
  encapsulation dot1q 300
 !

R1#show run int Gi2 | sec int
interface GigabitEthernet2
 description ** to R2 **
 ip address 10.1.0.1 255.255.255.252
 ip ospf network point-to-point
 negotiation auto
 no mop enabled
 no mop sysid

R1#show run int Lo10 | sec int
interface Loopback10
 ip address 1.1.1.1 255.255.255.255

R1#show run int nve10 | sec int
interface nve10
 no ip address
 source-interface Loopback10
 member vni 5000
  ingress-replication 3.3.3.3
 !
 no mop enabled
 no mop sysid

R1#show run | sec ^bridge-domain
bridge-domain 10 
 member vni 5000
 member GigabitEthernet1 service-instance 10

R1#show run | sec ^router
router ospf 10
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 10.1.0.0 0.0.0.3 area 0
R2 
    R2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to VTEP R1 **
 ip address 10.1.0.2 255.255.255.252
 ip ospf network point-to-point
 duplex auto
 speed auto
 media-type rj45

R2#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to VTEP R3 **
 ip address 10.2.0.2 255.255.255.252
 ip ospf network point-to-point
 duplex auto
 speed auto
 media-type rj45

R2#show run | sec ^router
router ospf 10
 router-id 2.2.2.2
 network 10.1.0.0 0.0.0.3 area 0
 network 10.2.0.0 0.0.0.3 area 0
R3 (VTEP) 
    R3#show run int Gi1 | sec int
interface GigabitEthernet1
 description ** to SW2 **
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 10 ethernet
  encapsulation dot1q 300
 !

R3#show run int Gi2 | sec int
interface GigabitEthernet2
 description ** to R2 **
 ip address 10.2.0.1 255.255.255.252
 ip ospf network point-to-point
 negotiation auto
 no mop enabled
 no mop sysid

R3#show run int Lo10 | sec int
interface Loopback10
 ip address 3.3.3.3 255.255.255.255

R3#show run int nve10 | sec int
interface nve10
 no ip address
 source-interface Loopback10
 member vni 5000
  ingress-replication 1.1.1.1
 !
 no mop enabled
 no mop sysid

R3#show run  | sec ^bridge-domain
bridge-domain 10 
 member vni 5000
 member GigabitEthernet1 service-instance 10

R3#show run  | sec ^router
router ospf 10
 router-id 3.3.3.3
 network 3.3.3.3 0.0.0.0 area 0
 network 10.2.0.0 0.0.0.3 area 0
SW1 
    SW1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW3 **
 switchport access vlan 300
 switchport mode dot1q-tunnel
 negotiation auto
 no cdp enable

SW1#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to SW4 **
 switchport access vlan 300
 switchport mode dot1q-tunnel
 negotiation auto
 no cdp enable

SW1#show run int Gi0/2 | sec int
interface GigabitEthernet0/2
 description ** to VTEP R1 **
 switchport trunk allowed vlan 300
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
SW2 
    SW2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW5 **
 switchport access vlan 300
 switchport mode dot1q-tunnel
 negotiation auto
 no cdp enable

SW2#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to SW6 **
 switchport access vlan 300
 switchport mode dot1q-tunnel
 negotiation auto
 no cdp enable

SW2#show run int Gi0/2 | sec int
interface GigabitEthernet0/2
 description ** to VTEP R3 **
 switchport trunk allowed vlan 300
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
SW3 
    SW3#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host1 **
 switchport access vlan 100
 switchport mode access
 negotiation auto

SW3#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to SW1 **
 switchport trunk allowed vlan 100
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
SW4 
    SW4#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to Host2 **
 switchport access vlan 200
 switchport mode access
 negotiation auto

SW4#show run int Gi0/1 | sec int
interface GigabitEthernet0/1
 description ** to SW1 **
 switchport trunk allowed vlan 200
 switchport trunk encapsulation dot1q
 switchport mode trunk
 negotiation auto
Host1 
    Host1#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW3 **
 mac-address 0000.0000.0001
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
Host2 
    Host2#show run int Gi0/0 | sec int
interface GigabitEthernet0/0
 description ** to SW4 **
 mac-address 0000.0000.0002
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45

R1#show bridge-domain 10
Bridge-domain 10 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 300 second(s)
Maximum address limit: 65536
    GigabitEthernet1 service instance 10
    vni 5000                                        « Two C-VLANs are transported using VNI 5000
   AED MAC address    Policy  Tag       Age  Pseudoport
   0   0000.0000.0002 forward dynamic   296  GigabitEthernet1.EFP10
   0   0000.0000.0004 forward dynamic   296  nve10.VNI5000, VxLAN 
                                             src: 1.1.1.1 dst: 3.3.3.3
   0   0000.0000.0001 forward dynamic   290  GigabitEthernet1.EFP10
   0   0000.0000.0003 forward dynamic   290  nve10.VNI5000, VxLAN 
                                             src: 1.1.1.1 dst: 3.3.3.3




R1#show nve interface nve 10
Interface: nve10, State: Admin Up, Oper Up, Encapsulation: Vxlan,
BGP host reachability: Disable, VxLAN dport: 4789
VNI number: L3CP 0 L2CP 0 L2DP 1
source-interface: Loopback10 (primary:1.1.1.1 vrf:0)
tunnel interface: Tunnel0




R1#show nve peers
Interface  VNI      Type Peer-IP          RMAC/Num_RTs   eVNI     state flags UP time
nve10      5000     L2DP 3.3.3.3              ----       -          --   -/-            « Remote VTEP is R3




Host1#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/11 ms                  « Host1 pings Host3




Host1#show arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.1.1             -   0000.0000.0001  ARPA   GigabitEthernet0/0
Internet  192.168.1.2             9   0000.0000.0003  ARPA   GigabitEthernet0/0         « Host1 learns Host3 MAC address through VXLAN tunnel




Host1#trace 192.168.1.2 probe 1
Type escape sequence to abort.
Tracing the route to 192.168.1.2
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.1.2 8 msec

Download section

Disclaimer: You download and use files from networkstudysite.com at your own risk.

Lab YAML files:

Packet captures:

Text files with configurations:

How to use these files:

With YAML files you can easily recreate the example labs on networkstudysite.com. Learn more in this short guide.

Explore 313 packet captures

You can find more information at the following external links:

By clicking on the links below, you are leaving the networkstudysite.com website.

Cisco - Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 10.3(x)

Cisco - Configure VXLAN Feature on Cisco IOS XE Devices

Cisco Press - Introduction to VXLAN

Cisco Learning Network - Virtual Extensible LAN (VXLAN) Overview

RFC 7348 - Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks

Disclaimer:

Use at your own risk: networkstudysite.com makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information found on this website. Full disclaimer on the About page.

Privacy Policy:

networkstudysite.com does not install browser cookies to collect or store your data.

Thank you for your interest in this blog post!

Looking for something else? View infographics, explore the archives or read the recommended posts below:

- View infographics -
- Discover archives -

Recommended posts:

» EVPN-VXLAN L2VNI Configuration with MP-BGP Control Plane

» EVPN-VXLAN L3VNI Configuration & Inter-VNI Routing

» Ethernet VPN Introduction and EVPN-MPLS Concepts Explained
Sitemap
About
Disclaimer
Privacy Policy
Arrow pointing to the top