BGP Additional Paths Explained with MPLS L3VPN Deployment

« 2023 June 28 »

What is BGP Additional Paths?

The BGP Additional Paths feature is a BGP Capability advertised between a Route Reflector (RR) and an RR client. When configured, the RR can send multiple unique iBGP paths (next-hop IPs) pointing to a single destination prefix. The RR client receives these paths and can calculate the best path to the destination. The benefit is that the RR client makes the best path calculation, and not the RR.

BGP Additional Paths packet capture OPEN message Capability

The packet capture highlights an important aspect of the BGP Additional Paths feature mentioned in RFC 7911. Namely, that the capability needs to be negotiated between BGP routers during establishment of a neighborship. The corresponding section of the RFC is quoted below.

For a BGP speaker to be able to send multiple paths to its peer, that BGP speaker MUST advertise the ADD-PATH Capability with the Send/Receive field set to either 2 or 3, and MUST receive from its peer the ADD-PATH Capability with the Send/Receive field set to either 1 or 3, for the corresponding <AFI, SAFI>.

RFC 7911, Advertisement of Multiple Paths in BGP

BGP Add-Path is especially useful considering the default behaviour of an RR. As per the default, an RR calculates its own best path to a destination, and advertises this single best path to its RR clients. This is called path hiding, namely the RR hides all paths that it thinks are not the best path. But herein is the issue, the path advertised by the RR may not be the best path available for all of its particular RR clients.

Preserve BGP routes with Route Reflector deployments
Method Description
Additional Paths This feature is configured on the RR and also on the RR client. The RR is enabled to advertise multiple available iBGP paths to destination prefixes, instead of the default behaviour which is path hiding (advertising only single path). As a result, the RR clients can receive multiple paths to a destination prefix, and can calculate their own best path through the nearest egress PE router.
Best External This feature is configured on the RR client only. In a Service Provider Autonomous System (AS) there may be primary and secondary egress Provider Edge (PE) routers to reach a neighboring AS. In order to reach the neighboring AS, if the primary PE advertises an iBGP route with a higher local preference value, the secondary PE receives this information. Now, the secondary PE will prefer the route through iBGP (higher local preference) instead of the eBGP route. As a result, the secondary PE router stops advertising its eBGP route to the RR, as it already has an iBGP route with a higher local preference value to the neighboring AS. To avoid this scenario, the BGP Best External feature makes the secondary PE advertise its eBGP route to the RR.
Diverse-Path This feature is configured on the RR and is commonly used with a Shadow RR. A Shadow RR is a backup RR that only advertises an alternative path (a backup, diverse path). This means, the Shadow RR does not advertise the best path, but instead its RR clients only receive the alternative (second best) path to a particular destination. The Shadow RR is also called a Diverse-Path Route Reflector.

How to configure BGP Additional Paths?

The BGP Additional Paths feature needs to be configured on the RR and also on the RR Client. The following example configurations (top RR, bottom RR Client) deal with IPv4 BGP prefixes using Cisco IOS.

RouteReflector#show run | sec ^router bgp 
router bgp 65010
 bgp router-id 7.7.7.7
 bgp log-neighbor-changes
 neighbor 10.0.1.2 remote-as 65010
 neighbor 10.0.4.2 remote-as 65010
 !
 address-family ipv4
  bgp additional-paths select all
  bgp additional-paths send
  neighbor 10.0.1.2 activate
  neighbor 10.0.1.2 route-reflector-client
  neighbor 10.0.1.2 next-hop-self
  neighbor 10.0.1.2 advertise additional-paths all
  neighbor 10.0.4.2 activate
  neighbor 10.0.4.2 route-reflector-client
  neighbor 10.0.4.2 next-hop-self
 exit-address-family


##################################################################
##                                                              ##
##                                                              ##
##   Above is Route Reflector, below is RR Client (IPv4, IOS)   ##
##                                                              ##
##                                                              ##
##################################################################


RRClient#show run | sec ^router bgp
router bgp 65010
 bgp router-id 2.2.2.2
 bgp log-neighbor-changes
 neighbor 10.0.1.1 remote-as 65010
 neighbor 10.1.0.2 remote-as 65001
 !
 address-family ipv4
  bgp additional-paths receive
  neighbor 10.0.1.1 activate
  neighbor 10.0.1.1 next-hop-self
  neighbor 10.1.0.2 activate
 exit-address-family

Meanwhile, the following example configurations (top RR, bottom RR Client) deal with VPNv4 BGP prefixes which are used in L3VPN MPLS. Cisco IOS XR enables configuration of BGP Add-Path in a L3VPN MPLS architecture.

RP/0/0/CPU0:RouteReflector#show run router bgp
Wed Jun 28 14:45:11.440 UTC
router bgp 65100
 bgp router-id 12.12.12.12
 address-family vpnv4 unicast
  additional-paths send
  additional-paths selection route-policy BGP-ADD-PATH
 !
 neighbor 5.5.5.5
  remote-as 65100
  capability additional-paths send
  update-source Loopback10
  address-family vpnv4 unicast
   route-reflector-client
  !
 !
 neighbor 6.6.6.6
  remote-as 65100
  update-source Loopback10
  address-family vpnv4 unicast
   route-reflector-client
  !
 !
 neighbor 7.7.7.7
  remote-as 65100
  update-source Loopback10
  address-family vpnv4 unicast
   route-reflector-client
  !
 !
!

RP/0/0/CPU0:RouteReflector#show run route-policy BGP-ADD-PATH 
Wed Jun 28 14:45:39.898 UTC
route-policy BGP-ADD-PATH
  set path-selection all advertise
end-policy
!


######################################################################
##                                                                  ##
##                                                                  ##
##   Above is Route Reflector, below is RR Client (VPNv4, IOS XR)   ##
##                                                                  ##
##                                                                  ##
######################################################################


RP/0/0/CPU0:RRClient#show run router bgp
Wed Jun 28 14:44:25.945 UTC
router bgp 65100
 bgp router-id 5.5.5.5
 address-family ipv4 unicast
 !
 address-family vpnv4 unicast
  additional-paths receive
 !
 neighbor 12.12.12.12
  remote-as 65100
  capability additional-paths receive
  update-source Loopback10
  address-family ipv4 unicast
  !
  address-family vpnv4 unicast
   next-hop-self
  !
 !
 vrf Green
  rd 65100:100
  address-family ipv4 unicast
  !
  neighbor 172.16.1.2
   remote-as 65001
   address-family ipv4 unicast
    route-policy eBGP-VRF in
    route-policy eBGP-VRF out
   !
  !
 !
!

Default behaviour of BGP Route Reflector leads to suboptimal routing

In the following scenario, Site 1 and Site 2 use VRF Green to connect through an MPLS L3VPN network with eBGP as the PE - CE routing protocol. Site 2 has one CE router connecting with two PE routers. The PE routers and the Route Reflector (RR R12) run IOS XR.

The MPLS L3VPN network uses the RR R12 to advertise Site-local VPNv4 prefixes between each PE router. On PE R6 and PE R7 the local preference value and other BGP attributes are left to their defaults. Also, PE R6 and PE R7 have the same Route Distinguisher (RD) attached to the Site 2 prefixes (RD 65100:200). As a result, the RR receives identical VPNv4 prefixes from R6 and R7.

By default, the RR chooses one best path to advertise to its RR clients. Thus, the RR has to decide which Site 2 route it advertises to PE R5. It can either advertise the Site 2 route through PE R6, or through PE R7. As the routes are identical, R12 chooses the best path based on the lowest IGP cost to reach Site 2. The lowest cost from R12 is through PE R7. So, R5 receives PE R7 as the BGP next-hop to reach Site 2.

This leads to suboptimal routing, as for R5, actually PE R6 would be a shorter path.

BGP Route Reflector uses path hiding and selects a single best path based on the IGP metric, when the best path is unavailable second path is advertised and implicit withdrawal removes best path
RP/0/0/CPU0:R12#show bgp vpnv4 unicast sum |  beg Ne
Wed Jun 28 14:33:26.048 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
5.5.5.5           0 65100     396     440      165    0    0 02:20:25          4
6.6.6.6           0 65100     387     427      165    0    0 02:20:28          4   « RR R12 receives routes from PE R6
7.7.7.7           0 65100     386     430      165    0    0 02:20:30          4





RP/0/0/CPU0:R12#show bgp vpnv4 unicast |  beg Ne    
Wed Jun 28 14:33:50.547 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:100
*>i192.168.1.0/29     5.5.5.5                  0    100      0 65001 ?
*>i192.168.1.8/29     5.5.5.5                  2    100      0 65001 ?
*>i192.168.1.16/28    5.5.5.5                  2    100      0 65001 ?
*>i192.168.1.32/27    5.5.5.5                  2    100      0 65001 ?
Route Distinguisher: 65100:200
* i192.168.2.0/29     6.6.6.6                  0    100      0 65002 ?
*>i                   7.7.7.7                  0    100      0 65002 ?   « RR R12 chooses best path through PE R7
* i192.168.2.8/29     6.6.6.6                  2    100      0 65002 ?
*>i                   7.7.7.7                  2    100      0 65002 ?
* i192.168.2.16/28    6.6.6.6                  2    100      0 65002 ?
*>i                   7.7.7.7                  2    100      0 65002 ?
* i192.168.2.32/27    6.6.6.6                  2    100      0 65002 ?
*>i                   7.7.7.7                  2    100      0 65002 ?







RP/0/0/CPU0:R12#show bgp vpnv4 unicas rd 65100:200 192.168.2.8/29
Wed Jun 28 14:35:37.659 UTC
BGP routing table entry for 192.168.2.8/29, Route Distinguisher: 65100:200
Versions:
  Process           bRIB/RIB  SendTblVer
  Speaker                160         160
Last Modified: Jun 28 12:13:02.705 for 02:22:35
Paths: (2 available, best #2)
  Advertised to update-groups (with more than one peer):
    0.2 
  Path #1: Received by speaker 0
  Not advertised to any peer
  65002, (Received from a RR-client)
    6.6.6.6 (metric 30) from 6.6.6.6 (6.6.6.6)
      Received Label 16608 
      Origin incomplete, metric 2, localpref 100, valid, internal, import-candidate, not-in-vrf
      Received Path ID 0, Local Path ID 0, version 0
      Extended community: RT:65100:200 
  Path #2: Received by speaker 0
  Advertised to update-groups (with more than one peer):
    0.2 
  65002, (Received from a RR-client)
    7.7.7.7 (metric 20) from 7.7.7.7 (7.7.7.7)      « RR R12 chooses best path based on IGP metric
      Received Label 16708 
      Origin incomplete, metric 2, localpref 100, valid, internal, best, group-best, import-candidate, not-in-vrf
      Received Path ID 0, Local Path ID 0, version 160
      Extended community: RT:65100:200 
	  
	  
	  
	  
	  
	  
	  

RP/0/0/CPU0:R5#show bgp vpnv4 unicast rd 65100:200 | beg Ne
Wed Jun 28 14:34:38.196 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:200
*>i192.168.2.0/29     7.7.7.7                  0    100      0 65002 ?   « PE R5 receives best path to Site 2 via PE R7 (suboptimal)
*>i192.168.2.8/29     7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.16/28    7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.32/27    7.7.7.7                  2    100      0 65002 ?

Processed 4 prefixes, 4 paths







RP/0/0/CPU0:R5#show bgp vpnv4 unicast sum | beg Ne
Wed Jun 28 14:33:12.362 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
12.12.12.12       0 65100     438     398      229    0    0 02:20:12          4   « PE R5 only receives prefixes pointing to PE R7







R10#trace vrf Green 192.168.2.9 pr 1
Type escape sequence to abort.
Tracing the route to 192.168.2.9
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.1.1 2 msec
  2 172.16.1.1 2 msec
  3 10.0.4.1 [MPLS: Labels 105/16708 Exp 0] 12 msec
  4 10.0.1.2 [MPLS: Labels 205/16708 Exp 0] 12 msec
  5 10.0.2.2 [MPLS: Labels 305/16708 Exp 0] 10 msec
  6 10.0.3.2 [MPLS: Labels 405/16708 Exp 0] 15 msec
  7 10.0.6.2 [MPLS: Label 16708 Exp 0] 11 msec
  8 172.16.3.2 11 msec         « Accordingly, traceroute from Site 1 to Site 2 shows path through PE R7
  9 192.168.2.2 13 msec

BGP Add-Path configuration with L3VPN MPLS on IOS XR

The BGP Additional Paths feature is configured on the RR R12 and the PE R5. This feature is negotiated between the BGP neighbors as a BGP capability. After applying and committing the configuration on the routers, the BGP neighborship was manually reset to trigger the use of this capability.

As a result, in order to reach Site 2 from Site 1, PE R5 receives two paths from the RR. One path is through the PE R7 and another path is through PE R6. PE R5 can now decide which one is the preferred path. Consequently, for PE R5 the shortest path (based on the IGP metric) is through PE R6.

BGP Route Reflector sends multiple paths to PE R5 due to the BGP Additional Paths feature

Configuration:

R5 (PE)
    RP/0/0/CPU0:R5#show run | beg hostname
    Wed Jun 28 15:01:08.777 UTC
    Building configuration...
    hostname R5
    vrf Green
     address-family ipv4 unicast
      import route-target
       65100:200
      !
      export route-target
       65100:100
      !
     !
    !
    interface Loopback10
     ipv4 address 5.5.5.5 255.255.255.255
    !
    interface MgmtEth0/0/CPU0/0
     shutdown
    !
    interface GigabitEthernet0/0/0/0.1
     description ** to CE R8 **
     vrf Green
     ipv4 address 172.16.1.1 255.255.255.252
     encapsulation dot1q 5
    !
    interface GigabitEthernet0/0/0/1
     description ** to R1 **
     ipv4 address 10.0.4.2 255.255.255.252
    !
    interface GigabitEthernet0/0/0/2
     shutdown
    !
    route-policy eBGP-VRF
      pass
    end-policy
    !
    router isis ISIS-Process
     is-type level-2-only
     net 49.0100.0000.0000.0005.00
     log adjacency changes
     address-family ipv4 unicast
      metric-style wide
      advertise passive-only
     !
     interface Loopback10
      passive
      address-family ipv4 unicast
      !
     !
     interface GigabitEthernet0/0/0/1
      point-to-point
      address-family ipv4 unicast
      !
     !
    !
    router bgp 65100
     bgp router-id 5.5.5.5
     address-family ipv4 unicast
     !
     address-family vpnv4 unicast
      additional-paths receive
     !
     neighbor 12.12.12.12
      remote-as 65100
      capability additional-paths receive
      update-source Loopback10
      address-family ipv4 unicast
      !
      address-family vpnv4 unicast
       next-hop-self
      !
     !
     vrf Green
      rd 65100:100
      address-family ipv4 unicast
      !
      neighbor 172.16.1.2
       remote-as 65001
       address-family ipv4 unicast
        route-policy eBGP-VRF in
        route-policy eBGP-VRF out
       !
      !
     !
    !
    mpls ldp
     router-id 5.5.5.5
     address-family ipv4
     !
     interface GigabitEthernet0/0/0/1
      address-family ipv4
      !
     !
    !
    mpls label range table 0 16500 16599
    end
    
R6 (PE)
    RP/0/0/CPU0:R6#show run | beg hostname
    Wed Jun 28 15:01:43.595 UTC
    Building configuration...
    hostname R6
    vrf Green
     address-family ipv4 unicast
      import route-target
       65100:100
      !
      export route-target
       65100:200
      !
     !
    !
    interface Loopback10
     ipv4 address 6.6.6.6 255.255.255.255
    !
    interface MgmtEth0/0/CPU0/0
     shutdown
    !
    interface GigabitEthernet0/0/0/0
     description ** to PE R7 **
     ipv4 address 10.0.7.1 255.255.255.252
    !
    interface GigabitEthernet0/0/0/1.1
     description ** to CE R9 **
     vrf Green
     ipv4 address 172.16.2.1 255.255.255.252
     encapsulation dot1q 5
    !
    interface GigabitEthernet0/0/0/2
     description ** to R3 **
     ipv4 address 10.0.5.2 255.255.255.252
    !
    route-policy eBGP-VRF
      pass
    end-policy
    !
    router isis ISIS-Process
     is-type level-2-only
     net 49.0100.0000.0000.0006.00
     log adjacency changes
     address-family ipv4 unicast
      metric-style wide
      advertise passive-only
     !
     interface Loopback10
      passive
      address-family ipv4 unicast
      !
     !
     interface GigabitEthernet0/0/0/0
      point-to-point
      address-family ipv4 unicast
      !
     !
     interface GigabitEthernet0/0/0/2
      point-to-point
      address-family ipv4 unicast
      !
     !
    !
    router bgp 65100
     bgp router-id 6.6.6.6
     address-family ipv4 unicast
     !
     address-family vpnv4 unicast
     !
     neighbor 12.12.12.12
      remote-as 65100
      update-source Loopback10
      address-family vpnv4 unicast
       next-hop-self
      !
     !
     vrf Green
      rd 65100:200
      address-family ipv4 unicast
      !
      neighbor 172.16.2.2
       remote-as 65002
       address-family ipv4 unicast
        route-policy eBGP-VRF in
        route-policy eBGP-VRF out
       !
      !
     !
    !
    mpls ldp
     router-id 6.6.6.6
     address-family ipv4
     !
     interface GigabitEthernet0/0/0/0
      address-family ipv4
      !
     !
     interface GigabitEthernet0/0/0/2
      address-family ipv4
      !
     !
    !
    mpls label range table 0 16600 16699
    end
    
R7 (PE)
    RP/0/0/CPU0:R7#show run | beg hostname 
    Wed Jun 28 15:02:10.143 UTC
    Building configuration...
    hostname R7
    vrf Green
     address-family ipv4 unicast
      import route-target
       65100:100
      !
      export route-target
       65100:200
      !
     !
    !
    interface Loopback10
     ipv4 address 7.7.7.7 255.255.255.255
    !
    interface MgmtEth0/0/CPU0/0
     shutdown
    !
    interface GigabitEthernet0/0/0/0
     description ** to PE R6 **
     ipv4 address 10.0.7.2 255.255.255.252
    !
    interface GigabitEthernet0/0/0/1.1
     description ** to CE R9 **
     vrf Green
     ipv4 address 172.16.3.1 255.255.255.252
     encapsulation dot1q 10
    !
    interface GigabitEthernet0/0/0/2
     description ** to R4 **
     ipv4 address 10.0.6.2 255.255.255.252
    !
    route-policy eBGP-VRF
      pass
    end-policy
    !
    router isis ISIS-Process
     is-type level-2-only
     net 49.0100.0000.0000.0007.00
     log adjacency changes
     address-family ipv4 unicast
      metric-style wide
      advertise passive-only
     !
     interface Loopback10
      passive
      address-family ipv4 unicast
      !
     !
     interface GigabitEthernet0/0/0/0
      point-to-point
      address-family ipv4 unicast
      !
     !
     interface GigabitEthernet0/0/0/2
      point-to-point
      address-family ipv4 unicast
      !
     !
    !
    router bgp 65100
     bgp router-id 7.7.7.7
     address-family ipv4 unicast
     !
     address-family vpnv4 unicast
     !
     neighbor 12.12.12.12
      remote-as 65100
      update-source Loopback10
      address-family vpnv4 unicast
       next-hop-self
      !
     !
     vrf Green
      rd 65100:200
      address-family ipv4 unicast
      !
      neighbor 172.16.3.2
       remote-as 65002
       address-family ipv4 unicast
        route-policy eBGP-VRF in
        route-policy eBGP-VRF out
       !
      !
     !
    !
    mpls ldp
     router-id 7.7.7.7
     address-family ipv4
     !
     interface GigabitEthernet0/0/0/0
      address-family ipv4
      !
     !
     interface GigabitEthernet0/0/0/2
      address-family ipv4
      !
     !
    !
    mpls label range table 0 16700 16799
    end
    
R12 (RR)
    RP/0/0/CPU0:R12#show run | beg hostname
    Wed Jun 28 15:02:40.298 UTC
    Building configuration...
    hostname R12
    interface Loopback10
     ipv4 address 12.12.12.12 255.255.255.255
    !
    interface MgmtEth0/0/CPU0/0
     shutdown
    !
    interface GigabitEthernet0/0/0/0
     ipv4 address 10.0.8.2 255.255.255.248
    !
    interface GigabitEthernet0/0/0/1
     shutdown
    !
    interface GigabitEthernet0/0/0/2
     shutdown
    !
    route-policy BGP-ADD-PATH
      set path-selection all advertise
    end-policy
    !
    router isis ISIS-Process
     is-type level-2-only
     net 49.0100.0000.0000.0012.00
     log adjacency changes
     address-family ipv4 unicast
      metric-style wide
      advertise passive-only
     !
     interface Loopback10
      passive
      address-family ipv4 unicast
      !
     !
     interface GigabitEthernet0/0/0/0
      point-to-point
      address-family ipv4 unicast
      !
     !
    !
    router bgp 65100
     bgp router-id 12.12.12.12
     address-family vpnv4 unicast
      additional-paths send
      additional-paths selection route-policy BGP-ADD-PATH
     !
     neighbor 5.5.5.5
      remote-as 65100
      capability additional-paths send
      update-source Loopback10
      address-family vpnv4 unicast
       route-reflector-client
      !
     !
     neighbor 6.6.6.6
      remote-as 65100
      update-source Loopback10
      address-family vpnv4 unicast
       route-reflector-client
      !
     !
     neighbor 7.7.7.7
      remote-as 65100
      update-source Loopback10
      address-family vpnv4 unicast
       route-reflector-client
      !
     !
    !
    end
    
R9 (CE)
    R9#show run | sec ^router  
    router ospf 10 vrf Green
     router-id 9.9.9.9
     capability vrf-lite
     redistribute bgp 65002 subnets
     network 192.168.2.0 0.0.0.7 area 0
    router bgp 65002
     bgp router-id 9.9.9.9
     bgp log-neighbor-changes
     !
     address-family ipv4 vrf Green
      redistribute ospf 10
      neighbor 172.16.2.1 remote-as 65100
      neighbor 172.16.2.1 activate
      neighbor 172.16.3.1 remote-as 65100
      neighbor 172.16.3.1 activate
     exit-address-family
    
    R9#show run | sec ^vrf
    vrf definition Green
     rd 65100:200
     !
     address-family ipv4
     exit-address-family
    
    R9#show run int Gi0/0.1 | sec int
    interface GigabitEthernet0/0.1
     description ** to PE R6 **
     encapsulation dot1Q 5
     vrf forwarding Green
     ip address 172.16.2.2 255.255.255.252
    
    R9#show run int Gi0/0.2 | sec int
    interface GigabitEthernet0/0.2
     description ** to PE R7 **
     encapsulation dot1Q 10
     vrf forwarding Green
     ip address 172.16.3.2 255.255.255.252
     
    R9#show run int Gi0/2 | sec int
    interface GigabitEthernet0/2
     description ** to LAN **
     vrf forwarding Green
     ip address 192.168.2.1 255.255.255.248
     ip ospf priority 10
     duplex auto
     speed auto
     media-type rj45
    
R1 (P, core)
    R1#show run | sec ^mpls
    mpls label range 100 199
    
    R1#show run | sec ^router
    router isis
     net 49.0100.0000.0000.0001.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
     
    R1#show run int Gi1 | sec int
    interface GigabitEthernet1
     description ** to R2 **
     ip address 10.0.1.1 255.255.255.252
     ip router isis 
     negotiation auto
     mpls ip
     no mop enabled
     no mop sysid
     isis network point-to-point
     
    R1#show run int Gi2 | sec int
    interface GigabitEthernet2
     description ** to R5 **
     ip address 10.0.4.1 255.255.255.252
     ip router isis 
     negotiation auto
     mpls ip
     no mop enabled
     no mop sysid
     isis network point-to-point 
    
RP/0/0/CPU0:R5#show bgp vpnv4 unicast sum | beg Ne
Wed Jun 28 14:56:49.335 UTC
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
12.12.12.12       0 65100     467     427      249    0    0 00:13:03          8    « PE R5 receives more paths from the RR








RP/0/0/CPU0:R5#show bgp vpnv4 unicast rd 65100:200 | beg Ne
Wed Jun 28 14:58:34.427 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:200
* i192.168.2.0/29     7.7.7.7                  0    100      0 65002 ?
*>i                   6.6.6.6                  0    100      0 65002 ?    « PE R5 receives also the paths through PE R6
* i192.168.2.8/29     7.7.7.7                  2    100      0 65002 ?
*>i                   6.6.6.6                  2    100      0 65002 ?
* i192.168.2.16/28    7.7.7.7                  2    100      0 65002 ?
*>i                   6.6.6.6                  2    100      0 65002 ?
* i192.168.2.32/27    7.7.7.7                  2    100      0 65002 ?
*>i                   6.6.6.6                  2    100      0 65002 ?

Processed 4 prefixes, 8 paths





R10#trace vrf Green 192.168.2.9 pr 1
Type escape sequence to abort.
Tracing the route to 192.168.2.9
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.1.1 1 msec
  2 172.16.1.1 2 msec
  3 10.0.4.1 [MPLS: Labels 104/16608 Exp 0] 21 msec
  4 10.0.1.2 [MPLS: Labels 204/16608 Exp 0] 22 msec
  5 10.0.2.2 [MPLS: Labels 304/16608 Exp 0] 21 msec
  6 10.0.5.2 [MPLS: Label 16608 Exp 0] 22 msec    « Traceroute from Site 1 to Site 2 shows that PE R6 is the preferred path
  7 172.16.2.2 21 msec
  8 192.168.2.2 23 msec

What happens when you use different Route Distinguishers on PE routers?

As shown in the previous example, the BGP Additional Paths feature configured on RR R12 advertises multiple paths to identical VPNv4 (Site 2) prefixes. The Site 2 VPNv4 prefixes are identical because they are assigned the same RD 65100:200 on both of the connected PE routers. The Site 2 prefixes are received from PE R6 and PE R7, and are shown in the following output.

RP/0/0/CPU0:R12#show bgp vpnv4 unicast rd 65100:200 | beg Ne
Wed Jun 28 15:57:36.112 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:200                                          « Same RD used for both prefixes, so the RR selects a single best path
* i192.168.2.0/29     6.6.6.6                  0    100      0 65002 ?
*>i                   7.7.7.7                  0    100      0 65002 ?
* i192.168.2.8/29     6.6.6.6                  2    100      0 65002 ?
*>i                   7.7.7.7                  2    100      0 65002 ?
* i192.168.2.16/28    6.6.6.6                  2    100      0 65002 ?
*>i                   7.7.7.7                  2    100      0 65002 ?
* i192.168.2.32/27    6.6.6.6                  2    100      0 65002 ?
*>i                   7.7.7.7                  2    100      0 65002 ?

Processed 4 prefixes, 8 paths

However, in case PE R6 and PE R7 use different RDs to advertise the Site 2 prefixes to the RR R12, then BGP Add-path is not needed. With different RDs assigned, the prefixes are treated as separate routes, and by default both of the routes are advertised from the RR R12 to PE R5 (without selecting a single best path/without BGP path hidings). This is illustrated below.

Differentiating prefixes by assigning different VPN Route Distinguishers on MPLS L3VPN PE routers
RP/0/0/CPU0:R12#show bgp vpnv4 unicast rd 65100:200 | beg Ne
Wed Jun 28 16:22:56.168 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:200                                           « Now only prefixes received from PE R6 have RD 65100:200
*>i192.168.2.0/29     6.6.6.6                  0    100      0 65002 ?
*>i192.168.2.8/29     6.6.6.6                  2    100      0 65002 ?
*>i192.168.2.16/28    6.6.6.6                  2    100      0 65002 ?
*>i192.168.2.32/27    6.6.6.6                  2    100      0 65002 ?

Processed 4 prefixes, 4 paths






RP/0/0/CPU0:R12#show bgp vpnv4 unicast | beg Ne             
Wed Jun 28 16:23:25.636 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:100
*>i192.168.1.0/29     5.5.5.5                  0    100      0 65001 ?
*>i192.168.1.8/29     5.5.5.5                  2    100      0 65001 ?
*>i192.168.1.16/28    5.5.5.5                  2    100      0 65001 ?
*>i192.168.1.32/27    5.5.5.5                  2    100      0 65001 ?
Route Distinguisher: 65100:200                                                « RD assigned by PE R6
*>i192.168.2.0/29     6.6.6.6                  0    100      0 65002 ?
*>i192.168.2.8/29     6.6.6.6                  2    100      0 65002 ?
*>i192.168.2.16/28    6.6.6.6                  2    100      0 65002 ?
*>i192.168.2.32/27    6.6.6.6                  2    100      0 65002 ?
Route Distinguisher: 65100:300                                                « RD assigned by PE R7
*>i192.168.2.0/29     7.7.7.7                  0    100      0 65002 ?
*>i192.168.2.8/29     7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.16/28    7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.32/27    7.7.7.7                  2    100      0 65002 ?

Processed 12 prefixes, 12 paths






RP/0/0/CPU0:R5#show bgp vpnv4 unicast | beg Ne      « PE R5 receives prefixes with both RDs, and selects the shortest path to Site 2
Wed Jun 28 16:23:47.687 UTC
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:100 (default for vrf Green)
*> 192.168.1.0/29     172.16.1.2               0             0 65001 ?
*> 192.168.1.8/29     172.16.1.2               2             0 65001 ?
*> 192.168.1.16/28    172.16.1.2               2             0 65001 ?
*> 192.168.1.32/27    172.16.1.2               2             0 65001 ?
*>i192.168.2.0/29     6.6.6.6                  0    100      0 65002 ?
* i                   7.7.7.7                  0    100      0 65002 ?
*>i192.168.2.8/29     6.6.6.6                  2    100      0 65002 ?
* i                   7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.16/28    6.6.6.6                  2    100      0 65002 ?
* i                   7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.32/27    6.6.6.6                  2    100      0 65002 ?
* i                   7.7.7.7                  2    100      0 65002 ?
Route Distinguisher: 65100:200               
*>i192.168.2.0/29     6.6.6.6                  0    100      0 65002 ?
*>i192.168.2.8/29     6.6.6.6                  2    100      0 65002 ?
*>i192.168.2.16/28    6.6.6.6                  2    100      0 65002 ?
*>i192.168.2.32/27    6.6.6.6                  2    100      0 65002 ?
Route Distinguisher: 65100:300
*>i192.168.2.0/29     7.7.7.7                  0    100      0 65002 ?
*>i192.168.2.8/29     7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.16/28    7.7.7.7                  2    100      0 65002 ?
*>i192.168.2.32/27    7.7.7.7                  2    100      0 65002 ?





R10#trace vrf Green 192.168.2.9 pr 1
Type escape sequence to abort.
Tracing the route to 192.168.2.9
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.1.1 1 msec
  2 172.16.1.1 2 msec
  3 10.0.4.1 [MPLS: Labels 104/16608 Exp 0] 22 msec
  4 10.0.1.2 [MPLS: Labels 204/16608 Exp 0] 20 msec
  5 10.0.2.2 [MPLS: Labels 304/16608 Exp 0] 21 msec
  6 10.0.5.2 [MPLS: Label 16608 Exp 0] 20 msec      « The shortest path through PE R6 is selected
  7 172.16.2.2 20 msec
  8 192.168.2.2 21 msec