MPLS Carrier Supporting Carrier and Backbone Transport VPN

« 2023 Feb 8 »

What is Carrier Supporting Carrier?

Carrier Supporting Carrier is a hierarchical MPLS VPN architecture where a Backbone Carrier provides transit services to a Customer Carrier. In such a design an MPLS VPN service is provisioned by a Backbone Carrier and is used by the Customer Carrier for its own MPLS VPN. This architecture is also called Carrier's Carrier or CSC and the backbone (transit) service can be provisioned in a wholesale business model.

MPLS Carrier Support Carrier CSC design and architecture network model providing Layer-3 VPN service

The above image focuses on key elements of CSC such as the Multiprotocol-BGP neighborship between the Customer Carrier Route Reflectors (RR). This MP-BGP neighborship is an integral part of CSC because the Customer Carrier network domains need to share the MPLS VPN label bindings to create an end-to-end LSP. Additionally, the CSC-PE and CSC-CE routers need a method to share routing information with each other which is discussed in the next section.

Carrier Supporting Carrier network design options

CSC can be configured in different ways, and the following table describes some of the important functional elements for each design.

Carrier's Carrier design options
Design option Description
Customer Carrier domains use different BGP AS Numbers The Customer Carrier RRs have a multihop eBGP neighborship, it is important that the advertised next-hop IP address should be the PE router, and not the RR itself. So the next-hop should not change. Thus, the RR's have the neighbor X.X.X.X next-hop-unchanged command configured under BGP.
Customer Carrier domains use the same BGP AS Number The Customer Carrier RRs establish an iBGP neighborship, but now the CSC-CE needs to be instructed to accept BGP routes that have the CSC-CE's own AS number in the AS path attribute. Thus, the CSC-CE needs to use neighbor X.X.X.X allowas-in under BGP.
CSC-PE and CSC-CE use eBGP Labeled Unicast (BGP-LU) eBGP is configured with Labeled Unicast between the CSC-PE and CSC-CE on a connected subinterface. The CSC-CE and CSC-PE need to be instructed to transmit and receive MPLS label switched traffic between each other. This is accomplished with the command mpls bgp forwarding configured on the CSC-CE router's CSC-PE-facing subinterface.
CSC-PE and CSC-CE use IGP such as IS-IS or OSPF An IGP routing protocol (IS-IS or OSPF) is configured together with LDP to connect the CSC-CE and CSC-PE. As a result, label bindings are shared and traffic is label switched between the CSC-CE / CSC-PE. In this case however, the interface-level command mpls ldp discovery transport-address interface is added on the CSC-CE and CSC-PE to instruct LDP to use the connected interface as the transport address to establish a neighborship, instead of using the LDP router ID (e.g. Loopback address).

How to configure MPLS Carrier Supporting Carrier?

The following topology introduces an MPLS Layer-3 VPN Carrier Supporting Carrier (CSC) architecture. Carrier A provides MPLS L3VPN services in two regions using BGP AS 65010 and AS 65020. Site 1 and Site 2 are example customer locations assigned to VRF Gold.

Because the network of Carrier A cannot cover the geographical distance between the two regions, Carrier B is contracted to provide an MPLS L3VPN transit service. This makes Carrier B the backbone carrier, and Carrier A the customer carrier. Moreover, this creates new device roles for both carriers. The CSC-PE belongs to Carrier B, and the CSC-CE belongs to Carrier A.

The displayed example topology uses the following technologies to connect the carriers and create an end-to-end Label Switched Path (LSP) between PE router R2 and PE R15:

  • PE R2 and PE R15 use BGP-LU and advertise their Loopback IP address to the RRs, also RR R3 and RR R14 use BGP-LU and advertise their Loopback IP address to the CSC-CE routers, the command neighbor X.X.X.X send-label is necessary in address-family IPv4
  • eBGP is configured between the CSC-CE and CSC-PE with BGP-LU, Carrier B transports the Loopback IP addresses of Carrier A's PE and RR routers in a dedicated VRF named "Backbone-Transit" between the CSC-PEs R6 and R11
  • on the CSC-PE the CSC-CE-facing interface is assigned into a VRF, but on the CSC-CE the CSC-PE-facing interface is not assigned into a VRF, the VLANs need to match, also the command mpls bgp forwarding is configured on the subinterface of R5, R6, R11, R12
  • the Loopback IP address of PE R15 and RR R14 is redistributed into the IGP IS-IS on CSC-CE R5, and vice versa the Loopback IP address of PE R2 and RR R3 is redistributed into the IGP IS-IS on CSC-CE R12, the IPs have MPLS labels assigned due to BGP-LU
  • RR R3 and RR R14 have an eBGP-multihop VPNv4 neighborship with the command neighbor X.X.X.X next-hop-unchanged configured on both routers to ensure that the PE Loopback IP addresses remain the next-hop for destination traffic
Carrier Support Carrier CSC MPLS service provider architecture configuration with backbone L3VPN transit network

Configuration:

R6 (CSC-PE)
    R6#show run all | sec ^mpls ip__
    mpls ip
    
    R6#show run | sec ^mpls         
    mpls label range 600 699
    mpls ldp router-id Loopback10 force
    
    R6#show run | sec ^vrf
    vrf definition Backbone-Transit
     rd 65100:1
     !
     address-family ipv4
      route-target export 65100:1
      route-target import 65100:2
     exit-address-family
     
    R6#show run int Lo10 | sec int 
    interface Loopback10
     ip address 6.6.6.6 255.255.255.255
    
    R6#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R7 **
     ip address 10.4.0.1 255.255.255.252
     ip ospf network point-to-point
     duplex auto
     speed auto
     media-type rj45
     mpls ip
    
    R6#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to CSC-CE R5 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R6#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to CSC-CE R5 in VRF Backbone-Transit **
     encapsulation dot1Q 10
     vrf forwarding Backbone-Transit
     ip address 172.16.1.1 255.255.255.252
     mpls bgp forwarding
    
    R6#show run | sec ^router
    router ospf 10
     router-id 6.6.6.6
     network 6.6.6.6 0.0.0.0 area 0
     network 10.4.0.0 0.0.0.3 area 0
    router bgp 65100
     bgp router-id 6.6.6.6
     bgp log-neighbor-changes
     neighbor 9.9.9.9 remote-as 65100
     neighbor 9.9.9.9 update-source Loopback10
     !
     address-family vpnv4
      neighbor 9.9.9.9 activate
      neighbor 9.9.9.9 send-community extended
     exit-address-family
     !
     address-family ipv4 vrf Backbone-Transit
      neighbor 172.16.1.2 remote-as 65010
      neighbor 172.16.1.2 activate
      neighbor 172.16.1.2 send-label
     exit-address-family
    
R11 (CSC-PE)
    R11#show run all | sec ^mpls ip__
    mpls ip
    
    R11#show run | sec ^mpls
    mpls label range 1100 1199
    mpls ldp router-id Loopback10 force
    
    R11#show run | sec ^vrf
    vrf definition Backbone-Transit
     rd 65100:2
     !
     address-family ipv4
      route-target export 65100:2
      route-target import 65100:1
     exit-address-family
    
    R11#show run int Lo10 | sec int
    interface Loopback10
     ip address 11.11.11.11 255.255.255.255
    
    R11#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R10 **
     ip address 10.8.0.1 255.255.255.252
     ip ospf network point-to-point
     duplex auto
     speed auto
     media-type rj45
     mpls ip
    
    R11#show run int Gi0/1 | sec int 
    interface GigabitEthernet0/1
     description ** to CSC-CE R12 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
     
    R11#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to CSC-CE R12 in VRF Backbone-Transit **
     encapsulation dot1Q 10
     vrf forwarding Backbone-Transit
     ip address 172.16.2.1 255.255.255.252
     mpls bgp forwarding
    
    R11#show run | sec ^router
    router ospf 10
     router-id 11.11.11.11
     network 10.8.0.0 0.0.0.3 area 0
     network 11.11.11.11 0.0.0.0 area 0
    router bgp 65100
     bgp router-id 11.11.11.11
     bgp log-neighbor-changes
     neighbor 9.9.9.9 remote-as 65100
     neighbor 9.9.9.9 update-source Loopback10
     !
     address-family vpnv4
      neighbor 9.9.9.9 activate
      neighbor 9.9.9.9 send-community extended
     exit-address-family
     !
     address-family ipv4 vrf Backbone-Transit
      neighbor 172.16.2.2 remote-as 65020
      neighbor 172.16.2.2 activate
      neighbor 172.16.2.2 send-label
     exit-address-family
    
R5 (CSC-CE)
    R5#show run all | sec ^mpls ip__
    mpls ip
    
    R5#show run | sec ^mpls  
    mpls label range 500 599
    mpls ldp router-id Loopback10 force
    
    R5#show run int Lo10 | sec int 
    interface Loopback10
     ip address 5.5.5.5 255.255.255.255
    
    R5#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R4 **
     ip address 10.3.0.2 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point 
    
    R5#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to CSC-PE R6 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R5#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to CSC-PE R6 in VLAN 10 (no VRF) **
     encapsulation dot1Q 10
     ip address 172.16.1.2 255.255.255.252
     mpls bgp forwarding
    
    R5#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0005.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     redistribute bgp 65010 route-map RM-CSC-REDISTRIBUTION
     passive-interface Loopback10
    router bgp 65010
     bgp router-id 5.5.5.5
     bgp log-neighbor-changes
     neighbor 3.3.3.3 remote-as 65010
     neighbor 3.3.3.3 update-source Loopback10
     neighbor 172.16.1.1 remote-as 65100
     !
     address-family ipv4
      neighbor 3.3.3.3 activate
      neighbor 3.3.3.3 send-label
      neighbor 172.16.1.1 activate
      neighbor 172.16.1.1 send-label
     exit-address-family
    
    R5#show run | sec ^route-map
    route-map RM-CSC-REDISTRIBUTION permit 10
     match ip address prefix-list PL-CSC-REDISTRIBUTION
    
    R5#show run | sec ^ip prefix
    ip prefix-list PL-CSC-REDISTRIBUTION seq 5 permit 14.14.14.14/32
    ip prefix-list PL-CSC-REDISTRIBUTION seq 10 permit 15.15.15.15/32
    
R12 (CSC-CE)
    R12#show run all | sec ^mpls ip__
    mpls ip
    
    R12#show run | sec ^mpls
    mpls label range 1200 1299
    mpls ldp router-id Loopback10 force
    
    R12#show run int Lo10 | sec int
    interface Loopback10
     ip address 12.12.12.12 255.255.255.255
    
    R12#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R13 **
     ip address 10.9.0.1 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point
    
    R12#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to CSC-PE R11 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R12#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to CSC-PE R11 in VLAN 10 (no VRF) **
     encapsulation dot1Q 10
     ip address 172.16.2.2 255.255.255.252
     mpls bgp forwarding
    
    R12#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0012.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     redistribute bgp 65020 route-map RM-CSC-REDISTRIBUTION
     passive-interface Loopback10
    router bgp 65020
     bgp router-id 12.12.12.12
     bgp log-neighbor-changes
     neighbor 14.14.14.14 remote-as 65020
     neighbor 14.14.14.14 update-source Loopback10
     neighbor 172.16.2.1 remote-as 65100
     !
     address-family ipv4
      neighbor 14.14.14.14 activate
      neighbor 14.14.14.14 send-label
      neighbor 172.16.2.1 activate
      neighbor 172.16.2.1 send-label
     exit-address-family
    
    R12#show run | sec ^route-map
    route-map RM-CSC-REDISTRIBUTION permit 10
     match ip address prefix-list PL-CSC-REDISTRIBUTION
    
    R12#show run | sec ^ip prefix
    ip prefix-list PL-CSC-REDISTRIBUTION seq 5 permit 2.2.2.2/32
    ip prefix-list PL-CSC-REDISTRIBUTION seq 10 permit 3.3.3.3/32
    
R2 (PE)
    R2#show run all | sec ^mpls ip__
    mpls ip
    
    R2#show run | sec ^mpls
    mpls label range 200 299
    mpls ldp router-id Loopback10 force
    
    R2#show run | sec ^vrf
    vrf definition Gold
     rd 65010:1
     !
     address-family ipv4
      route-target export 65010:1
      route-target import 65020:1
     exit-address-family
    
    R2#show run int Lo10 | sec int
    interface Loopback10
     ip address 2.2.2.2 255.255.255.255
    
    R2#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R4 **
     ip address 10.1.0.1 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point 
    
    R2#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to CE R1 **
     no ip address
     duplex auto
     speed auto
    
    R2#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to CE R1 in VRF Gold **
     encapsulation dot1Q 10
     vrf forwarding Gold
     ip address 10.0.1.1 255.255.255.252
    
    R2#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0002.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
    router bgp 65010
     bgp router-id 2.2.2.2
     bgp log-neighbor-changes
     neighbor 3.3.3.3 remote-as 65010
     neighbor 3.3.3.3 update-source Loopback10
     !
     address-family ipv4
      network 2.2.2.2 mask 255.255.255.255
      neighbor 3.3.3.3 activate
      neighbor 3.3.3.3 send-label
     exit-address-family
     !
     address-family vpnv4
      neighbor 3.3.3.3 activate
      neighbor 3.3.3.3 send-community extended
     exit-address-family
     !        
     address-family ipv4 vrf Gold
      network 10.0.1.0 mask 255.255.255.252
      neighbor 10.0.1.2 remote-as 65001
      neighbor 10.0.1.2 activate
     exit-address-family
    
R15 (PE)
    R15#show run all | sec ^mpls ip__
    mpls ip
    
    R15#show run | sec ^mpls
    mpls label range 1500 1599
    mpls ldp router-id Loopback10 force
    
    R15#show run | sec ^vrf
    vrf definition Gold
     rd 65020:1
     !
     address-family ipv4
      route-target export 65020:1
      route-target import 65010:1
     exit-address-family
     
    R15#show run int Lo10 | sec int
    interface Loopback10
     ip address 15.15.15.15 255.255.255.255
    
    R15#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R13 **
     ip address 10.11.0.1 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point
     
    R15#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to CE R16 **
     no ip address
     duplex auto
     speed auto
    
    R15#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to CE R16 in VRF Gold **
     encapsulation dot1Q 10
     vrf forwarding Gold
     ip address 10.0.2.1 255.255.255.252
    
    R15#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0015.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
    router bgp 65020
     bgp router-id 15.15.15.15
     bgp log-neighbor-changes
     neighbor 14.14.14.14 remote-as 65020
     neighbor 14.14.14.14 update-source Loopback10
     !
     address-family ipv4
      network 15.15.15.15 mask 255.255.255.255
      neighbor 14.14.14.14 activate
      neighbor 14.14.14.14 send-label
     exit-address-family
     !
     address-family vpnv4
      neighbor 14.14.14.14 activate
      neighbor 14.14.14.14 send-community extended
     exit-address-family
     !        
     address-family ipv4 vrf Gold
      network 10.0.2.0 mask 255.255.255.252
      neighbor 10.0.2.2 remote-as 65002
      neighbor 10.0.2.2 activate
     exit-address-family
    
R9 (RR Backbone Carrier)
    R9#show run int Lo10 | sec int
    interface Loopback10
     ip address 9.9.9.9 255.255.255.255
    
    R9#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R8 **
     ip address 10.7.0.2 255.255.255.252
     ip ospf network point-to-point
     duplex auto
     speed auto
     media-type rj45
    
    R9#show run | sec ^router
    router ospf 10
     router-id 9.9.9.9
     network 9.9.9.9 0.0.0.0 area 0
     network 10.7.0.0 0.0.0.3 area 0
    router bgp 65100
     bgp router-id 9.9.9.9
     bgp log-neighbor-changes
     neighbor 6.6.6.6 remote-as 65100
     neighbor 6.6.6.6 update-source Loopback10
     neighbor 11.11.11.11 remote-as 65100
     neighbor 11.11.11.11 update-source Loopback10
     !
     address-family vpnv4
      neighbor 6.6.6.6 activate
      neighbor 6.6.6.6 send-community extended
      neighbor 6.6.6.6 route-reflector-client
      neighbor 11.11.11.11 activate
      neighbor 11.11.11.11 send-community extended
      neighbor 11.11.11.11 route-reflector-client
     exit-address-family
    
R3 (RR)
    R3#show run int Lo10 | sec int
    interface Loopback10
     ip address 3.3.3.3 255.255.255.255
    
    R3#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R4 **
     ip address 10.2.0.1 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     isis circuit-type level-2-only
     isis network point-to-point 
    
    R3#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0003.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
    router bgp 65010
     bgp router-id 3.3.3.3
     bgp log-neighbor-changes
     neighbor 2.2.2.2 remote-as 65010
     neighbor 2.2.2.2 update-source Loopback10
     neighbor 5.5.5.5 remote-as 65010
     neighbor 5.5.5.5 update-source Loopback10
     neighbor 14.14.14.14 remote-as 65020
     neighbor 14.14.14.14 ebgp-multihop 255
     neighbor 14.14.14.14 update-source Loopback10
     !
     address-family ipv4
      network 3.3.3.3 mask 255.255.255.255
      neighbor 2.2.2.2 activate
      neighbor 2.2.2.2 send-label
      neighbor 5.5.5.5 activate
      neighbor 5.5.5.5 route-reflector-client
      neighbor 5.5.5.5 send-label
      no neighbor 14.14.14.14 activate
     exit-address-family
     !
     address-family vpnv4
      neighbor 2.2.2.2 activate
      neighbor 2.2.2.2 send-community extended
      neighbor 2.2.2.2 route-reflector-client
      neighbor 14.14.14.14 activate
      neighbor 14.14.14.14 send-community extended
      neighbor 14.14.14.14 next-hop-unchanged
     exit-address-family
    
R14 (RR)
    R14#show run int Lo10 | sec int
    interface Loopback10
     ip address 14.14.14.14 255.255.255.255
    
    R14#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R13 **
     ip address 10.10.0.2 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     isis circuit-type level-2-only
     isis network point-to-point
    
    R14#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0014.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
    router bgp 65020
     bgp router-id 14.14.14.14
     bgp log-neighbor-changes
     neighbor 3.3.3.3 remote-as 65010
     neighbor 3.3.3.3 ebgp-multihop 255
     neighbor 3.3.3.3 update-source Loopback10
     neighbor 12.12.12.12 remote-as 65020
     neighbor 12.12.12.12 update-source Loopback10
     neighbor 15.15.15.15 remote-as 65020
     neighbor 15.15.15.15 update-source Loopback10
     !
     address-family ipv4
      network 14.14.14.14 mask 255.255.255.255
      no neighbor 3.3.3.3 activate
      neighbor 12.12.12.12 activate
      neighbor 12.12.12.12 route-reflector-client
      neighbor 12.12.12.12 send-label
      neighbor 15.15.15.15 activate
      neighbor 15.15.15.15 send-label
     exit-address-family
     !
     address-family vpnv4
      neighbor 3.3.3.3 activate
      neighbor 3.3.3.3 send-community extended
      neighbor 3.3.3.3 next-hop-unchanged
      neighbor 15.15.15.15 activate
      neighbor 15.15.15.15 send-community extended
      neighbor 15.15.15.15 route-reflector-client
     exit-address-family
    
R1 (CE)
    R1#show run | sec ^vrf
    vrf definition Gold
     rd 1:1
     !
     address-family ipv4
     exit-address-family
    
    R1#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to Host1 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R1#show run int Gi0/0.10 | sec int
    interface GigabitEthernet0/0.10
     description ** to Host1 in VRF Gold (in native VLAN) **
     encapsulation dot1Q 1 native
     vrf forwarding Gold
     ip address 192.168.1.2 255.255.255.0
    
    R1#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to PE R2 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R1#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to PE R2 in VRF Gold **
     encapsulation dot1Q 10
     vrf forwarding Gold
     ip address 10.0.1.2 255.255.255.252
    
    R1#show run | sec ^router
    router bgp 65001
     bgp router-id 1.1.1.1
     bgp log-neighbor-changes
     !
     address-family ipv4 vrf Gold
      network 192.168.1.0
      neighbor 10.0.1.1 remote-as 65010
      neighbor 10.0.1.1 activate
     exit-address-family
    
R16 (CE)
    R16#show run | sec ^vrf
    vrf definition Gold
     rd 1:1
     !
     address-family ipv4
     exit-address-family
    
    R16#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to Host2 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R16#show run int Gi0/0.10 | sec int
    interface GigabitEthernet0/0.10
     description ** to Host2 in VRF Gold (in native VLAN) **
     encapsulation dot1Q 1 native
     vrf forwarding Gold
     ip address 192.168.2.2 255.255.255.0
    
    R16#show run int Gi0/1 | sec int   
    interface GigabitEthernet0/1
     description ** to PE R15 **
     no ip address
     duplex auto
     speed auto
     media-type rj45
    
    R16#show run int Gi0/1.10 | sec int
    interface GigabitEthernet0/1.10
     description ** to PE R15 in VRF Gold **
     encapsulation dot1Q 10
     vrf forwarding Gold
     ip address 10.0.2.2 255.255.255.252
    
    R16#show run | sec ^router
    router bgp 65002
     bgp router-id 16.16.16.16
     bgp log-neighbor-changes
     !
     address-family ipv4 vrf Gold
      network 192.168.2.0
      neighbor 10.0.2.1 remote-as 65020
      neighbor 10.0.2.1 activate
     exit-address-family
    
R4
    R4#show run all | sec ^mpls ip__
    mpls ip
    
    R4#show run | sec ^mpls
    mpls label range 400 499
    mpls ldp router-id Loopback10 force
    
    R4#show run int Lo10 | sec int
    interface Loopback10
     ip address 4.4.4.4 255.255.255.255
    
    R4#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to PE R2 **
     ip address 10.1.0.2 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point
    
    R4#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to RR R3 **
     ip address 10.2.0.2 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     isis circuit-type level-2-only
     isis network point-to-point
    
    R4#show run int Gi0/2 | sec int
    interface GigabitEthernet0/2
     description ** to CSC-CE R5 **
     ip address 10.3.0.1 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point 
    
    R4#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0004.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
    
R13
    R13#show run all | sec ^mpls ip__
    mpls ip
    
    R13#show run | sec ^mpls
    mpls label range 1300 1399
    mpls ldp router-id Loopback10 force
    
    R13#show run int Lo10 | sec int
    interface Loopback10
     ip address 13.13.13.13 255.255.255.255
    
    R13#show run int Gi0/0 | sec int 
    interface GigabitEthernet0/0
     description ** to PE R15 **
     ip address 10.11.0.2 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point
    
    R13#show run int Gi0/1 | sec int 
    interface GigabitEthernet0/1
     description ** to RR R14 **
     ip address 10.10.0.1 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     isis circuit-type level-2-only
     isis network point-to-point
    
    R13#show run int Gi0/2 | sec int
    interface GigabitEthernet0/2
     description ** to CSC-CE R12 **
     ip address 10.9.0.2 255.255.255.252
     ip router isis 
     duplex auto
     speed auto
     media-type rj45
     mpls ip
     isis circuit-type level-2-only
     isis network point-to-point
    
    R13#show run | sec ^router
    router isis
     net 49.0010.0000.0000.0013.00
     is-type level-2-only
     advertise passive-only
     metric-style wide
     log-adjacency-changes
     passive-interface Loopback10
    
R8
    R8#show run all | sec ^mpls ip__
    mpls ip
    
    R8#show run | sec ^mpls
    mpls label range 800 899
    mpls ldp router-id Loopback10 force
    
    R8#show run int Lo10 | sec int
    interface Loopback10
     ip address 8.8.8.8 255.255.255.255
    
    R8#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to R7 **
     ip address 10.5.0.2 255.255.255.252
     ip ospf network point-to-point
     duplex auto
     speed auto
     media-type rj45
     mpls ip
    
    R8#show run int Gi0/1 | sec int
    interface GigabitEthernet0/1
     description ** to R10 **
     ip address 10.6.0.1 255.255.255.252
     ip ospf network point-to-point
     duplex auto
     speed auto
     media-type rj45
     mpls ip
    
    R8#show run int Gi0/2 | sec int
    interface GigabitEthernet0/2
     description ** to RR R9 **
     ip address 10.7.0.1 255.255.255.252
     ip ospf network point-to-point
     duplex auto
     speed auto
     media-type rj45
    
    R8#show run | sec ^router
    router ospf 10
     router-id 8.8.8.8
     network 8.8.8.8 0.0.0.0 area 0
     network 10.5.0.0 0.0.0.3 area 0
     network 10.6.0.0 0.0.0.3 area 0
     network 10.7.0.0 0.0.0.3 area 0
    
Host1
    Host1#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to CE R1 **
     ip address 192.168.1.1 255.255.255.0
     duplex auto
     speed auto
     media-type rj45
     
    Host1#show run | sec ^ip route
    ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 192.168.1.2
    
Host2
    Host2#show run int Gi0/0 | sec int
    interface GigabitEthernet0/0
     description ** to CE R16 **
     ip address 192.168.2.1 255.255.255.0
     duplex auto
     speed auto
     media-type rj45
    
    Host2#show run | sec ^ip route
    ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 192.168.2.2
    
Host1#trace 192.168.2.1 probe 1
Type escape sequence to abort.
Tracing the route to 192.168.2.1
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.1.2 2 msec
  2 10.0.1.1 2 msec
  3 10.1.0.2 [MPLS: Labels 404/1505 Exp 0] 12 msec
  4 10.3.0.2 [MPLS: Labels 505/1505 Exp 0] 11 msec
  5 172.16.1.1 [MPLS: Labels 612/1505 Exp 0] 12 msec
  6 10.4.0.2 [MPLS: Labels 704/1103/1505 Exp 0] 10 msec           « In Backbone Carrier network three MPLS labels are assigned
  7 10.5.0.2 [MPLS: Labels 804/1103/1505 Exp 0] 11 msec
  8 10.6.0.2 [MPLS: Labels 1000/1103/1505 Exp 0] 12 msec
  9 172.16.2.1 [MPLS: Labels 1103/1505 Exp 0] 15 msec
 10 172.16.2.2 [MPLS: Labels 1203/1505 Exp 0] 11 msec
 11 10.9.0.2 [MPLS: Labels 1302/1505 Exp 0] 9 msec
 12 10.0.2.1 [MPLS: Label 1505 Exp 0] 11 msec                     « 1507 is the VPN service label assigned by PE R15
 13 10.0.2.2 12 msec
 14 192.168.2.1 14 msec




Host1#ping 192.168.2.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.2.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/14/16 ms        « Host1 can ping Host2 through the CSC network





Host2#trace 192.168.1.1 probe 1
Type escape sequence to abort.
Tracing the route to 192.168.1.1
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.2.2 3 msec
  2 10.0.2.1 1 msec
  3 10.11.0.2 [MPLS: Labels 1303/205 Exp 0] 11 msec
  4 10.9.0.1 [MPLS: Labels 1204/205 Exp 0] 10 msec
  5 172.16.2.1 [MPLS: Labels 1113/205 Exp 0] 12 msec
  6 10.8.0.2 [MPLS: Labels 1004/613/205 Exp 0] 10 msec
  7 10.6.0.1 [MPLS: Labels 802/613/205 Exp 0] 8 msec
  8 10.5.0.1 [MPLS: Labels 700/613/205 Exp 0] 10 msec
  9 172.16.1.1 [MPLS: Labels 613/205 Exp 0] 9 msec
 10 172.16.1.2 [MPLS: Labels 503/205 Exp 0] 10 msec
 11 10.3.0.1 [MPLS: Labels 402/205 Exp 0] 9 msec
 12 10.0.1.1 [MPLS: Label 205 Exp 0] 9 msec                  « 205 is the VPN service label assigned by PE R2
 13 10.0.1.2 10 msec
 14 192.168.1.1 12 msec
 





Host2#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 9/10/11 ms          « Host2 can ping Host1 through the CSC network

Between the CSC-CE and the CSC-PE (in order words, between the Customer Carrier and the Backbone Carrier) BGP Labeled Unicast is used. This means, the CSC-CE router advertises the Customer Carrier BGP Route Reflector's IP address and the PE's address together with an MPLS VPN service label. This is necessary to forward (tunnel) the traffic through the Backbone Carrier's transit network.

Carrier Supporting Carrier MPLS CSC architecture BGP Labeled Unicast Update packet capture

The following router outputs verify several mechanisms used in this topology. To facilitate troubleshooting the MPLS label range is defined on each label switch router (LSR), this means R2 assigns labels in the range 200 - 299, R6 uses 600 - 699, R15 uses 1500 1599, and so on.

R5#show ip bgp ipv4 unicast label
   Network          Next Hop      In label/Out label
   2.2.2.2/32       2.2.2.2         503 /imp-null            « Labels visible in above packet capture, advetised by CSC-CE R5
   3.3.3.3/32       3.3.3.3         501 /imp-null
   14.14.14.14/32   172.16.1.1      nolabel/611              « MPLS labels received from CSC-PE R6 through eBGP-LU
   15.15.15.15/32   172.16.1.1      nolabel/612





R6#show ip bgp vpnv4 vrf Backbone-Transit label              « This is the point of view from the CSC-PE
   Network          Next Hop      In label/Out label
Route Distinguisher: 65100:1 (Backbone-Transit)
   2.2.2.2/32       172.16.1.2      613/503
   3.3.3.3/32       172.16.1.2      614/501
   14.14.14.14/32   11.11.11.11     611/1100
   15.15.15.15/32   11.11.11.11     612/1103


   
   


R2#show ip bgp vpnv4 all | beg Network
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65010:1 (default for vrf Gold)
 *>   10.0.1.0/30      0.0.0.0                  0         32768 i
 *>i  10.0.2.0/30      15.15.15.15              0    100      0 65020 i
 *>   192.168.1.0      10.0.1.2                 0             0 65001 i
 *>i  192.168.2.0      15.15.15.15              0    100      0 65020 65002 i
Route Distinguisher: 65020:1
 *>i  10.0.2.0/30      15.15.15.15              0    100      0 65020 i
 *>i  192.168.2.0      15.15.15.15              0    100      0 65020 65002 i         « PE R2 receives remote Site 2 prefix with next-hop PE R15
 
 



R2#show ip bgp vpnv4 rd 65020:1 192.168.2.0
BGP routing table entry for 65020:1:192.168.2.0/24, version 17          « BGP VPNv4 prefix of Site 2
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Refresh Epoch 1
  65020 65002                                                           « Site 2 is assigned to BGP AS 65002 due to PE-CE protocol eBGP
    15.15.15.15 (metric 20) (via default) from 3.3.3.3 (3.3.3.3)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:65020:1
      mpls labels in/out nolabel/1505                                   « PE R15 assigns VPN service label 1505 to Site 2 prefix
      rx pathid: 0, tx pathid: 0x0






R3#show ip bgp vpnv4 all summary | begin Nei 
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
2.2.2.2         4        65010     195     196        5    0    0 02:51:37        2
14.14.14.14     4        65020     192     194        5    0    0 02:50:59        2      « RR R3 has eBGP VPNv4 neighborship with RR R14






R6#show ip bgp vpnv4 all | beg Ne
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65100:1 (default for vrf Backbone-Transit)              « Prefixes advertised in the Backbone Carrier MPLS transit L3VPN 
 *>   2.2.2.2/32       172.16.1.2                             0 65010 i
 *>   3.3.3.3/32       172.16.1.2                             0 65010 i
 *>i  14.14.14.14/32   11.11.11.11              0    100      0 65020 i
 *>i  15.15.15.15/32   11.11.11.11              0    100      0 65020 i
Route Distinguisher: 65100:2
 *>i  14.14.14.14/32   11.11.11.11              0    100      0 65020 i
 *>i  15.15.15.15/32   11.11.11.11              0    100      0 65020 i

Packet capture in Backbone Carrier's network

Within the Backbone Carrier's network there are 3 MPLS labels assigned to the Customer Carrier's traffic. The Customer Carrier provisions a Layer-3 VPN service connecting the two sites (VRF Gold). The Backbone Carrier uses its own label stack to essentially tunnel the Customer Carrier's MPLS L3VPN through its transit network. The following packet capture shows further details.

Wireshark packet capture of traffic in CSC Backbone Carrier's transit network with labels stack

The MPLS label 1103 is assigned by the CSC-PE router R11 and enables the use of the Backbone Carrier's transit network to transport MPLS VPN traffic for the Customer Carrier. The Customer Carrier's PE router R15 assigns the VPN service label 1505 to the packet which enables the end-to-end LSP between the two customer sites in VRF Gold (between Host1 and Host2).