Pseudowire Headend Termination - in 8 steps - PART1
PROBLEM: Swansea, Newport and Cardiff subscribers have been, temporarily, terminated on vACX hardware at those sites. The vACX routers are typically used for mobile backhaul and generally don't have the 'grunt' to terminate subscribers. They don't support per-unit-scheduling, for example. We need to get them off these routers.
GOAL: Terminate all subscribers from the Access Provider at Cardiff, on to the new vMX960 at Llanelli.
Juniper's Configuring the Broadband Edge as a Service Node Within Seamless MPLS Network Designs article was the main inspiration for this post.
Let's log on to Llanelli and get cracking! Before we begin, let's check reachability to the loopbacks:
root@Llanelli> ping 172.16.99.1 count 1 PING 172.16.99.1 (172.16.99.1): 56 data bytes 64 bytes from 172.16.99.1: icmp_seq=0 ttl=64 time=0.066 ms --- 172.16.99.1 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.066/0.066/0.066/0.000 ms root@Llanelli> ping 172.16.99.2 count 1 PING 172.16.99.2 (172.16.99.2): 56 data bytes 64 bytes from 172.16.99.2: icmp_seq=0 ttl=64 time=93.739 ms --- 172.16.99.2 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 93.739/93.739/93.739/0.000 ms root@Llanelli> ping 172.16.99.3 count 1 PING 172.16.99.3 (172.16.99.3): 56 data bytes 64 bytes from 172.16.99.3: icmp_seq=0 ttl=64 time=21.710 ms --- 172.16.99.3 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 21.710/21.710/21.710/0.000 ms root@Llanelli> ping 172.16.99.4 count 1 PING 172.16.99.4 (172.16.99.4): 56 data bytes 64 bytes from 172.16.99.4: icmp_seq=0 ttl=63 time=281.251 ms --- 172.16.99.4 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 281.251/281.251/281.251/0.000 ms
- Add secondary IP to lo0.0:
set interfaces lo0 unit 0 family inet address 203.0.113.1/32 - Configure the dynamic VLAN profile with versioning:
set system dynamic-profile-options versioning
set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" proxy-arp restricted
set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" vlan-tags outer "$junos-stacked-vlan-id"
set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" vlan-tags inner "$junos-vlan-id"
set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet unnumbered-address lo0.0
set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet unnumbered-address preferred-source-address 203.0.113.1 - Enable tunnel-services on the PFE:
set chassis fpc 0 pic 0 tunnel-services bandwidth 1g
set chassis fpc 0 pic 0 traffic-manager egress-shaping-overhead 0
set chassis network-services enhanced-ip
set chassis pseudowire-service device-count 4 - Configure the pseudowire (ps0) interface for single and double tagged traffic:
set interfaces ps0 anchor-point lt-0/0/10
set interfaces ps0 flexible-vlan-tagging
set interfaces ps0 auto-configure stacked-vlan-ranges dynamic-profile DYNINTF-DHCP-INET accept any
set interfaces ps0 auto-configure stacked-vlan-ranges dynamic-profile DYNINTF-DHCP-INET ranges any,any
set interfaces ps0 auto-configure vlan-ranges dynamic-profile DYNINTF-DHCP-INET accept any
set interfaces ps0 auto-configure vlan-ranges dynamic-profile DYNINTF-DHCP-INET ranges any
set interfaces ps0 auto-configure remove-when-no-subscribers
set interfaces ps0 no-gratuitous-arp-request
set interfaces ps0 unit 0 encapsulation ethernet-ccc - Configure DHCP and bind it to the transport logical (ps0.0) interface:
set system services dhcp-local-server pool-match-order ip-address-first
set system services dhcp-local-server authentication username-include interface-name
set system services dhcp-local-server group local interface ps0.0 - Enable subscriber management:
set system services subscriber-management enable
set system configuration-database max-db-size 104857600 - Configure the access profile and DHCP scope. Apply the access-profile:
set access profile local authentication-order none
set access address-assignment pool TEST-NET-3 family inet network 203.0.113.0/24
set access address-assignment pool TEST-NET-3 family inet range 1 low 203.0.113.2
set access address-assignment pool TEST-NET-3 family inet range 1 high 203.0.113.250
set access address-assignment pool TEST-NET-3 family inet dhcp-attributes maximum-lease-time 3600
set access address-assignment pool TEST-NET-3 family inet dhcp-attributes domain-name msbnet.co.uk
set access address-assignment pool TEST-NET-3 family inet dhcp-attributes router 203.0.113.1
set access-profile local - Configure the l2circuit at both sites:
# Llanelli
set protocols l2circuit neighbor 172.16.99.4 interface ps0.0 virtual-circuit-id 1
set protocols l2circuit neighbor 172.16.99.4 interface ps0.0 ignore-mtu-mismatch
# Cardiff
set protocols l2circuit neighbor 172.16.99.1 interface ge-0/0/3.1 virtual-circuit-id 1
set protocols l2circuit neighbor 172.16.99.1 interface ge-0/0/3.1 encapsulation-type ethernet
set protocols l2circuit neighbor 172.16.99.1 interface ge-0/0/3.1 ignore-mtu-mismatch
set interfaces ge-0/0/3 unit 1 encapsulation vlan-ccc
set interfaces ge-0/0/3 unit 1 vlan-id-range 2-10
Finally, commit the configuration at both sites. Llanelli displays the follow message upon commit:
root@Llanelli# commit and-quit
[edit system services subscriber-management]
'enable'
warning: Chassis configuration for subscriber-management has been changed. A system reboot is mandatory. Please reboot the system NOW. Continuing without a reboot might result in unexpected system behavior.
Message from syslogd@Llanelli at Sep 20 17:18:05 ...
Llanelli fpc0 CMLC: Going disconnected; Routing engine chassis socket closed abruptly
commit complete
Exiting configuration mode
If we take a sneaky peak at the PFE directly after the commit, we see the following:
root@Llanelli> show chassis fpc
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer
0 Offline ---Restarted by cli command---
1 Empty
2 Empty
3 Empty
4 Empty
5 Empty
6 Empty
7 Empty
8 Empty
9 Empty
10 Empty
11 Empty
Enabling tunnel services caused the PFE to restart. It is at this point it gives birth to the logical tunnel interface, lt-0/0/10, amongst others. Periodically check the chassis to see if the PFE has come back up:
root@Llanelli> show chassis fpc
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer
0 Online Testing 20 0 30 33 32 2047 7 0
1 Empty
2 Empty
3 Empty
4 Empty
5 Empty
6 Empty
7 Empty
8 Empty
9 Empty
10 Empty
11 Empty
Excellent! Now reboot the routing engine as indicated after the commit:
root@Llanelli> request system reboot
Reboot the system ? [yes,no] (no) yes
*** FINAL System shutdown message from root@Llanelli ***
System going down IMMEDIATELY
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 done
All buffers synced.
Uptime: 1h4m47s
Khelp module "jsocket" can't unload until its refcount drops from 4 to 0.
Rebooting...
cpu_reset: Stopping other CPUsOnce the routing engine has come back up, it whistles to the packet forwarding engine to come hither. Once they're back in sync, we can proceed:
root@Llanelli> show chassis fpc
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer
0 Online Absent 0 0 0 0 0 0 0 0
1 Empty
2 Empty
3 Empty
4 Empty
5 Empty
6 Empty
7 Empty
8 Empty
9 Empty
10 Empty
11 Empty
Not yet.
root@Llanelli> show chassis fpc
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer
0 Online Testing 40 0 17 4 1 2047 7 0
1 Empty
2 Empty
3 Empty
4 Empty
5 Empty
6 Empty
7 Empty
8 Empty
9 Empty
10 Empty
11 EmptyJob done.
Now let's check the pseudowire has come up:
root@Llanelli> show l2circuit connections status
Layer-2 Circuit Connections:
Legend for connection status (St)
EI -- encapsulation invalid NP -- interface h/w not present
MM -- mtu mismatch Dn -- down
EM -- encapsulation mismatch VC-Dn -- Virtual circuit Down
CM -- control-word mismatch Up -- operational
VM -- vlan id mismatch CF -- Call admission control failure
OL -- no outgoing label IB -- TDM incompatible bitrate
NC -- intf encaps not CCC/TCC TM -- TDM misconfiguration
BK -- Backup Connection ST -- Standby Connection
CB -- rcvd cell-bundle size bad SP -- Static Pseudowire
LD -- local site signaled down RS -- remote site standby
RD -- remote site signaled down HS -- Hot-standby Connection
XX -- unknown
Legend for interface status
Up -- operational
Dn -- down
Neighbor: 172.16.99.4
Interface Type St Time last up # Up trans
ps0.0(vc 1) rmt Up Sep 20 17:29:48 2019 1
Remote PE: 172.16.99.4, Negotiated control-word: Yes (Null)
Incoming label: 16, Outgoing label: 299872
Negotiated PW status TLV: No
Local interface: ps0.0, Status: Up, Encapsulation: ETHERNET
Flow Label Transmit: No, Flow Label Receive: No
It has!
Now, let's check that our Cardiff subscribers have found their way over to us:
root@Llanelli> show subscribers Total subscribers: 0, Active Subscribers: 0
Oh dear. What have I missed?
A quick glance at Cardiff's access port highlights the error of my ways. I've enabled the CVLANs on the pseudowire instead of the SVLAN. Easily corrected:
root@Cardiff> show configuration interfaces ge-0/0/3 | display set set interfaces ge-0/0/3 description "Access Provider 1" set interfaces ge-0/0/3 flexible-vlan-tagging set interfaces ge-0/0/3 encapsulation flexible-ethernet-services set interfaces ge-0/0/3 unit 1 encapsulation vlan-ccc set interfaces ge-0/0/3 unit 1 vlan-id-range 2-10 <--------- CVLANs root@Cardiff> edit Entering configuration mode root@Cardiff# set interfaces ge-0/0/3 unit 1 vlan-id 101 root@Cardiff# commit and-quit commit complete Exiting configuration mode
Let's check if that worked:
root@Llanelli> show subscribers Interface IP Address/VLAN ID User Name LS:RI ps0.3221225472 0x8100.101 0x8100.2 default:default ps0.3221225473 0x8100.101 0x8100.3 default:default ps0.3221225474 0x8100.101 0x8100.4 default:default ps0.3221225475 0x8100.101 0x8100.5 default:default ps0.3221225476 0x8100.101 0x8100.6 default:default ps0.3221225477 0x8100.101 0x8100.7 default:default ps0.3221225478 0x8100.101 0x8100.8 default:default ps0.3221225479 0x8100.101 0x8100.9 default:default ps0.3221225480 0x8100.101 0x8100.10 default:default ps0.3221225472 203.0.113.2 ps0:101-2 default:default ps0.3221225473 203.0.113.3 ps0:101-3 default:default ps0.3221225474 203.0.113.4 ps0:101-4 default:default ps0.3221225475 203.0.113.5 ps0:101-5 default:default ps0.3221225476 203.0.113.6 ps0:101-6 default:default ps0.3221225477 203.0.113.7 ps0:101-7 default:default ps0.3221225478 203.0.113.8 ps0:101-8 default:default ps0.3221225479 203.0.113.9 ps0:101-9 default:default ps0.3221225480 203.0.113.10 ps0:101-10 default:default
Can we ping a subscriber?
root@Llanelli> ping 203.0.113.2 count 3 PING 203.0.113.2 (203.0.113.2): 56 data bytes 64 bytes from 203.0.113.2: icmp_seq=0 ttl=255 time=34.714 ms 64 bytes from 203.0.113.2: icmp_seq=1 ttl=255 time=40.958 ms 64 bytes from 203.0.113.2: icmp_seq=2 ttl=255 time=41.267 ms --- 203.0.113.2 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 34.714/38.980/41.267/3.019 ms
Job done!
We've covered the basics of pseudowire headend termination.
In part 2, we'll look at RADIUS and CoS profiles, assuming I can coax vMX to comply :)
Versions
vMX: 18.2R1.9
IOS: 15.2(4)M7
Configs
2_Llanelli
2_Swansea
2_Newport
2_Cardiff
2_AccessProvider1