English
We recently had the opportunity of experimenting with Arista 7280R switches in our lab and discovered an intriguing use case for employing a pair of these switches as datacenter gateway switches, much like one of our valued customers.
Why is this a smart solution?
In compact settings where only a handful of servers are housed within a single rack, the integration of router and switch functionalities into a single device can often present a more economical solution. This approach not only tends to be cost-effective initially but also contributes to sustained financial efficiency over time. However, it’s important to exercise caution with this strategy, especially in critical environments where overburdening a single device with multiple workloads is typically advised against. Nevertheless, this consolidated approach can be entirely suitable, depending on the specific requirements and objectives of the intended application.
How does it work?
In the example that follows, you’ll observe a pair of Arista 7280R switches seamlessly assuming the roles of provider edges (PEs) and another pair as customer edges (CEs). In this scenario, the PEs have been configured to serve as both L3 gateways and redundant L2 switches simultaneously. This setup leverages the power of VXLAN-EVPN overlay technology, enabling the effortless deployment of loop-free layer 2 overlay networks. In our setup we are using different IPv4 and IPv6 subnets on each side, separated by two VLANs.
PE1 VLAN, Interface, IGP, BGP and ESI config:
hostname pe1 ! vlan 10 !! Facing CE1 name VL10 ! vlan 20 !! Facing CE2 name VL20 ! interface Port-Channel1 description "--- PE2 ---" mtu 9000 no switchport ip address 192.0.2.128/31 isis enable underlay isis network point-to-point ! interface Port-Channel3 description "--- CE1 ---" mtu 9214 switchport trunk allowed vlan 10 switchport mode trunk ! evpn ethernet-segment identifier 0064:5120:0000:0000:0003 route-target import 64:51:20:00:00:03 lacp system-id 6451.2000.0000 ! interface Port-Channel4 description "--- CE2 ---" mtu 9214 switchport trunk allowed vlan 20 switchport mode trunk ! evpn ethernet-segment identifier 0064:5120:0000:0000:0004 route-target import 64:51:20:00:00:04 lacp system-id 6451.2000.0000 ! interface Ethernet1 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Ethernet2 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Ethernet3 description "--- LAG member of Port-Channel3 ---" channel-group 3 mode active ! interface Ethernet4 description "--- LAG member of Port-Channel4 ---" channel-group 4 mode active ! interface Loopback0 description "--- L3 INTERFACE ---" ip address 192.0.2.1/32 isis enable underlay isis passive ! interface Vlan10 description "--- L3 INTERFACE ---" ipv6 nd ra disabled all ip address virtual 198.51.100.1/25 ipv6 address virtual 2001:db8:10::1/64 ! interface Vlan20 description "--- L3 INTERFACE ---" ipv6 nd ra disabled all ip address virtual 198.51.100.129/25 ipv6 address virtual 2001:db8:20::1/64 ! interface Vxlan1 description "--- EVPN VXLAN VTEP ---" vxlan source-interface Loopback0 vxlan udp-port 4789 vxlan vlan 10 vni 1010 vxlan vlan 20 vni 1020 ! ip virtual-router mac-address 00:00:00:00:00:48 ! ip routing ! ipv6 unicast-routing ! router bgp 64512 router-id 192.0.2.1 neighbor 192.0.2.2 remote-as 64512 neighbor 192.0.2.2 update-source 192.0.2.1 neighbor 192.0.2.2 description "PE2" neighbor 192.0.2.2 send-community extended ! vlan 10 rd 192.0.2.1:1010 route-target both 1010:1010 redistribute learned ! vlan 20 rd 192.0.2.1:2020 route-target both 2020:2020 redistribute learned ! address-family evpn neighbor 192.0.2.2 activate ! router isis underlay net 49.0100.0000.1001.00 is-hostname PE1 is-type level-2 log-adjacency-changes ! address-family ipv4 unicast ! end
PE2 VLAN, Interface, IGP, BGP and ESI config:
hostname pe2 ! vlan 10 !! Facing CE1 name VL10 ! vlan 20 !! Facing CE2 name VL20 ! interface Port-Channel1 description "--- PE1 ---" mtu 9000 no switchport ip address 192.0.2.129/31 isis enable underlay isis network point-to-point ! interface Port-Channel3 description "--- CE1 ---" mtu 9214 switchport trunk allowed vlan 10 switchport mode trunk ! evpn ethernet-segment identifier 0064:5120:0000:0000:0003 route-target import 64:51:20:00:00:03 lacp system-id 6451.2000.0000 ! interface Port-Channel4 description "--- CE2 ---" mtu 9214 switchport trunk allowed vlan 20 switchport mode trunk ! evpn ethernet-segment identifier 0064:5120:0000:0000:0004 route-target import 64:51:20:00:00:04 lacp system-id 6451.2000.0000 ! interface Ethernet1 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Ethernet2 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Ethernet3 description "--- LAG member of Port-Channel3 ---" channel-group 3 mode active ! interface Ethernet4 description "--- LAG member of Port-Channel4 ---" channel-group 4 mode active ! interface Loopback0 description "--- L3 INTERFACE ---" ip address 192.0.2.2/32 isis enable underlay isis passive ! interface Vlan10 description "--- L3 INTERFACE ---" ipv6 nd ra disabled all ip address virtual 198.51.100.1/25 ipv6 address virtual 2001:db8:10::1/64 ! interface Vlan20 description "--- L3 INTERFACE ---" ipv6 nd ra disabled all ip address virtual 198.51.100.129/25 ipv6 address virtual 2001:db8:20::1/64 ! interface Vxlan1 description "--- EVPN VXLAN VTEP ---" vxlan source-interface Loopback0 vxlan udp-port 4789 vxlan vlan 10 vni 1010 vxlan vlan 20 vni 1020 ! ip virtual-router mac-address 00:00:00:00:00:48 ! ip routing ! ipv6 unicast-routing ! router bgp 64512 router-id 192.0.2.2 neighbor 192.0.2.1 remote-as 64512 neighbor 192.0.2.1 update-source 192.0.2.2 neighbor 192.0.2.1 description "PE1" neighbor 192.0.2.1 send-community extended ! vlan 10 rd 192.0.2.2:1010 route-target both 1010:1010 redistribute learned ! vlan 20 rd 192.0.2.2:2020 route-target both 2020:2020 redistribute learned ! address-family evpn neighbor 192.0.2.1 activate ! router isis underlay net 49.0100.0000.1002.00 is-hostname PE1 is-type level-2 log-adjacency-changes ! address-family ipv4 unicast ! end
CE1 Interface and IP config:
hostname ce1 ! interface Port-Channel1 description "--- PE 1/2 ---" no switchport ! interface Port-Channel1.10 encapsulation dot1q vlan 10 ip address 198.51.100.2/25 ipv6 address 2001:db8:10::2/64 ipv6 nd ra disabled all ! interface Ethernet1 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Ethernet2 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Management1 ip address 10.0.0.15/24 ! ip routing ! ipv6 unicast-routing ! ip route 0.0.0.0/0 198.51.100.1 ! ipv6 route ::/0 2001:db8:10::1 ! end
CE1 Interface and IP config:
hostname ce2 ! interface Port-Channel1 description "--- PE 1/2 ---" no switchport ! interface Port-Channel1.20 encapsulation dot1q vlan 20 ip address 198.51.100.130/25 ipv6 address 2001:db8:20::2/64 ipv6 nd ra disabled all ! interface Ethernet1 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Ethernet2 description "--- LAG member of Port-Channel1 ---" channel-group 1 mode active ! interface Management1 ip address 10.0.0.15/24 ! ip routing ! ipv6 unicast-routing ! ip route 0.0.0.0/0 198.51.100.129 ! ipv6 route ::/0 2001:db8:20::1 ! end
Some verification on PE1 and PE2:
pe1#show interfaces vxlan 1 Vxlan1 is up, line protocol is up (connected) Hardware is Vxlan Description: "--- EVPN VXLAN VTEP ---" Source interface is Loopback0 and is active with 192.0.2.1 Listening on UDP port 4789 Replication/Flood Mode is headend with Flood List Source: EVPN Remote MAC learning via EVPN VNI mapping to VLANs Static VLAN to VNI mapping is [10, 1010] [20, 1020] Note: All Dynamic VLANs used by VCS are internal VLANs. Use 'show vxlan vni' for details. Static VRF to VNI mapping is not configured Headend replication flood vtep list is: 10 192.0.2.2 20 192.0.2.2 Shared Router MAC is 0000.0000.0000 pe1#show isis neighbors detail Instance VRF System Id Type Interface SNPA State Hold time Circuit Id underlay default PE1 L2 Port-Channel1 P2P UP 27 17 Area addresses: 49 SNPA: P2P Router ID: 0.0.0.0 Advertised Hold Time: 30 State Changed: 01:07:42 ago at 2023-12-06 08:58:41 IPv4 Interface Address: 192.0.2.129 IPv6 Interface Address: none Interface name: Port-Channel1 Graceful Restart: Supported Supported Address Families: IPv4 Neighbor Supported Address Families: IPv4 pe1#show bgp summary BGP summary information for VRF default Router identifier 192.0.2.1, local AS number 64512 Neighbor AS Session State AFI/SAFI AFI/SAFI State NLRI Rcd NLRI Acc --------- ----------- ------------- ----------------------- -------------- ---------- ---------- 192.0.2.2 64512 Established IPv4 Unicast Negotiated 0 0 192.0.2.2 64512 Established L2VPN EVPN Negotiated 12 12 pe1#show bgp evpn route-type mac-ip BGP routing table information for VRF default Router identifier 192.0.2.1, local AS number 64512 Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP c - Contributing to ECMP, % - Pending best path selection Origin codes: i - IGP, e - EGP, ? - incomplete AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop Network Next Hop Metric LocPref Weight Path * > RD: 192.0.2.1:2020 mac-ip 0c00.1a72.671a - - - 0 i * > RD: 192.0.2.2:2020 mac-ip 0c00.1a72.671a 192.0.2.2 - 100 0 i * > RD: 192.0.2.1:2020 mac-ip 0c00.1a72.671a 198.51.100.130 - - - 0 i * > RD: 192.0.2.2:2020 mac-ip 0c00.1a72.671a 198.51.100.130 192.0.2.2 - 100 0 i * > RD: 192.0.2.1:2020 mac-ip 0c00.1a72.671a 2001:db8:20::2 - - - 0 i * > RD: 192.0.2.2:2020 mac-ip 0c00.1a72.671a 2001:db8:20::2 192.0.2.2 - 100 0 i * > RD: 192.0.2.1:1010 mac-ip 0c00.5643.bb9b - - - 0 i * > RD: 192.0.2.2:1010 mac-ip 0c00.5643.bb9b 192.0.2.2 - 100 0 i * > RD: 192.0.2.1:1010 mac-ip 0c00.5643.bb9b 198.51.100.2 - - - 0 i * > RD: 192.0.2.2:1010 mac-ip 0c00.5643.bb9b 198.51.100.2 192.0.2.2 - 100 0 i * > RD: 192.0.2.1:1010 mac-ip 0c00.5643.bb9b 2001:db8:10::2 - - - 0 i * > RD: 192.0.2.2:1010 mac-ip 0c00.5643.bb9b 2001:db8:10::2 192.0.2.2 - 100 0 i pe2#show bgp evpn route-type mac-ip BGP routing table information for VRF default Router identifier 192.0.2.2, local AS number 64512 Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP c - Contributing to ECMP, % - Pending best path selection Origin codes: i - IGP, e - EGP, ? - incomplete AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop Network Next Hop Metric LocPref Weight Path * > RD: 192.0.2.1:2020 mac-ip 0c00.1a72.671a 192.0.2.1 - 100 0 i * > RD: 192.0.2.2:2020 mac-ip 0c00.1a72.671a - - - 0 i * > RD: 192.0.2.1:2020 mac-ip 0c00.1a72.671a 198.51.100.130 192.0.2.1 - 100 0 i * > RD: 192.0.2.2:2020 mac-ip 0c00.1a72.671a 198.51.100.130 - - - 0 i * > RD: 192.0.2.1:2020 mac-ip 0c00.1a72.671a 2001:db8:20::2 192.0.2.1 - 100 0 i * > RD: 192.0.2.2:2020 mac-ip 0c00.1a72.671a 2001:db8:20::2 - - - 0 i * > RD: 192.0.2.1:1010 mac-ip 0c00.5643.bb9b 192.0.2.1 - 100 0 i * > RD: 192.0.2.2:1010 mac-ip 0c00.5643.bb9b - - - 0 i * > RD: 192.0.2.1:1010 mac-ip 0c00.5643.bb9b 198.51.100.2 192.0.2.1 - 100 0 i * > RD: 192.0.2.2:1010 mac-ip 0c00.5643.bb9b 198.51.100.2 - - - 0 i * > RD: 192.0.2.1:1010 mac-ip 0c00.5643.bb9b 2001:db8:10::2 192.0.2.1 - 100 0 i * > RD: 192.0.2.2:1010 mac-ip 0c00.5643.bb9b 2001:db8:10::2 - - - 0 i
Some verification on CE1 and CE2:
ce1#show ipv6 neighbors IPv6 Address Age Hardware Addr Interface 2001:db8:10::1 0:00:19 0000.0000.0048 Po1.10 ce1#show arp interface Port-Channel1.10 Address Age (sec) Hardware Addr Interface 198.51.100.1 0:11:52 0000.0000.0048 Port-Channel1.10 ce1#ping 2001:db8:20::2 PING 2001:db8:20::2(2001:db8:20::2) 52 data bytes 60 bytes from 2001:db8:20::2: icmp_seq=1 ttl=63 time=3.70 ms 60 bytes from 2001:db8:20::2: icmp_seq=2 ttl=63 time=3.02 ms 60 bytes from 2001:db8:20::2: icmp_seq=3 ttl=63 time=2.73 ms 60 bytes from 2001:db8:20::2: icmp_seq=4 ttl=63 time=2.91 ms 60 bytes from 2001:db8:20::2: icmp_seq=5 ttl=63 time=2.83 ms --- 2001:db8:20::2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 15ms rtt min/avg/max/mdev = 2.727/3.037/3.701/0.345 ms, ipg/ewma 3.761/3.355 ms ce1#ping 2001:db8:10::1 PING 2001:db8:10::1(2001:db8:10::1) 52 data bytes 60 bytes from 2001:db8:10::1: icmp_seq=1 ttl=64 time=1.89 ms 60 bytes from 2001:db8:10::1: icmp_seq=2 ttl=64 time=1.39 ms 60 bytes from 2001:db8:10::1: icmp_seq=3 ttl=64 time=1.44 ms 60 bytes from 2001:db8:10::1: icmp_seq=4 ttl=64 time=1.27 ms 60 bytes from 2001:db8:10::1: icmp_seq=5 ttl=64 time=1.36 ms --- 2001:db8:10::1 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 8ms rtt min/avg/max/mdev = 1.270/1.467/1.886/0.216 ms, ipg/ewma 2.010/1.668 ms ce1#ping 198.51.100.1 PING 198.51.100.1 (198.51.100.1) 72(100) bytes of data. 80 bytes from 198.51.100.1: icmp_seq=1 ttl=64 time=1.74 ms 80 bytes from 198.51.100.1: icmp_seq=2 ttl=64 time=1.34 ms 80 bytes from 198.51.100.1: icmp_seq=3 ttl=64 time=1.18 ms 80 bytes from 198.51.100.1: icmp_seq=4 ttl=64 time=1.22 ms 80 bytes from 198.51.100.1: icmp_seq=5 ttl=64 time=1.17 ms --- 198.51.100.1 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 8ms rtt min/avg/max/mdev = 1.171/1.330/1.739/0.212 ms, ipg/ewma 2.001/1.524 ms ce1#ping 198.51.100.130 PING 198.51.100.130 (198.51.100.130) 72(100) bytes of data. 80 bytes from 198.51.100.130: icmp_seq=1 ttl=63 time=4.18 ms 80 bytes from 198.51.100.130: icmp_seq=2 ttl=63 time=3.48 ms 80 bytes from 198.51.100.130: icmp_seq=3 ttl=63 time=3.51 ms 80 bytes from 198.51.100.130: icmp_seq=4 ttl=63 time=3.40 ms 80 bytes from 198.51.100.130: icmp_seq=5 ttl=63 time=3.38 ms --- 198.51.100.130 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 17ms rtt min/avg/max/mdev = 3.383/3.588/4.175/0.297 ms, ipg/ewma 4.141/3.868 ms ce2#show ipv6 neighbors IPv6 Address Age Hardware Addr Interface 2001:db8:20::1 0:00:18 0000.0000.0048 Po1.20 ce2#show arp interface Port-Channel1.20 Address Age (sec) Hardware Addr Interface 198.51.100.129 0:02:40 0000.0000.0048 Port-Channel1.20 ce2#ping 2001:db8:20::1 PING 2001:db8:20::1(2001:db8:20::1) 52 data bytes 60 bytes from 2001:db8:20::1: icmp_seq=1 ttl=64 time=4.10 ms 60 bytes from 2001:db8:20::1: icmp_seq=2 ttl=64 time=2.86 ms 60 bytes from 2001:db8:20::1: icmp_seq=3 ttl=64 time=2.86 ms 60 bytes from 2001:db8:20::1: icmp_seq=4 ttl=64 time=2.67 ms 60 bytes from 2001:db8:20::1: icmp_seq=5 ttl=64 time=2.38 ms --- 2001:db8:20::1 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 16ms rtt min/avg/max/mdev = 2.381/2.974/4.102/0.590 ms, ipg/ewma 4.069/3.507 ms ce2#ping 2001:db8:10::2 PING 2001:db8:10::2(2001:db8:10::2) 52 data bytes 60 bytes from 2001:db8:10::2: icmp_seq=1 ttl=63 time=3.35 ms 60 bytes from 2001:db8:10::2: icmp_seq=2 ttl=63 time=3.07 ms 60 bytes from 2001:db8:10::2: icmp_seq=3 ttl=63 time=2.83 ms 60 bytes from 2001:db8:10::2: icmp_seq=4 ttl=63 time=2.70 ms 60 bytes from 2001:db8:10::2: icmp_seq=5 ttl=63 time=2.81 ms --- 2001:db8:10::2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 13ms rtt min/avg/max/mdev = 2.700/2.953/3.349/0.231 ms, ipg/ewma 3.172/3.138 ms ce2#ping 198.51.100.129 PING 198.51.100.129 (198.51.100.129) 72(100) bytes of data. 80 bytes from 198.51.100.129: icmp_seq=1 ttl=64 time=3.05 ms 80 bytes from 198.51.100.129: icmp_seq=2 ttl=64 time=1.86 ms 80 bytes from 198.51.100.129: icmp_seq=3 ttl=64 time=1.33 ms 80 bytes from 198.51.100.129: icmp_seq=4 ttl=64 time=1.15 ms 80 bytes from 198.51.100.129: icmp_seq=5 ttl=64 time=1.20 ms --- 198.51.100.129 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 12ms rtt min/avg/max/mdev = 1.151/1.716/3.050/0.712 ms, ipg/ewma 3.119/2.346 ms ce2#ping 198.51.100.2 PING 198.51.100.2 (198.51.100.2) 72(100) bytes of data. 80 bytes from 198.51.100.2: icmp_seq=1 ttl=63 time=4.26 ms 80 bytes from 198.51.100.2: icmp_seq=2 ttl=63 time=3.16 ms 80 bytes from 198.51.100.2: icmp_seq=3 ttl=63 time=2.97 ms 80 bytes from 198.51.100.2: icmp_seq=4 ttl=63 time=2.65 ms 80 bytes from 198.51.100.2: icmp_seq=5 ttl=63 time=2.66 ms --- 198.51.100.2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 16ms rtt min/avg/max/mdev = 2.654/3.141/4.262/0.592 ms, ipg/ewma 4.117/3.670 ms