ETHERNET VPN
CZY TYLKO DLA DATA CENTER?
Emil Gągała
PLNOG, 30.09.2014
WHAT IS ETHERNET VPN
A new standards based protocol to
inter-connects L2 domains
Improves network efficiency
Multi-vendor industry wide
BGP based state
exchange
WAN
2 Copyright © 2013 Juniper Networks, Inc.
LAG
Multi-vendor industry wide
initiative -- JNPR, CSCO, ALU, ATT,
Verizon, Bloomberg ….
Ideally suited for Datacenter
Interconnectivity but ...
... NOT only
EVPN router
EVPN router
LAN
ETHERNET VPN SUPPORT
Authors listed on draft-ietf-l2vpn-evpn-08 include:
� A. Sajassi – Cisco Systems� J. Drake – Juniper Networks� W. Henderickx – Alcatel-Lucent� R. Aggarwal– Arktan
3 Copyright © 2013 Juniper Networks, Inc.
� R. Aggarwal– Arktan� N. Bitar – Verizon� A. Isaac – Bloomberg� J. Uttaro – AT&T
4 Copyright © 2013 Juniper Networks, Inc.
VPLS TO EVPN COMPARISON
VPLS EMULATES AN ETHERNET SWITCH
Common Characteristics:
�Forwarding of Ethernet Frames�Forwarding of Unicast frames with an unknown MAC address�Replication of broadcast and multicast frames�Loop prevention�Dynamic Learning of MAC address
5 Copyright © 2013 Juniper Networks, Inc.
Site 1 Site 2
�Dynamic Learning of MAC address
Virtual Private LAN Service (VPLS) provides VLAN Extension over a shared IP/MPLS network.
VPLS CHARACTERISTICS
Full Mesh
VLAN Separation
Any-to-Any connectivity regardless of physical path
Separate VPLS instances per VLAN. Allows network-wide segmentation with very large scale
6 Copyright © 2013 Juniper Networks, Inc.
VLAN Separation
Provisioning
Multicast, Broadcast and Flooding
Availability
with very large scale
New site Auto Discovery
Scale forwarding with Multicast & Point-to-Multipoint capabilities
Underlying MPLS offers ECMP, Fast Reroute
EVPN provides VLAN Extension over a shared IP/MPLS network.
EVPN REQUIREMENTS (ON TOP OF VPLS)
All-Active Multi-Homing
Better Control Over MAC
All available paths should be used (CE-PE, PE-PE)
MAC learning happens in control plane
7 Copyright © 2013 Juniper Networks, Inc.
Better Control Over MAC Learning
ARP/ND Flooding Minimization
L3 Egress Traffic Forwarding Optimization
Reducing Unknown Unicast Flooding
MAC learning happens in control plane
Additional attributes added during MAC advertisement
Usage of Default Gateway Extended Community
By using MAC learning in control plane
8 Copyright © 2013 Juniper Networks, Inc.
EVPN: VALUE PROPOSITION AND USE CASES
WHY ETHERNET VPN
EVPN Use Cases:
� Next generation L2VPN technology that replaces VPLS, VPWS
� As DC Interconnect – allowing L2 stretch between two data
centers over WAN
� EVPN as control plane with VxLAN IP overlay DC networks
Which customers are interested in EVPN and why ?:
9 Copyright © 2013 Juniper Networks, Inc.
Which customers are interested in EVPN and why ?:
� Service providers that offer E-LAN / E-LINE services
� Today, use a PE router for L2 services with VPLS, VPWS
� EVPN technology improves their service offering
� Data Center Builders – SPs, Enterprises, Content providers
� Today, use a DC WAN Edge Router
� EVPN allows multi-tenant L2 stretch between DCs and within DC
USE CASE #1: EVPN FOR NEXT GENERATION ETHERNET SERVICES
BGP signaling on WAN exchange MAC/IP routes
BGP signaling on WAN exchange MAC/IP routesEVPN
PE2
EVPNPE1
EVPN PE3
EVPNPE4CE
CE
MPLS 9
MP-BGP
10 Copyright © 2013 Juniper Networks, Inc.
Benefits:• Allows more efficient, feature rich E-LAN and E-LINE services
• Solves shortcomings of VPLS; offers IP VPN like policy control
• Supports explosive traffic growth
• Active/Active multi-homing with load balancing
• Improves network efficiency
• Minimizes flooding of BUM traffic/improves MAC learning
USE CASE #2: EVPN FOR DATA CENTER INTERCONNECT
VLAN 1MAC1
VLAN 1MAC11
Data Plane LearningData Plane Learning BGP Control Plane based learning on WANBGP Control Plane based learning on WAN Data Plane LearningData Plane Learning
E-VPN CloudE-VPN Cloud
Legacy L2 Cloud
Legacy L2 Cloud
VXLAN CloudVXLAN Cloud
11 Copyright © 2013 Juniper Networks, Inc.
VLAN 2MAC 2
VLAN 2MAC22
Data Center Site1 Data Center Site 2 Data Center Interconnect
Benefits:• Seamless interconnect for DCI – L3 aware L2 stretch between DCs
• Seamless workload migration - VM mobility across DCs
• Wide Applicability – Interconnects Native L2 and overlay DC
technologies like VxLAN, MPLS in DC
USE CASE #3: EVPN-VXLAN FOR DC OVERLAY
EVPN acts as control plane protocol� VNID used in place of Eth tag ID for EVPN
signaling
VxLAN is data plane encapsulation
Benefits of EVPN-VxLAN DC Overlay:• Allows Simple All IP fabric in DC
Overlay environment
IP Fabric
Network Orchestrator
Management Plane API needed only
12 Copyright © 2013 Juniper Networks, Inc.
• Allows Simple All IP fabric in DC• No need for multi-layers L2 setup in DCs
• Allows L2 connectivity for VMs / applications
• Deliver a L2VPN straight to a hypervisor
• L2 Multi-tenancy in all IP DC• Each tenant can have 4094 VNIDs
• VXLAN Tunnel Endpoints (VTEP) exist on
networks equipment and hypervisors
• All benefits of EVPN applicable in a DC
TOR
VDSVDS
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
14 Copyright © 2013 Juniper Networks, Inc.
EVPN BUILDING BLOCKS & OPERATIONS
EVPN INSTANCE AND EVPN SERVICE INTERFACES
EVPN Instance (EVI) represents a VPN in the MPLS/IP network
One or multiple broadcast domains can be part of the same EVI
Each broadcast domain is uniquely identified inside EVI by Ethernet Tag
PEVLANs
(all or N)
PEVLANs
PEVLANs
16 Copyright © 2013 Juniper Networks, Inc.
VLAN Bundle SI/Port Based SI
(all or N)
BD EVI
• All CEs MUST use same CE-VIDs
• Encap. MPLS frames MUST remain tagged
• No Tag translation allowed
VLAN Based SI
BD EVI
• One-to-One mapping• Different CE-VIDs can be
used on CEs• Tag translation allowed• Ethernet Tag is set to 0
BD EVI
VLAN Aware Bundle SI
BD
• Many-to-One mapping• Different CE-VIDs can be
used on CEs BUT• Normalized tag MUST be used• Ethernet tag == NormalizedTag
EVIBD
EVPN BASIC CONFIGURATION –VLAN-BASED SERVICE INTERFACE
interfaces {ge-1/0/1 {
flexible-vlan-tagging;encapsulation extended-vlan-bridge;unit 10 {
vlan-id 10;family bridge;
}}ge-1/0/2 {
flexible-vlan-tagging;encapsulation extended - vlan - bridge;
protocols {bgp {
group iBGP-EVPN {type internal;local-address 11.99.0.13;family evpn {
signaling;}neighbor 11.99.0.86;
}}
}
17 Copyright © 2013 Juniper Networks, Inc.
encapsulation extended - vlan - bridge;unit 30 {
vlan-id 30;family bridge;
}}
}
}routing-instances {
EVPN-1 {instance-type evpn;vlan-id 200; interface ge-1/0/1.10;interface ge-1/0/2.30;route-distinguisher 11.99.0.13:200;vrf-target target:65320:200;protocols {
evpn;}
}}
EVPN INFORMATION EXCHANGE OVERVIEW
MPLS or IP
detours
LAG
LAG
Route Reflector
VLAN 1MAC1, IP1.1
VLAN 2MAC2, IP2.1
VLAN 1MAC11, IP1.11
VLAN 2MAC22, IP2.22
18 Copyright © 2013 Juniper Networks, Inc.
ESI
Route Distinguisher
Ethernet Tag
MAC Address
EVPN reachability advertisement
IPv4 or IPv6 Address
Service Tag
• EVPN advertises MAC (L2) and IP (ARP) bindings for each segment along with service tags
• Allowing Control Plane based L2 and ARP learning• Minimizes flooding across WAN• Allows proxy-ARP to respond queries locally
• IRB MAC address exchange allows same gateway MAC address across sites
• VM mobility: egress traffic optimization
ETHERNET TAG IDENTIFIER
� An Ethernet Tag ID is a 32-bit field � Contains a 12-bit or a 24-bit identifier to identify a broadcast
domain in an EVPN instance. � 12-bit identifier is used for normalized VLAN ID for EVPN (MPLS)� 24-bit identifier is used for VNID for EVPN-VxLAN� 24- bit identified is used for I-SID for PBB-EVPN.
� An EVI can have one or more broadcast domains – VLANs -
19 Copyright © 2013 Juniper Networks, Inc.
� An EVI can have one or more broadcast domains – VLANs -assigned to a given EVPN instance
ETHERNET SEGMENT IDENTIFIER (ESI)
If CE is multi-homed to two or more PEs, the set of Ethernet links constitutes an “Ethernet Segment”.
A/P or A/A multi-homing is supported
An Ethernet Segment MUST have a non-reserved ESI that is unique network wide. ESI can be auto-provisioned
20 Copyright © 2013 Juniper Networks, Inc.
CE
PE1
PE2ESI Auto-Provisioning with MC-LAG
MPLS
System Prio System MAC Address Port Key
CE PE1
PE2ESI Auto-Provisioning with MC-LAG
BPDU
MPLS
Bridge Prio Root Bridge MAC 0x0000
CE BPDUL2
EVPN ACTIVE/STANDBY MULTI-HOMING: CONFIGURATION
interfaces {ge-1/0/1 {
flexible-vlan-tagging;encapsulation extended-vlan-bridge;esi {
00:10:11:00:00:00:00:00:00:01;single-active;
}unit 10 {
vlan-id 10;family bridge;
interfaces {ge-10/0/3 {
vlan-tagging;encapsulation extended-vlan-bridge;esi {
00:20:22:00:00:00:00:00:00:02;single-active;
}unit 20 {
vlan-id 20;family bridge;
PE1: PE2:
21 Copyright © 2013 Juniper Networks, Inc.
family bridge;}
}ge-1/0/2 {
flexible-vlan-tagging;encapsulation extended-vlan-bridge;esi {
00:20:22:00:00:00:00:00:00:02;single-active;
}unit 30 {
vlan-id 30;family bridge;
}}
family bridge;}
}}
EVPN NLRI
BGP AFI 25 (L2VPN)/ SAFI 70 (EVPN)� Format = Route Type : Length : Route-type Specific
EVPN Route Types1) Ethernet Auto-Discovery (A-D) Route
� Used for fast convergence (withdrawal), and active/active multi-homing (split-horizon label)
� 2 variants: per ESI and per EVI
22 Copyright © 2013 Juniper Networks, Inc.
� 2 variants: per ESI and per EVI
2) MAC/IP Advertisement Route� Used for remote MAC address learning, known unicast traffic
3) Inclusive Multicast Route� Used for BUM (broadcast, unknown unicast, multicast) traffic
4) Ethernet Segment Route� Used for auto-discovery of multi-homed Ethernet segments and Designated
Forwarder election
ETHERNET AUTO-DISCOVERY PER ESI – TYPE 1BGP signaling on WAN
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS 9
Loop Avoidance via split horizonFast Convergence
Ethernet AD route per ESI announces
ESI mode
25 Copyright © 2013 Juniper Networks, Inc.
• Ethernet AD route per ESI signals All active or sin gle active mode of operation for a multi-homed CE
• Advertises Split Horizon label for L2 BUM traffic• Enables forwarding state for the advertised ESI• On withdrawal of AD route per ESI, all PEs adjust N Hs or invalidate MAC
routes associated with that ESI, allowing rapid con vergence
ESI
RD
Ethernet Tag
Service Tag
Auto Discovery message per L2
Segment
ETHERNET AUTO-DISCOVERY PER EVI – TYPE 1MP BGP signaling between PEs
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS 9
Allows Load-balancingRapid convergence
Ethernet AD route per EVI includes all connected ESIs for
that EVI
26 Copyright © 2013 Juniper Networks, Inc.
• Each Multi-homed PE advertises AD route per EVI for all connected ESIs to advertise “service label” (aka “aliasing label ”)
• Ethernet A-D per EVI route is used for ’Aliasing’ ( load-balancing)• Remote PEs use AD per EVI route and MAC route toget her to load-
balance traffic • Load balancing for L2 as well as L3 traffic
• AD route per EVI and AD route per ESI BOTH are reqd for multi-homing
ESI
RD of EVI
Ethernet Tag
Service Tag
Auto Discovery message per EVI
EVPN ROUTE TYPE 1 –ETHERNET AUTO-DISCOVERY ROUTE
juniper@mx-re1> show route table EVPN-1.evpn.0 detail
1:11.99.0.86:0::202200000000000002::0/304 (1 entry, 1 announced) ���� Format = Type:RD::ESI::Label/304
*BGP Preference: 170/-101Route Distinguisher: 11.99.0.86:0 ���� RD set to PE IP address followed by zero
[…]Source: 11.99.0.86Protocol next hop: 11.99.0.86[…]
27 Copyright © 2013 Juniper Networks, Inc.
[…]Local AS: 65320 Peer AS: 65320Age: 35:05 Metric2: 1 Validation State: unverified Task: BGP_65320.11.99.0.86+179Announcement bits (1): 0-EVPN-1-evpn AS path: ICommunities: target:65320:200 esi-label:100000(label 0) ���� ESI Label:
flag = 0: active/active, flag = 1: active/standbyImport AcceptedLocalpref: 100Router ID: 11.99.0.86Primary Routing Table bgp.evpn.0
MAC ROUTE – TYPE 2MP BGP signaling between EVPN PEs
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS 9
Establishes Reachability Each PE learns MAC on CE-PE link and advertises
its reachability in EVPN MAC route
28 Copyright © 2013 Juniper Networks, Inc.
ESI
RD of EVI
Ethernet Tag
MAC Address
MAC reachability advertisement
IPv4 or IPv6 Address
Service Tag
• Advertises host MAC (and host IP) reachability with “service label”
• Allows Control Plane based MAC learning for remote PEs• On MX, service label is same as one advertised in AD per EVI route• Minimizes flooding across WAN• Allows PE to do proxy-ARP for remote hosts locally
• IRB MAC address route has default GW extended commu nity• Used in VM motion when default GW of VM remains sam e
• If IRB MACs and IP are same across MH PEs, avoids f looding after node failure
RT
INCLUSIVE MULTICAST ROUTE – TYPE 3BGP signaling on WAN
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS 9
MP-BGP
Sets up path for BUM trafficPer VLAN per EVI
29 Copyright © 2013 Juniper Networks, Inc.
� Allows PE to send BUM traffic from a CE on a VLAN i n an EVI, to all the other PEs that span that VLAN in that EVPN instance
• Uses Existing MVPN defined constructs for signalli ng and transport• P2MP Tunnel : If advertising PE uses a P-Multicast tree for EVPN,
the PMSI Tunnel attribute MUST contain tree identit y • Ingres Replication : Route includes PMSI Tunnel att ribute with
Tunnel Type set to Ingress Replication and Tunnel I D as PE address.
• Able to carry the traffic of more than one EVPN ins tance on the same tree using ’Aggregation’
Eth TAG
RD of EVI
Advertising PE IP
Next Hop (PE IP)
Inclusive multicast Ethernet TAG route
Route Target
PMSI Tunnel Attr
EVPN ROUTES TYPE 2 & TYPE 3juniper@mx-re1> show route table EVPN-1.evpn.0
2:11.99.0.13:200::200::00:00:0a:0a:02:01/304 *[EVPN/170] 00:04:36
Indirect2:11.99.0.13:200::200::00:00:0b:0a:00:12/304
*[EVPN/170] 00:04:36Indirect
2:11.99.0.13:200::200::00:00:0b:0a:01:11/304*[EVPN/170] 00:04:36
Indirect3:11.99.0.13:200::200::11.99.0.13/304
*[EVPN/170] 00:53:47Indirect
Local MACAdvertisement Routes
(Format = Type:RD::Eth-Tag-iD::MAC/304)
Local Inclusive Multicast
Ethernet Tag Route
30 Copyright © 2013 Juniper Networks, Inc.
Indirect
2:11.99.0.86:200::200::00:00:0b:0a:00:0a/304 *[BGP/170] 00:26:58, localpref 100, from 11.99.0.86
AS path: I, validation-state: unverified> to 11.0.100.18 via xe-2/0/0.10
to 11.0.100.22 via xe-2/0/1.102:11.99.0.86:200::200::00:00:0b:0a:00:0b/304
*[BGP/170] 00:26:58, localpref 100, from 11.99.0.86AS path: I, validation-state: unverified
> to 11.0.100.18 via xe-2/0/0.10to 11.0.100.22 via xe-2/0/1.10
3:11.99.0.86:200::200::11.99.0.86/304 *[BGP/170] 00:27:01, localpref 100, from 11.99.0.86
AS path: I, validation-state: unverified> to 11.0.100.18 via xe-2/0/0.10
to 11.0.100.22 via xe-2/0/1.10
Ethernet Tag Route
Remote MAC
Advertisement Routes
Remote Inclusive Multicast
Ethernet Tag Route
ETHERNET SEGMENT ROUTE – TYPE 4BGP signaling on WAN
PE2
PE1 (DF)
PE3
PE4CE
CE
MPLS 9
Simplifies Configuration
Loop Avoidance via DF selection
Ethernet Segment route sent to every EVPN peer for ESI
discovery
31 Copyright © 2013 Juniper Networks, Inc.
• Ethernet Segment Identifier allows multi-homing of CEs to PE• PEs connected to the same Ethernet segment discover each
other by exchanging of Ethernet Segment route.• Include ES-Import extended community with value aut o-
derived from the MAC address portion of ESI• Only PEs that host that ESI import this route• DF selection is carried out based on ES routes
ESI
RD
IP Addr Length
Originator’s IP Addr
ES Route
EVPN ACTIVE/STANDBY MULTI-HOMING: ROUTE TYPE 4 – ETHERNET SEGMENT ROUTE
juniper@mx-re1> show route table bgp.evpn.0 detail
4:11.99.0.86:0::202200000000000002:11.99.0.86/304 (1 entry, 0 announced) ���� Format = Type:RD::ESI:Originating-Router-IP/304
*BGP Preference: 170/-101Route Distinguisher: 11.99.0.86:0 ���� RD set to PE IP address followed by zero[…]Source: 11.99.0.86Protocol next hop: 11.99.0.86[…]Local AS: 65320 Peer AS: 65320
32 Copyright © 2013 Juniper Networks, Inc.
Local AS: 65320 Peer AS: 65320Age: 8:37 Metric2: 1 Validation State: unverified Task: BGP_65320.11.99.0.86+179AS path: ICommunities: es-import-target:22-0-0-0-0-0 ���� ES-Import Route Target – auto-
derived from ESI (byte 3 to byte 8)
Import AcceptedLocalpref: 100Router ID: 11.99.0.86Secondary Tables: __default_evpn__.evpn.0
LOOP PREVENTION ON MULTI-HOMED SEGMENTS
ESI Label is used to prevent loops on multi-homed ESI segments
ESI Label is distributed as part of Ethernet A-D Route (ESI Label Extended Community)
ESI Label is downstream assigned MPLS label in case of ingress replication
ESI Label is upstream assigned in case of P2MP LSP
34 Copyright © 2013 Juniper Networks, Inc.
CE1
PE1
PE2
MPLS
PE3
PE4
CE2
LAG LAG
HOW TO PREVENT DUPLICATE COPIES ON MULTI-HOMED SEGMENTS?
Designated Forwarder (DF) is elected for each EVI or entire Ethernet Segment.
DF is responsible for forwarding of BUM traffic
Default procedure for DF election is <ESI, EVI> allowing to load-balance BUM traffic (for different EVIs) across multiple PEs
37 Copyright © 2013 Juniper Networks, Inc.
PEs
CE1
PE1
PE2
MPLS
PE3 CE2
LAG
HOW TO LOAD BALANCE TRAFFIC TOWARDS ALL A/A PES ON THE ETHERNET SEGMENT?
EVPN introducing a concept of Aliasing.
Each PE signals that it has reachability to a given Ethernet segment (using Ethernet A-D Route)
Remote PE should install all PEs as next-hop which are attached to the same Ethernet Segment
38 Copyright © 2013 Juniper Networks, Inc.
CE1
PE1
PE2DF
MPLS
PE3 CE2
LAGMAC1
ESI1 MAC1 -> ESI1 -> (PE1, PE2)
FAST CONVERGENCE IN ACTIVE/BACKUP ATTACHED ETHERNET SEGMENT ?
EVPN introducing a concept of Backup-Path.
Each PE signals that it has reachability to a given Ethernet segment (using Ethernet A-D Route)
Remote PE should install backup paths to all further PEs which have reachability to particular Ethernet Segment
40 Copyright © 2013 Juniper Networks, Inc.
CE1
PE1
PE2DF
MPLS
PE3 CE2
LAGMAC1
ESI1 MAC1 -> ESI1 -> (PE1 BACKUP, PE2 ACTIVE)
ARP PROXY
PE can snoop ARP messages for locally attached hosts.
MAC/IP binding can be then redistributed to other PEs by using MAC Advertisement Route.
ARP REQUEST FOR IP3
41 Copyright © 2013 Juniper Networks, Inc.
CE1 PE1
PE2DF
MPLS
PE3 CE3MAC1, IP1
ARP REQUEST FOR IP3
CE2MAC2, IP2
MAC3, IP3
ARP REPLY FOR IP3
ARP REQUEST FOR IP3
ARP REPLY FOR IP3
MAC MOBILITY AND DUPLICATED MACS
Each time MAC moves to different Ethernet Segment incremented Sequence Number is included in MAC Advertisement Route by PE which is attached to the new segment
Advertisement should be disabled if local PE learns same address N times within M seconds
42 Copyright © 2013 Juniper Networks, Inc.
CE1
PE1
PE2DF
MPLS
PE3 CE2
LAGMAC1
ESI1
MAC moves from ESI1 to ESI2
MAC1
ESI2
MAC MOVE – BASED ON LATEST LEARNED MAC ADVERTISEMENT ROUTEMAC 00:00:0b:0a:01:11 initially connected to PE1:juniper@mx-re1> show evpn mac-table Routing instance : EVPN-1
MAC MAC Logical NH RTRaddress flags interface Index ID[…]00:00:0b:0a:01:11 D,SE ge-1/0/1.10
MAC 00:00:0b:0a:01:11 moves to PE2:juniper@mx2-re1> show evpn mac-tableRouting instance : EVPN-1
MAC MAC Logical NH RTRaddress flags interface Index ID[…]00:00:0b:0a:01:11 D ge - 10/0/3.20
45 Copyright © 2013 Juniper Networks, Inc.
00:00:0b:0a:01:11 D ge - 10/0/3.20
PE2 advertises new MAC address. PE1 deletes MAC add ress from local table:May 22 13:50:38.228221 EVPN instance EVPN-1 [VLAN: 200, Refcount: 3, Intfs: 2 (2 up), IRBs: 0 (0 up), Remote PEs: 1, Flags: 0x8] Received MAC advertisement route (type 2) from BGPMay 22 13:50:38.228244 EVPN instance EVPN-1 [VLAN: 200, Refcount: 3, Intfs: 2 (2 up), IRBs: 0 (0 up), Remote PEs: 1, Flags: 0x8] Processing ADD for MAC 00:00:0b :0a:01:11 from 11.99.0.86 with ESI 0, VLAN 200, lab el 301072May 22 13:50:38.228282 EVPN MAC peer EVPN- 1::200::00:00:0b:0a:01:11::11.99.0.86 [MAC: no, MAC +IPs: 0, Active: yes] CreatedMay 22 13:50:38.228325 EVPN MAC 00:00:0b:0a:01:11 ( remote) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 < Adv>] Created and added to MAC database
May 22 13:50:38.731442 EVPN MAC 00:00:0b:0a:01:11 ( local) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 <A dv>] Deleting MAC advertisement routeMay 22 13:50:38.731458 EVPN route (local) [Instance : EVPN-1, Type: MAC advertisement (2), ESI: 0, VLAN : 200] Withdrawing MAC routeMay 22 13:50:38.731543 EVPN MAC 00:00:0b:0a:01:11 ( local) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 <A dv>] Deleted from MAC database
IRB SUPPORT WITHIN EVPN
IRB allows to forward not only L2 but L3 traffic as well on the same PE
In case of multiple locations (e.g. DC locations) it is desired to use local forwarding for L3 traffic to avoid trombone effect
Each PE that acts as a Default GW for a given EVPN should advertise its Default GW IP and MAC address using MAC
46 Copyright © 2013 Juniper Networks, Inc.
advertise its Default GW IP and MAC address using MAC Advertisement Route (with Default Gateway Extended Community).
All receiving PE should reply to all ARP requests received to this IP address and should forward traffic destined to this MAC address locally
EVPN WITH IRB – EVPN MAC ROUTE WITH DEFAULT GATEWAY
juniper@mx-re0> show route table EVPN-1.evpn.0
2:11.99.0.13:200::200::84:18:88:2a:5f:f0::11.10.0.62/304 (1 entry, 1 announced)���� MAC route includes default gateway IP address
*EVPN Preference: 170
[…]
AS path: I
Communities: evpn-default-gateway ���� Default Gateway Extended Community
Route Label: 303632
47 Copyright © 2013 Juniper Networks, Inc.
Route Label: 303632
ESI: 00:00:00:00:00:00:00:00:00:00
EVPN IN OPERATION – TRAFFIC FLOW OVERVIEWBGP signaling on WAN
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS
PE 3 as a non -DF @@2
@@1
@@4
@@5
9
PE2 Drops Traffic as it’s originated from same ESI segment
@@6
@@7
@@7
@@7
48 Copyright © 2013 Juniper Networks, Inc.
DP Learning
MPLS LABEL USED FOR FORWARDING
- Label per EVI- Per EVI+VLAN
- Per MAC
PE1 receives broadcast traffic from CE1. PE1 adds PSN
and IM label and forwards 3 copies
PE 4 as DF will forward BUM
traffic into segment
BGP MAC ADV ROUTE
• EVPN NRLI• MAC M1 via
PE1
PE 3 as a non -DF for a given VLAN (EVI) will drop the
traffic
@@ @@5
BGP MAC ADVROUTE
RD ESI
MAC IP LEN
ETH TAG MAC LEN
IP ADDR MPLS LBL
@@6 @@7
49 Copyright © 2013 Juniper Networks, Inc.
VIRTUAL MOBILE TRAFFIC OPTIMIZER
VM DEFAULT GATEWAY PROBLEM
Default G/W
Data Centre (A) Data Centre (B)
50 Copyright © 2013 Juniper Networks, Inc.
PKTPKT
PKT
• VM does not update default g/w IP or MAC address• Need a mechanism to ensure traffic exits via neares t g/w
Efficient Routing with VMTOTraffic Trombones without VMTO
OPTIMIZING INTER-VLAN TRAFFIC FLOWS
52 Copyright © 2013 Juniper Networks, Inc.
PRIVATE MPLS WAN PRIVATE MPLS WAN
VLAN 10 VLAN 10 VLAN 10VLAN 10
VLAN 20
Server 1
DC 1
20.20.20.100/24
WITHOUT VMTO: EGRESS TROMBONE EFFECT
53 Copyright © 2013 Juniper Networks, Inc.
DC 2 VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
Server 2 Server 3
PRIVATE MPLS WAN
Active VRRPDG:
10.10.10.1
Standby VRRPDG:
10.10.10.1
Standby VRRPDG:
10.10.10.1
Standby VRRPDG:
10.10.10.1
Task: Server 3 in Data Center 3 needs to send packets
to Server 1 in Data Center 1.
Problem: Server 3’s active Default Gateway for VLAN 10
is in Data Center 2.
Effect: 1. Traffic must travel via Layer 2 from Data Center 3 to Data Center 2 to reach VLAN 10’s
active Default Gateway.2. The packet must reach the Default Gateway
in order to be routed towards Data Center 1. This results in duplicate traffic on WAN links and suboptimal routing – hence the “Egress
Trombone Effect.”
VLAN 20
Server 1
DC 1
20.20.20.100/24
WITH VMTO: NO EGRESS TROMBONE EFFECT
54 Copyright © 2013 Juniper Networks, Inc.
DC 2 VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
Server 2 Server 3
PRIVATE MPLS WAN
Active RVIDG:
10.10.10.1
Active RVIDG:
10.10.10.1
Active RVIDG:
10.10.10.1
Active RVIDG:
10.10.10.1
Task: Server 3 in Data Center 3 needs to send packets
to Server 1 in Data Center 1.
Solution: Virtualize and distribute the Default Gateway
so it is active on every router that participates in the VLAN.
Effect: 1. Egress packets can be sent to any router on
VLAN 10, allowing the routing to be done in the local data center. This eliminates the
“Egress Trombone Effect” and creates the most optimal forwarding path for the inter-
data center traffic.
VM EGRESS TRAFFIC OPTIMIZATION
EVPN advantages over VPLS:- No need for VRRP, Multi-homing, MC-LAG (less machinery and
protocol dependencies)
- IRB within EVPN VRF is configured on all PEs with a same IP address (copy&paste IRB config on all PEs)
- Each PE has a mapping between Default GW IP and all PEs MACs
- If VM moves from DC1 to DC2 it continue to use “old” MAC address from PE located in DC1. However, both PEs in DC2 forward traffic
56 Copyright © 2013 Juniper Networks, Inc.
from PE located in DC1. However, both PEs in DC2 forward traffic destined to this MAC locally.
IRB MAC on MX240-4IRB MAC on MX480-3IRB MAC on MX480-4
VLAN 20
Server 1
DC 1
20.20.20.100/24
WITHOUT VMTO: INGRESS TROMBONE EFFECT
Route Mask Cost Next Hop
10.10.10.0 24 5 Datacenter 2
10.10.10.0 24 10 Datacenter 3
DC 1’s Edge Router Table Without VMTO
57 Copyright © 2013 Juniper Networks, Inc.
DC 2VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
Server 2 Server 3
PRIVATE MPLS WAN
Task: Server 1 in Data Center 1 needs to send packets
to Server 3 in Data Center 3.
Problem: Data Center 1’s edge router prefers the path to Data Center 2 for the 10.10.10.0/24 subnet. It
has no knowledge of individual host IPs.
Effect:1. Traffic from Server 1 is first routed across
the WAN to Data Center 2 due to a lower cost route for the 10.10.10.0/24 subnet.
2. Then the edge router in Data Center 2 will send the packet via Layer 2 to Data Center 3.
10.10.10.0/24 Cost 1010.10.10.0/24 Cost
5
VLAN 20
Server 1
DC 1
20.20.20.100/24
WITH VMTO: NO INGRESS TROMBONE EFFECT
Route Mask Cost Next Hop
10.10.10.0 24 5 Datacenter 2
10.10.10.0 24 10 Datacenter 3
10.10.10.100 32 5 Datacenter 2
10.10.10.200 32 5 Datacenter 3
DC 1’s Edge Router Table WITH VMTO
10.10.10.100/32 Cost 510.10.10.200/32 Cost 5
58 Copyright © 2013 Juniper Networks, Inc.
DC 2VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
Server 2 Server 3
PRIVATE MPLS WAN
Effect: 1. Ingress traffic destined for Server 3 is sent
directly across the WAN from Data Center 1 to Data Center 3. This eliminates the “Ingress
Trombone Effect” and creates the most optimal forwarding path for the inter-data
center traffic.
Task: Server 1 in Data Center 1 needs to send packets
to Server 3 in Data Center 3.
Solution: In addition to sending a summary route of
10.10.10.0/24, the data center edge routers also send host routes which represent the location
of local servers.
10.10.10.100/32 Cost 5
10.10.10.0/24 Cost 5
10.10.10.0/24 Cost 10
72 Copyright © 2013 Juniper Networks, Inc.
SUMMARY
EVPN FORWARDING SUMMARY
MPLS or IP
detoursBGP Control Plane based learning on WANBGP Control Plane based learning on WAN
DP learning over LAN
DP learning over LAN
DP learning over LAN
DP learning over LAN
LAG
LAG
MAC1…….……...LAN PortsMAC11………MPLS nexthop
VLAN 1MAC1
VLAN 1MAC11
VLAN 2
MAC2…….……...LAN PortsMAC22….……MPLS nexthop
MAC2……..….MPLS nexthopMAC22….……..…LAN ports
MAC1…………MPLS nexthopMAC11…………...LAN ports
73 Copyright © 2013 Juniper Networks, Inc.
MX Series
MX Series
detours
MPLS transport label(s) including detour or IP
transport label
MPLS transport label(s) including detour or IP
transport label
Service labelService label
Ethernet FrameEthernet Frame
P2P connections for unicast traffic
P2P connections for unicast traffic
P2MP connections for
multicast or unknown traffic
P2MP connections for
multicast or unknown traffic
Hash based LB on
Ethernet switch
Hash based LB on
Ethernet switch
VLAN 2MAC2
VLAN 2MAC22
EVPN VS VPLSNEXTGEN Cloud DC Attributes for L2-Stretch EVPN VPLS
Flexible physical network topologies (hub-n-spoke, mesh, ring) � �
Scale to 100K+ hosts within and across multiple DCs � �
Active-Active points of attachment (hosts, routers) �
VPN (secure isolation, overlapping MAC, IP addresses) � �
Near Hitless Host Mobility without renumbering L2 and L3 addresses �
77 Copyright © 2013 Juniper Networks, Inc.
�
Ability to span VLANs across racks in different locations � �
Controlled learning with Policies �
Minimize or eliminate flooding of unknown unicast �
Fast convergence from edge failures based on local repair �
Multicast at scale with ability to trade bandwidth vs. state � �
Value Adds: Auto-Cfg, Non-Ethernet links, FRR on transit links � �
Top Related