PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center

ETHERNET VPN

CZY TYLKO DLA DATA CENTER?

Emil Gągała

PLNOG, 30.09.2014

WHAT IS ETHERNET VPN

A new standards based protocol to

inter-connects L2 domains

Improves network efficiency

Multi-vendor industry wide

BGP based state

exchange

WAN

2 Copyright © 2013 Juniper Networks, Inc.

LAG

Multi-vendor industry wide

initiative -- JNPR, CSCO, ALU, ATT,

Verizon, Bloomberg ….

Ideally suited for Datacenter

Interconnectivity but ...

... NOT only

EVPN router

EVPN router

LAN

ETHERNET VPN SUPPORT

Authors listed on draft-ietf-l2vpn-evpn-08 include:

� A. Sajassi – Cisco Systems� J. Drake – Juniper Networks� W. Henderickx – Alcatel-Lucent� R. Aggarwal– Arktan


� R. Aggarwal– Arktan� N. Bitar – Verizon� A. Isaac – Bloomberg� J. Uttaro – AT&T


VPLS TO EVPN COMPARISON

VPLS EMULATES AN ETHERNET SWITCH

Common Characteristics:

�Forwarding of Ethernet Frames�Forwarding of Unicast frames with an unknown MAC address�Replication of broadcast and multicast frames�Loop prevention�Dynamic Learning of MAC address


Site 1 Site 2

�Dynamic Learning of MAC address

Virtual Private LAN Service (VPLS) provides VLAN Extension over a shared IP/MPLS network.

VPLS CHARACTERISTICS

Full Mesh

VLAN Separation

Any-to-Any connectivity regardless of physical path

Separate VPLS instances per VLAN. Allows network-wide segmentation with very large scale


VLAN Separation

Provisioning

Multicast, Broadcast and Flooding

Availability

with very large scale

New site Auto Discovery

Scale forwarding with Multicast & Point-to-Multipoint capabilities

Underlying MPLS offers ECMP, Fast Reroute

EVPN provides VLAN Extension over a shared IP/MPLS network.

EVPN REQUIREMENTS (ON TOP OF VPLS)

All-Active Multi-Homing

Better Control Over MAC

All available paths should be used (CE-PE, PE-PE)

MAC learning happens in control plane


Better Control Over MAC Learning

ARP/ND Flooding Minimization

L3 Egress Traffic Forwarding Optimization

Reducing Unknown Unicast Flooding

MAC learning happens in control plane

Additional attributes added during MAC advertisement

Usage of Default Gateway Extended Community

By using MAC learning in control plane


EVPN: VALUE PROPOSITION AND USE CASES

WHY ETHERNET VPN

EVPN Use Cases:

� Next generation L2VPN technology that replaces VPLS, VPWS

� As DC Interconnect – allowing L2 stretch between two data

centers over WAN

� EVPN as control plane with VxLAN IP overlay DC networks

Which customers are interested in EVPN and why ?:


Which customers are interested in EVPN and why ?:

� Service providers that offer E-LAN / E-LINE services

� Today, use a PE router for L2 services with VPLS, VPWS

� EVPN technology improves their service offering

� Data Center Builders – SPs, Enterprises, Content providers

� Today, use a DC WAN Edge Router

� EVPN allows multi-tenant L2 stretch between DCs and within DC

USE CASE #1: EVPN FOR NEXT GENERATION ETHERNET SERVICES

BGP signaling on WAN exchange MAC/IP routes

BGP signaling on WAN exchange MAC/IP routesEVPN

PE2

EVPNPE1

EVPN PE3

EVPNPE4CE

CE

MPLS 9

MP-BGP


Benefits:• Allows more efficient, feature rich E-LAN and E-LINE services

• Solves shortcomings of VPLS; offers IP VPN like policy control

• Supports explosive traffic growth

• Active/Active multi-homing with load balancing

• Improves network efficiency

• Minimizes flooding of BUM traffic/improves MAC learning

USE CASE #2: EVPN FOR DATA CENTER INTERCONNECT

VLAN 1MAC1

VLAN 1MAC11

Data Plane LearningData Plane Learning BGP Control Plane based learning on WANBGP Control Plane based learning on WAN Data Plane LearningData Plane Learning

E-VPN CloudE-VPN Cloud

Legacy L2 Cloud

Legacy L2 Cloud

VXLAN CloudVXLAN Cloud


VLAN 2MAC 2

VLAN 2MAC22

Data Center Site1 Data Center Site 2 Data Center Interconnect

Benefits:• Seamless interconnect for DCI – L3 aware L2 stretch between DCs

• Seamless workload migration - VM mobility across DCs

• Wide Applicability – Interconnects Native L2 and overlay DC

technologies like VxLAN, MPLS in DC

USE CASE #3: EVPN-VXLAN FOR DC OVERLAY

EVPN acts as control plane protocol� VNID used in place of Eth tag ID for EVPN

signaling

VxLAN is data plane encapsulation

Benefits of EVPN-VxLAN DC Overlay:• Allows Simple All IP fabric in DC

Overlay environment

IP Fabric

Network Orchestrator

Management Plane API needed only


• Allows Simple All IP fabric in DC• No need for multi-layers L2 setup in DCs

• Allows L2 connectivity for VMs / applications

• Deliver a L2VPN straight to a hypervisor

• L2 Multi-tenancy in all IP DC• Each tenant can have 4094 VNIDs

• VXLAN Tunnel Endpoints (VTEP) exist on

networks equipment and hypervisors

• All benefits of EVPN applicable in a DC

TOR

VDSVDS

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM


EVPN BUILDING BLOCKS & OPERATIONS

EVPN INSTANCE AND EVPN SERVICE INTERFACES

EVPN Instance (EVI) represents a VPN in the MPLS/IP network

One or multiple broadcast domains can be part of the same EVI

Each broadcast domain is uniquely identified inside EVI by Ethernet Tag

PEVLANs

(all or N)

PEVLANs

PEVLANs


VLAN Bundle SI/Port Based SI

(all or N)

BD EVI

• All CEs MUST use same CE-VIDs

• Encap. MPLS frames MUST remain tagged

• No Tag translation allowed

VLAN Based SI

BD EVI

• One-to-One mapping• Different CE-VIDs can be

used on CEs• Tag translation allowed• Ethernet Tag is set to 0

BD EVI

VLAN Aware Bundle SI

BD

• Many-to-One mapping• Different CE-VIDs can be

used on CEs BUT• Normalized tag MUST be used• Ethernet tag == NormalizedTag

EVIBD

EVPN BASIC CONFIGURATION –VLAN-BASED SERVICE INTERFACE

interfaces {ge-1/0/1 {

flexible-vlan-tagging;encapsulation extended-vlan-bridge;unit 10 {

vlan-id 10;family bridge;

}}ge-1/0/2 {

flexible-vlan-tagging;encapsulation extended - vlan - bridge;

protocols {bgp {

group iBGP-EVPN {type internal;local-address 11.99.0.13;family evpn {

signaling;}neighbor 11.99.0.86;

}}

}


encapsulation extended - vlan - bridge;unit 30 {


}}

}

}routing-instances {

EVPN-1 {instance-type evpn;vlan-id 200; interface ge-1/0/1.10;interface ge-1/0/2.30;route-distinguisher 11.99.0.13:200;vrf-target target:65320:200;protocols {

evpn;}

}}

EVPN INFORMATION EXCHANGE OVERVIEW

MPLS or IP

detours

LAG

LAG

Route Reflector

VLAN 1MAC1, IP1.1

VLAN 2MAC2, IP2.1

VLAN 1MAC11, IP1.11

VLAN 2MAC22, IP2.22


ESI

Route Distinguisher

Ethernet Tag

MAC Address

EVPN reachability advertisement

IPv4 or IPv6 Address

Service Tag

• EVPN advertises MAC (L2) and IP (ARP) bindings for each segment along with service tags

• Allowing Control Plane based L2 and ARP learning• Minimizes flooding across WAN• Allows proxy-ARP to respond queries locally

• IRB MAC address exchange allows same gateway MAC address across sites

• VM mobility: egress traffic optimization

ETHERNET TAG IDENTIFIER

� An Ethernet Tag ID is a 32-bit field � Contains a 12-bit or a 24-bit identifier to identify a broadcast

domain in an EVPN instance. � 12-bit identifier is used for normalized VLAN ID for EVPN (MPLS)� 24-bit identifier is used for VNID for EVPN-VxLAN� 24- bit identified is used for I-SID for PBB-EVPN.

� An EVI can have one or more broadcast domains – VLANs -


� An EVI can have one or more broadcast domains – VLANs -assigned to a given EVPN instance

ETHERNET SEGMENT IDENTIFIER (ESI)

If CE is multi-homed to two or more PEs, the set of Ethernet links constitutes an “Ethernet Segment”.

A/P or A/A multi-homing is supported

An Ethernet Segment MUST have a non-reserved ESI that is unique network wide. ESI can be auto-provisioned


CE

PE1

PE2ESI Auto-Provisioning with MC-LAG

MPLS

System Prio System MAC Address Port Key

CE PE1

PE2ESI Auto-Provisioning with MC-LAG

BPDU

MPLS

Bridge Prio Root Bridge MAC 0x0000

CE BPDUL2

EVPN ACTIVE/STANDBY MULTI-HOMING: CONFIGURATION


flexible-vlan-tagging;encapsulation extended-vlan-bridge;esi {

00:10:11:00:00:00:00:00:00:01;single-active;

}unit 10 {



vlan-tagging;encapsulation extended-vlan-bridge;esi {

00:20:22:00:00:00:00:00:00:02;single-active;

}unit 20 {


PE1: PE2:


family bridge;}

}ge-1/0/2 {

flexible-vlan-tagging;encapsulation extended-vlan-bridge;esi {

00:20:22:00:00:00:00:00:00:02;single-active;

}unit 30 {


}}

family bridge;}

}}

EVPN NLRI

BGP AFI 25 (L2VPN)/ SAFI 70 (EVPN)� Format = Route Type : Length : Route-type Specific

EVPN Route Types1) Ethernet Auto-Discovery (A-D) Route

� Used for fast convergence (withdrawal), and active/active multi-homing (split-horizon label)

� 2 variants: per ESI and per EVI


� 2 variants: per ESI and per EVI

2) MAC/IP Advertisement Route� Used for remote MAC address learning, known unicast traffic

3) Inclusive Multicast Route� Used for BUM (broadcast, unknown unicast, multicast) traffic

4) Ethernet Segment Route� Used for auto-discovery of multi-homed Ethernet segments and Designated

Forwarder election

ETHERNET AUTO-DISCOVERY PER ESI – TYPE 1BGP signaling on WAN

PE2

PE1 (DF)

PE3

PE4

CE

CE

MPLS 9

Loop Avoidance via split horizonFast Convergence

Ethernet AD route per ESI announces

ESI mode


• Ethernet AD route per ESI signals All active or sin gle active mode of operation for a multi-homed CE

• Advertises Split Horizon label for L2 BUM traffic• Enables forwarding state for the advertised ESI• On withdrawal of AD route per ESI, all PEs adjust N Hs or invalidate MAC

routes associated with that ESI, allowing rapid con vergence

ESI

RD

Ethernet Tag

Service Tag

Auto Discovery message per L2

Segment

ETHERNET AUTO-DISCOVERY PER EVI – TYPE 1MP BGP signaling between PEs

PE2

PE1 (DF)

PE3

PE4

CE

CE

MPLS 9

Allows Load-balancingRapid convergence

Ethernet AD route per EVI includes all connected ESIs for

that EVI


• Each Multi-homed PE advertises AD route per EVI for all connected ESIs to advertise “service label” (aka “aliasing label ”)

• Ethernet A-D per EVI route is used for ’Aliasing’ ( load-balancing)• Remote PEs use AD per EVI route and MAC route toget her to load-

balance traffic • Load balancing for L2 as well as L3 traffic

• AD route per EVI and AD route per ESI BOTH are reqd for multi-homing

ESI

RD of EVI

Ethernet Tag

Service Tag

Auto Discovery message per EVI

EVPN ROUTE TYPE 1 –ETHERNET AUTO-DISCOVERY ROUTE

juniper@mx-re1> show route table EVPN-1.evpn.0 detail

1:11.99.0.86:0::202200000000000002::0/304 (1 entry, 1 announced) �� Format = Type:RD::ESI::Label/304

*BGP Preference: 170/-101Route Distinguisher: 11.99.0.86:0 �� RD set to PE IP address followed by zero

[…]Source: 11.99.0.86Protocol next hop: 11.99.0.86[…]


[…]Local AS: 65320 Peer AS: 65320Age: 35:05 Metric2: 1 Validation State: unverified Task: BGP_65320.11.99.0.86+179Announcement bits (1): 0-EVPN-1-evpn AS path: ICommunities: target:65320:200 esi-label:100000(label 0) �� ESI Label:

flag = 0: active/active, flag = 1: active/standbyImport AcceptedLocalpref: 100Router ID: 11.99.0.86Primary Routing Table bgp.evpn.0

MAC ROUTE – TYPE 2MP BGP signaling between EVPN PEs

PE2

PE1 (DF)

PE3

PE4

CE

CE

MPLS 9

Establishes Reachability Each PE learns MAC on CE-PE link and advertises

its reachability in EVPN MAC route


ESI

RD of EVI

Ethernet Tag

MAC Address

MAC reachability advertisement

IPv4 or IPv6 Address

Service Tag

• Advertises host MAC (and host IP) reachability with “service label”

• Allows Control Plane based MAC learning for remote PEs• On MX, service label is same as one advertised in AD per EVI route• Minimizes flooding across WAN• Allows PE to do proxy-ARP for remote hosts locally

• IRB MAC address route has default GW extended commu nity• Used in VM motion when default GW of VM remains sam e

• If IRB MACs and IP are same across MH PEs, avoids f looding after node failure

RT

INCLUSIVE MULTICAST ROUTE – TYPE 3BGP signaling on WAN

PE2

PE1 (DF)

PE3

PE4

CE

CE

MPLS 9

MP-BGP

Sets up path for BUM trafficPer VLAN per EVI


� Allows PE to send BUM traffic from a CE on a VLAN i n an EVI, to all the other PEs that span that VLAN in that EVPN instance

• Uses Existing MVPN defined constructs for signalli ng and transport• P2MP Tunnel : If advertising PE uses a P-Multicast tree for EVPN,

the PMSI Tunnel attribute MUST contain tree identit y • Ingres Replication : Route includes PMSI Tunnel att ribute with

Tunnel Type set to Ingress Replication and Tunnel I D as PE address.

• Able to carry the traffic of more than one EVPN ins tance on the same tree using ’Aggregation’

Eth TAG

RD of EVI

Advertising PE IP

Next Hop (PE IP)

Inclusive multicast Ethernet TAG route

Route Target

PMSI Tunnel Attr

EVPN ROUTES TYPE 2 & TYPE 3juniper@mx-re1> show route table EVPN-1.evpn.0

2:11.99.0.13:200::200::00:00:0a:0a:02:01/304 *[EVPN/170] 00:04:36

Indirect2:11.99.0.13:200::200::00:00:0b:0a:00:12/304

*[EVPN/170] 00:04:36Indirect

2:11.99.0.13:200::200::00:00:0b:0a:01:11/304*[EVPN/170] 00:04:36

Indirect3:11.99.0.13:200::200::11.99.0.13/304

*[EVPN/170] 00:53:47Indirect

Local MACAdvertisement Routes

(Format = Type:RD::Eth-Tag-iD::MAC/304)

Local Inclusive Multicast

Ethernet Tag Route


Indirect

2:11.99.0.86:200::200::00:00:0b:0a:00:0a/304 *[BGP/170] 00:26:58, localpref 100, from 11.99.0.86

AS path: I, validation-state: unverified> to 11.0.100.18 via xe-2/0/0.10

to 11.0.100.22 via xe-2/0/1.102:11.99.0.86:200::200::00:00:0b:0a:00:0b/304

*[BGP/170] 00:26:58, localpref 100, from 11.99.0.86AS path: I, validation-state: unverified

> to 11.0.100.18 via xe-2/0/0.10to 11.0.100.22 via xe-2/0/1.10

3:11.99.0.86:200::200::11.99.0.86/304 *[BGP/170] 00:27:01, localpref 100, from 11.99.0.86

AS path: I, validation-state: unverified> to 11.0.100.18 via xe-2/0/0.10

to 11.0.100.22 via xe-2/0/1.10

Ethernet Tag Route

Remote MAC

Advertisement Routes

Remote Inclusive Multicast

Ethernet Tag Route

ETHERNET SEGMENT ROUTE – TYPE 4BGP signaling on WAN

PE2

PE1 (DF)

PE3

PE4CE

CE

MPLS 9

Simplifies Configuration

Loop Avoidance via DF selection

Ethernet Segment route sent to every EVPN peer for ESI

discovery


• Ethernet Segment Identifier allows multi-homing of CEs to PE• PEs connected to the same Ethernet segment discover each

other by exchanging of Ethernet Segment route.• Include ES-Import extended community with value aut o-

derived from the MAC address portion of ESI• Only PEs that host that ESI import this route• DF selection is carried out based on ES routes

ESI

RD

IP Addr Length

Originator’s IP Addr

ES Route

EVPN ACTIVE/STANDBY MULTI-HOMING: ROUTE TYPE 4 – ETHERNET SEGMENT ROUTE

juniper@mx-re1> show route table bgp.evpn.0 detail

4:11.99.0.86:0::202200000000000002:11.99.0.86/304 (1 entry, 0 announced) �� Format = Type:RD::ESI:Originating-Router-IP/304

*BGP Preference: 170/-101Route Distinguisher: 11.99.0.86:0 �� RD set to PE IP address followed by zero[…]Source: 11.99.0.86Protocol next hop: 11.99.0.86[…]Local AS: 65320 Peer AS: 65320


Local AS: 65320 Peer AS: 65320Age: 8:37 Metric2: 1 Validation State: unverified Task: BGP_65320.11.99.0.86+179AS path: ICommunities: es-import-target:22-0-0-0-0-0 �� ES-Import Route Target – auto-

derived from ESI (byte 3 to byte 8)

Import AcceptedLocalpref: 100Router ID: 11.99.0.86Secondary Tables: __default_evpn__.evpn.0

LOOP PREVENTION ON MULTI-HOMED SEGMENTS

ESI Label is used to prevent loops on multi-homed ESI segments

ESI Label is distributed as part of Ethernet A-D Route (ESI Label Extended Community)

ESI Label is downstream assigned MPLS label in case of ingress replication

ESI Label is upstream assigned in case of P2MP LSP


CE1

PE1

PE2

MPLS

PE3

PE4

CE2

LAG LAG

HOW TO PREVENT DUPLICATE COPIES ON MULTI-HOMED SEGMENTS?

Designated Forwarder (DF) is elected for each EVI or entire Ethernet Segment.

DF is responsible for forwarding of BUM traffic

Default procedure for DF election is <ESI, EVI> allowing to load-balance BUM traffic (for different EVIs) across multiple PEs


PEs

CE1

PE1

PE2

MPLS

PE3 CE2

LAG

HOW TO LOAD BALANCE TRAFFIC TOWARDS ALL A/A PES ON THE ETHERNET SEGMENT?

EVPN introducing a concept of Aliasing.

Each PE signals that it has reachability to a given Ethernet segment (using Ethernet A-D Route)

Remote PE should install all PEs as next-hop which are attached to the same Ethernet Segment


CE1

PE1

PE2DF

MPLS

PE3 CE2

LAGMAC1

ESI1 MAC1 -> ESI1 -> (PE1, PE2)

FAST CONVERGENCE IN ACTIVE/BACKUP ATTACHED ETHERNET SEGMENT ?

EVPN introducing a concept of Backup-Path.

Each PE signals that it has reachability to a given Ethernet segment (using Ethernet A-D Route)

Remote PE should install backup paths to all further PEs which have reachability to particular Ethernet Segment


CE1

PE1

PE2DF

MPLS

PE3 CE2

LAGMAC1

ESI1 MAC1 -> ESI1 -> (PE1 BACKUP, PE2 ACTIVE)

ARP PROXY

PE can snoop ARP messages for locally attached hosts.

MAC/IP binding can be then redistributed to other PEs by using MAC Advertisement Route.

ARP REQUEST FOR IP3


CE1 PE1

PE2DF

MPLS

PE3 CE3MAC1, IP1

ARP REQUEST FOR IP3

CE2MAC2, IP2

MAC3, IP3

ARP REPLY FOR IP3

ARP REQUEST FOR IP3

ARP REPLY FOR IP3

MAC MOBILITY AND DUPLICATED MACS

Each time MAC moves to different Ethernet Segment incremented Sequence Number is included in MAC Advertisement Route by PE which is attached to the new segment

Advertisement should be disabled if local PE learns same address N times within M seconds


CE1

PE1

PE2DF

MPLS

PE3 CE2

LAGMAC1

ESI1

MAC moves from ESI1 to ESI2

MAC1

ESI2

MAC MOVE – BASED ON LATEST LEARNED MAC ADVERTISEMENT ROUTEMAC 00:00:0b:0a:01:11 initially connected to PE1:juniper@mx-re1> show evpn mac-table Routing instance : EVPN-1

MAC MAC Logical NH RTRaddress flags interface Index ID[…]00:00:0b:0a:01:11 D,SE ge-1/0/1.10

MAC 00:00:0b:0a:01:11 moves to PE2:juniper@mx2-re1> show evpn mac-tableRouting instance : EVPN-1

MAC MAC Logical NH RTRaddress flags interface Index ID[…]00:00:0b:0a:01:11 D ge - 10/0/3.20


00:00:0b:0a:01:11 D ge - 10/0/3.20

PE2 advertises new MAC address. PE1 deletes MAC add ress from local table:May 22 13:50:38.228221 EVPN instance EVPN-1 [VLAN: 200, Refcount: 3, Intfs: 2 (2 up), IRBs: 0 (0 up), Remote PEs: 1, Flags: 0x8] Received MAC advertisement route (type 2) from BGPMay 22 13:50:38.228244 EVPN instance EVPN-1 [VLAN: 200, Refcount: 3, Intfs: 2 (2 up), IRBs: 0 (0 up), Remote PEs: 1, Flags: 0x8] Processing ADD for MAC 00:00:0b :0a:01:11 from 11.99.0.86 with ESI 0, VLAN 200, lab el 301072May 22 13:50:38.228282 EVPN MAC peer EVPN- 1::200::00:00:0b:0a:01:11::11.99.0.86 [MAC: no, MAC +IPs: 0, Active: yes] CreatedMay 22 13:50:38.228325 EVPN MAC 00:00:0b:0a:01:11 ( remote) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 < Adv>] Created and added to MAC database

May 22 13:50:38.731442 EVPN MAC 00:00:0b:0a:01:11 ( local) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 <A dv>] Deleting MAC advertisement routeMay 22 13:50:38.731458 EVPN route (local) [Instance : EVPN-1, Type: MAC advertisement (2), ESI: 0, VLAN : 200] Withdrawing MAC routeMay 22 13:50:38.731543 EVPN MAC 00:00:0b:0a:01:11 ( local) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 <A dv>] Deleted from MAC database

IRB SUPPORT WITHIN EVPN

IRB allows to forward not only L2 but L3 traffic as well on the same PE

In case of multiple locations (e.g. DC locations) it is desired to use local forwarding for L3 traffic to avoid trombone effect

Each PE that acts as a Default GW for a given EVPN should advertise its Default GW IP and MAC address using MAC


advertise its Default GW IP and MAC address using MAC Advertisement Route (with Default Gateway Extended Community).

All receiving PE should reply to all ARP requests received to this IP address and should forward traffic destined to this MAC address locally

EVPN WITH IRB – EVPN MAC ROUTE WITH DEFAULT GATEWAY

juniper@mx-re0> show route table EVPN-1.evpn.0

2:11.99.0.13:200::200::84:18:88:2a:5f:f0::11.10.0.62/304 (1 entry, 1 announced)�� MAC route includes default gateway IP address

*EVPN Preference: 170

[…]

AS path: I

Communities: evpn-default-gateway �� Default Gateway Extended Community

Route Label: 303632


Route Label: 303632

ESI: 00:00:00:00:00:00:00:00:00:00

EVPN IN OPERATION – TRAFFIC FLOW OVERVIEWBGP signaling on WAN

PE2

PE1 (DF)

PE3

PE4

CE

CE

MPLS

PE 3 as a non -DF @@2

@@1

@@4

@@5

9

PE2 Drops Traffic as it’s originated from same ESI segment

@@6

@@7

@@7

@@7


DP Learning

MPLS LABEL USED FOR FORWARDING

- Label per EVI- Per EVI+VLAN

- Per MAC

PE1 receives broadcast traffic from CE1. PE1 adds PSN

and IM label and forwards 3 copies

PE 4 as DF will forward BUM

traffic into segment

BGP MAC ADV ROUTE

• EVPN NRLI• MAC M1 via

PE1

PE 3 as a non -DF for a given VLAN (EVI) will drop the

traffic

@@ @@5

BGP MAC ADVROUTE

RD ESI

MAC IP LEN

ETH TAG MAC LEN

IP ADDR MPLS LBL

@@6 @@7


VIRTUAL MOBILE TRAFFIC OPTIMIZER

VM DEFAULT GATEWAY PROBLEM

Default G/W

Data Centre (A) Data Centre (B)


PKTPKT

PKT

• VM does not update default g/w IP or MAC address• Need a mechanism to ensure traffic exits via neares t g/w

Efficient Routing with VMTOTraffic Trombones without VMTO

OPTIMIZING INTER-VLAN TRAFFIC FLOWS


PRIVATE MPLS WAN PRIVATE MPLS WAN

VLAN 10 VLAN 10 VLAN 10VLAN 10

VLAN 20

Server 1

DC 1

20.20.20.100/24

WITHOUT VMTO: EGRESS TROMBONE EFFECT


DC 2 VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

Server 2 Server 3

PRIVATE MPLS WAN

Active VRRPDG:

10.10.10.1

Standby VRRPDG:

10.10.10.1

Standby VRRPDG:

10.10.10.1

Standby VRRPDG:

10.10.10.1

Task: Server 3 in Data Center 3 needs to send packets

to Server 1 in Data Center 1.

Problem: Server 3’s active Default Gateway for VLAN 10

is in Data Center 2.

Effect: 1. Traffic must travel via Layer 2 from Data Center 3 to Data Center 2 to reach VLAN 10’s

active Default Gateway.2. The packet must reach the Default Gateway

in order to be routed towards Data Center 1. This results in duplicate traffic on WAN links and suboptimal routing – hence the “Egress

Trombone Effect.”

VLAN 20

Server 1

DC 1

20.20.20.100/24

WITH VMTO: NO EGRESS TROMBONE EFFECT


DC 2 VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

Server 2 Server 3

PRIVATE MPLS WAN

Active RVIDG:

10.10.10.1

Active RVIDG:

10.10.10.1

Active RVIDG:

10.10.10.1

Active RVIDG:

10.10.10.1



Solution: Virtualize and distribute the Default Gateway

so it is active on every router that participates in the VLAN.

Effect: 1. Egress packets can be sent to any router on

VLAN 10, allowing the routing to be done in the local data center. This eliminates the

“Egress Trombone Effect” and creates the most optimal forwarding path for the inter-

data center traffic.

VM EGRESS TRAFFIC OPTIMIZATION

EVPN advantages over VPLS:- No need for VRRP, Multi-homing, MC-LAG (less machinery and

protocol dependencies)

- IRB within EVPN VRF is configured on all PEs with a same IP address (copy&paste IRB config on all PEs)

- Each PE has a mapping between Default GW IP and all PEs MACs

- If VM moves from DC1 to DC2 it continue to use “old” MAC address from PE located in DC1. However, both PEs in DC2 forward traffic


from PE located in DC1. However, both PEs in DC2 forward traffic destined to this MAC locally.

IRB MAC on MX240-4IRB MAC on MX480-3IRB MAC on MX480-4

VLAN 20

Server 1

DC 1

20.20.20.100/24

WITHOUT VMTO: INGRESS TROMBONE EFFECT

Route Mask Cost Next Hop

10.10.10.0 24 5 Datacenter 2

10.10.10.0 24 10 Datacenter 3

DC 1’s Edge Router Table Without VMTO


DC 2VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

Server 2 Server 3

PRIVATE MPLS WAN



Problem: Data Center 1’s edge router prefers the path to Data Center 2 for the 10.10.10.0/24 subnet. It

has no knowledge of individual host IPs.

Effect:1. Traffic from Server 1 is first routed across

the WAN to Data Center 2 due to a lower cost route for the 10.10.10.0/24 subnet.

2. Then the edge router in Data Center 2 will send the packet via Layer 2 to Data Center 3.

10.10.10.0/24 Cost 1010.10.10.0/24 Cost

5

VLAN 20

Server 1

DC 1

20.20.20.100/24

WITH VMTO: NO INGRESS TROMBONE EFFECT

Route Mask Cost Next Hop

10.10.10.0 24 5 Datacenter 2

10.10.10.0 24 10 Datacenter 3

10.10.10.100 32 5 Datacenter 2

10.10.10.200 32 5 Datacenter 3

DC 1’s Edge Router Table WITH VMTO

10.10.10.100/32 Cost 510.10.10.200/32 Cost 5


DC 2VLAN 10

10.10.10.100/24

DC 3

10.10.10.200/24

VLAN 10

Server 2 Server 3

PRIVATE MPLS WAN

Effect: 1. Ingress traffic destined for Server 3 is sent

directly across the WAN from Data Center 1 to Data Center 3. This eliminates the “Ingress

Trombone Effect” and creates the most optimal forwarding path for the inter-data

center traffic.



Solution: In addition to sending a summary route of

10.10.10.0/24, the data center edge routers also send host routes which represent the location

of local servers.

10.10.10.100/32 Cost 5

10.10.10.0/24 Cost 5

10.10.10.0/24 Cost 10


SUMMARY

EVPN FORWARDING SUMMARY

MPLS or IP

detoursBGP Control Plane based learning on WANBGP Control Plane based learning on WAN

DP learning over LAN




LAG

LAG

MAC1…….……...LAN PortsMAC11………MPLS nexthop

VLAN 1MAC1

VLAN 1MAC11

VLAN 2

MAC2…….……...LAN PortsMAC22….……MPLS nexthop

MAC2……..….MPLS nexthopMAC22….……..…LAN ports

MAC1…………MPLS nexthopMAC11…………...LAN ports


MX Series

MX Series

detours

MPLS transport label(s) including detour or IP

transport label

MPLS transport label(s) including detour or IP

transport label

Service labelService label

Ethernet FrameEthernet Frame

P2P connections for unicast traffic

P2P connections for unicast traffic

P2MP connections for

multicast or unknown traffic

P2MP connections for

multicast or unknown traffic

Hash based LB on

Ethernet switch

Hash based LB on

Ethernet switch

VLAN 2MAC2

VLAN 2MAC22

EVPN VS VPLSNEXTGEN Cloud DC Attributes for L2-Stretch EVPN VPLS

Flexible physical network topologies (hub-n-spoke, mesh, ring) � �

Scale to 100K+ hosts within and across multiple DCs � �

Active-Active points of attachment (hosts, routers) �

VPN (secure isolation, overlapping MAC, IP addresses) � �

Near Hitless Host Mobility without renumbering L2 and L3 addresses �


�

Ability to span VLANs across racks in different locations � �

Controlled learning with Policies �

Minimize or eliminate flooding of unknown unicast �

Fast convergence from edge failures based on local repair �

Multicast at scale with ability to trade bandwidth vs. state � �

Value Adds: Auto-Cfg, Non-Ethernet links, FRR on transit links � �

PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center

Internet

Transcript of PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center