Press J to jump to the feed. Press question mark to learn the rest of the keyboard shortcuts
27
Posted byCCNA R&S2 years ago
Archived

What is Metro Ethernet WAN Service?

Two weeks ago I did a write up on What is MPLS as WAN service? from the point of the enterprise business. To continue the private WANs subject, this post is about Metro Ethernet as WAN service from the enterprise point of view, also called L2VPN / Layer 2 VPN.

I will use similar scenario as in the previous thread.

What is Metro Ethernet WAN Service?

You have two sites across the country - SITE-1 and SITE-2. A site can be branch office, HQ, DC, it doesn't really matter but the main point is to have interconnection between the sites with SLA, guaranteed bandwidth, latency and quick time to resolve in case there is a problem with the ISP. The standard consumer Internet service won't do because the bandwidth and the latency aren't guaranteed and the time to fix is way longer than you can afford. An ISP offers you L3VPN (MPLS as WAN service) and L2VPN (Metro Ethernet). The L3VPN sounds good but you don't want to exchange any routing information with the ISP so you choose Metro Ethernet (L2VPN). With this type of service, the ISP responsibility is to get your frames from SITE-1 to SITE-2 in accordance with the SLA. And re-encapsulate the frames if necessary (I'll get to that later).

I'll use the following diagram for the scenario. Each site has a LAN subnet; I am using /32 loopback address again for simplicity. Site 1's LAN IP is 5.5.5.5/32, site 2's is 6.6.6.6/32.

Again the ISP has to provision two lines for you - site 1 to the nearest ISP POP and site 2 to its nearest POP. Again on layer 1 the line can be copper or fiber Ethernet, T1, etc. and on Layer 2 the service can be Frame Relay, Ethernet, ATM, PPP, etc. Due to limitation of ISPs fiber coverage they have to provision a frame relay line to SITE-1. SITE-2 is in their fiber coverage and you get Ethernet there.

One of the main differences this time is that you are in control of your IP addressing. Second big difference is that your sites can see each other on layer 2, which means that the WAN interfaces on both sites can and should be in the same subnet. Since there are only two sites, this is point-to-point connection and to be efficient with the IP addressing you decide to assign 192.168.1.0/30 - .1 for SITE-1 and .2 for SITE-2. This is how you configure the interfaces on each site:

SITE 1:

Layer 2:

CE-SITE-1#show run | s Serial1/1
interface Serial1/1
 ip address 192.168.1.1 255.255.255.252
 encapsulation frame-relay
 serial restart-delay 0
 frame-relay map ip 192.168.1.2 100 broadcast
 frame-relay interface-dlci 100

The frame relay DLCI is given by the ISP. The frame relay map is required because the other side is Ethernet and won't respond to the inverse ARP request, nor this side will respond to the ARP request from the other side. So we have to create static mapping between the other side IP address and the local DLCI number. The broadcast keyword is required to run routing protocol between the sites.

Layer 3:

CE-SITE-1#show ip int br | e una
Interface                  IP-Address      OK? Method Status                Protocol
Serial1/1.100              192.168.1.1     YES manual up                    up
Loopback0                  5.5.5.5         YES manual up                    up

SITE 2:

CE-SITE-2#show ip int br | e una
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            192.168.1.2     YES manual up                    up
Loopback0                  6.6.6.6         YES manual up                    up

When the ISP provisions the whole service you should have connectivity between the sites:

CE-SITE-1#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 64/68/76 ms

If you ping the broadcast address the other side should reply and it does. This confirms that the interconnection between the sites is one broadcast domain:

CE-SITE-1#ping 255.255.255.255
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 255.255.255.255, timeout is 2 seconds:

Reply to request 0 from 192.168.1.2, 116 ms
Reply to request 1 from 192.168.1.2, 84 ms
Reply to request 2 from 192.168.1.2, 64 ms
Reply to request 3 from 192.168.1.2, 44 ms
Reply to request 4 from 192.168.1.2, 64 ms

What actually happens is that the ISP gets your frame and sends it across their MPLS network. On the other end if the Layer 2 protocol is the same, the frame is directly forwarded to the site. However, if the Layer 2 protocol is different it is first re-encapsulated to the respective protocol and then send to the site (that's simplified overview, in reality it's a bit more complex). In our case from site 1 you are sending frame relay frame and you are receiving Ethernet frame on site 2 and vice versa – site 2 sends Ethernet frame, site 1 receives frame-relay frame. No matter what layer 2 protocol is used on each site the layer 3 header is not touched at all. When CE-SITE-1 sends the IP packet with TTL 255, no matter how many routers will be crossed inside the ISP network, the TTL will be decreased to 254 by CE-SITE-2. That's because the ISP will carry and change only the layer 2 header but not the layer 3 header.

To have routing between your LANs on each site you have to run a protocol. With the L3VPN you could have different routing protocol on each site, with the L2VPN you must have the same routing protocol. That's because the sites will exchange hello messages and routing information directly between each other, as opposed to L3VPN the routing information is exchanged between the SITE CE and the ISP PE.

In this case, I will use OSPF:

SITE 1:

CE-SITE-1#show run | s router ospf
router ospf 1
 network 5.5.5.5 0.0.0.0 area 0
 network 192.168.1.0 0.0.0.3 area 0

SITE 2:

CE-SITE-2#show run | s router ospf
router ospf 1
 network 6.6.6.6 0.0.0.0 area 0
 network 192.168.1.0 0.0.0.3 area 0

At this point both sites should form OSPF neighborship. However, they don't. Because on site one we are using serial interface, it defaulted to NON_BROADCAST OSPF network type:

SITE 1:

CE-SITE-1#show ip ospf interface s1/1
Serial1/1 is up, line protocol is up
  Internet Address 192.168.1.1/30, Area 0, Attached via Network Statement
  Process ID 1, Router ID 5.5.5.5, Network Type NON_BROADCAST, Cost: 64

Changing the OSPF Network type to BROADCAST or POINT-TO-POINT fixes the problem.

CE-SITE-1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
CE-SITE-1(config)#interface Serial1/1
CE-SITE-1(config-if)#ip ospf network broadcast
CE-SITE-1(config-if)#end

Now we see the neighborship:

CE-SITE-1#show ip ospf neighbor

Neighbor ID     Pri   State           Dead Time   Address         Interface
6.6.6.6           1   FULL/DR         00:00:37    192.168.1.2     Serial1/1

When we look at the routing table we see only one OSPF route, the subnet between the sites appears as connected route.

SITE 1:

CE-SITE-1#show ip route | b Ga
Gateway of last resort is not set

      5.0.0.0/32 is subnetted, 1 subnets
C        5.5.5.5 is directly connected, Loopback0
      6.0.0.0/32 is subnetted, 1 subnets
O        6.6.6.6 [110/65] via 192.168.1.2, 00:02:25, Serial1/1
      192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.1.0/30 is directly connected, Serial1/1
L        192.168.1.1/32 is directly connected, Serial1/1

SITE 2:

CE-SITE-2#show ip route ospf | b Ga
Gateway of last resort is not set

      5.0.0.0/32 is subnetted, 1 subnets
O        5.5.5.5 [110/2] via 192.168.1.1, 00:02:47, FastEthernet0/0

When you run trace route between the LANs they appear as one hop away of each other. That's because of the TTL value is being decreased only by CE-SITE-2:

SITE 1:

CE-SITE-1#traceroute 6.6.6.6 source 5.5.5.5
Type escape sequence to abort.
Tracing the route to 6.6.6.6
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.1.2 36 msec 60 msec 48 msec

That's basically L2VPN or Metro Ethernet as WAN service. The example here, where you have point-to-point L2VPN between two sites, Cisco calls it AToM (Any Transport over MPLS). There is alternative that it doesn't require MPLS network and it can be run over normal routed network - L2TP (Layer 2 Tunneling Protocol). From the enterprise point of view there is no difference between the two - the ISP gets your frame from one site and delivers it to another without touching the layer 3 information.

Where is the VPLS you promised you would ask?

If we have more than two sites and we want all of them to have OSPF/EIGRP adjacency, then we have VPLS (Virtual Private LAN Service). The VPLS is point-to-multipoint connection where the ISP connects your sites in full mesh topology. Each site has neighborship adjacency with every other site. All router WAN interfaces are in the same subnet. In this example all routers are in 192.168.1.0/29. Every CE has 3 neighbors. In case of OSPF it is very important to have all WAN interfaces configured to OSPF Network Type BROADCAST or POINT-TO-MULTIPOINT, otherwise the routers won't form neighborship. Another very important point, if any site has frame relay line you should create frame-relay map for each neighbor. Unfortunately, I can't simulate VPLS because the IOS images I have don't support VPLS or only the control plane is supported (no data plane forwarding).

If we simplify the whole thing it will seem as if all the sites are connected to a switch. They appear as if all are in one broadcast domain. When running routing protocol, each site receives each other’s hello messages and they form direct adjacency.

BONUS: Here is EIGRP and eBGP between the two sites over the Metro Ethernet connection:

EIGRP:

CE-SITE-1#show run | s router eigrp
router eigrp 1
 network 5.5.5.5 0.0.0.0
 network 192.168.1.0 0.0.0.3

CE-SITE-2#show run | s router eigrp
router eigrp 1
 network 6.6.6.6 0.0.0.0
 network 192.168.1.0 0.0.0.3


CE-SITE-1#show ip eigrp neighbors
EIGRP-IPv4 Neighbors for AS(1)
H   Address                 Interface       Hold Uptime   SRTT   RTO  Q  Seq
                                            (sec)         (ms)       Cnt Num
0   192.168.1.2             Se1/1             10 00:00:55  107   642  0  3

CE-SITE-2#show ip eigrp neighbors
EIGRP-IPv4 Neighbors for AS(1)
H   Address                 Interface       Hold Uptime   SRTT   RTO  Q  Seq
                                            (sec)         (ms)       Cnt Num
0   192.168.1.1             Fa0/0            156 00:01:15 1290  5000  0  3


CE-SITE-1#show ip route | b Ga
Gateway of last resort is not set

      5.0.0.0/32 is subnetted, 1 subnets
C        5.5.5.5 is directly connected, Loopback0
      6.0.0.0/32 is subnetted, 1 subnets
D        6.6.6.6 [90/2297856] via 192.168.1.2, 00:01:47, Serial1/1
      192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.1.0/30 is directly connected, Serial1/1
L        192.168.1.1/32 is directly connected, Serial1/1

CE-SITE-2#show ip route eigrp | b Ga
Gateway of last resort is not set

      5.0.0.0/32 is subnetted, 1 subnets
D        5.5.5.5 [90/156160] via 192.168.1.1, 00:02:08, FastEthernet0/0

BGP:

CE-SITE-1#show run | s router bgp
router bgp 65001
 bgp log-neighbor-changes
 network 5.5.5.5 mask 255.255.255.255
 neighbor 192.168.1.2 remote-as 65002

CE-SITE-2#show run | s router bgp
router bgp 65002
 bgp log-neighbor-changes
 network 6.6.6.6 mask 255.255.255.255
 neighbor 192.168.1.1 remote-as 65001


CE-SITE-1#show ip bgp summary
BGP router identifier 5.5.5.5, local AS number 65001
BGP table version is 3, main routing table version 3
2 network entries using 272 bytes of memory
2 path entries using 112 bytes of memory
2/2 BGP path/bestpath attribute entries using 256 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 664 total bytes of memory
BGP activity 2/0 prefixes, 2/0 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.1.2     4        65002       7       7        3    0    0 00:01:47        1

CE-SITE-2#show ip bgp summary
BGP router identifier 6.6.6.6, local AS number 65002
BGP table version is 3, main routing table version 3
2 network entries using 272 bytes of memory
2 path entries using 112 bytes of memory
2/2 BGP path/bestpath attribute entries using 256 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 664 total bytes of memory
BGP activity 2/0 prefixes, 2/0 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.1.1     4        65001       7       7        3    0    0 00:02:16        1


CE-SITE-1#show ip route | b Ga
Gateway of last resort is not set

      5.0.0.0/32 is subnetted, 1 subnets
C        5.5.5.5 is directly connected, Loopback0
      6.0.0.0/32 is subnetted, 1 subnets
B        6.6.6.6 [20/0] via 192.168.1.2, 00:02:35
      192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.1.0/30 is directly connected, Serial1/1
L        192.168.1.1/32 is directly connected, Serial1/1

CE-SITE-2#show ip route bgp | b Ga
Gateway of last resort is not set

      5.0.0.0/32 is subnetted, 1 subnets
B        5.5.5.5 [20/0] via 192.168.1.1, 00:02:55
6 comments
89% Upvoted
This thread is archived
New comments cannot be posted and votes cannot be cast
level 1
CCNA RS/W, BCNP, BCvRE2 points · 2 years ago

Great write-up!

level 1

This is fantastic.. Thank you so much for doing all this!

level 1

My only question is this. You said you could use Metro Ethernet if you don't want to exchange routing information with the ISP.. But why? What is the benefit of MPLS vs MetroE, or vice versa? I'm just trying to understand from a practicality point of view.

level 2
1 point · 2 years ago

MetroE would be for layer 2 connectivity. But with things like OTV and EIGRP OTP it doesn't matter. Because both of those both solve the exchanging the routes/layer 2 issue. Where MetroE and MPLS differ greatly is that MetroE tends to be point to point and MPLS multi-point. MetroE tends to be cheaper since they don't have to connect you to every site and account for the bandwidth between them.

level 1
CCNA R&S1 point · 2 years ago

Wait...I thought no one ever used Frame Relay these days...so I promptly forgot everything I learned about it. What.

level 2
CCNA R&SOriginal Poster3 points · 2 years ago

I've personally seen frame-relay CE-to-PE-only connection on a customer site in Belgium and another customer in US. The service through the ISP was L3VPN/MPLS WAN, only the last mile was frame relay. This is generally where you could see frame relay these days and ATM in fact.

Community Details

35.9k

Subscribers

108

Online

Create Post
r/ccna Rules
1.
No posting of illegal materials
2.
No posting of braindumps
3.
Be courteous and helpful
4.
Don't ask others to complete your labs
Cookies help us deliver our Services. By using our Services or clicking I agree, you agree to our use of cookies. Learn More.