Press J to jump to the feed. Press question mark to learn the rest of the keyboard shortcuts
Posted byCCNP R&S3 months ago

Do large carriers just treat the internet an MPLS L3VPN (putting it into a VRF)? If not, why?

67% Upvoted
What are your thoughts? Log in or Sign uplog insign up
level 1

Generally internet goes into the global routing table and is used for VPNv4 and VPNv6 peering, as well as label distribution.

level 2
CCNP R&SOriginal Poster1 point · 3 months ago

So this means you have to run BGP everywhere then, yes? Isn't that quite a downside?

level 3

I don't know any ISP that doesn't use BGP. BGP runs the internet. Why would that be a downside? MPLS is basically BGP and label switching.

level 4
CCNP | ISP Operations4 points · 3 months ago

We use IS-IS in most of our core. But mostly BGP everywhere else.

level 4
CCNP R&SOriginal Poster4 points · 3 months ago

The key phrase here isn't run BGP as much as it is run BGP everywhere. If you put the internet routing table in a VRF you could have a BGP-free core and only run it on the edge routers.

level 5

You can still have BGP free core using OSPF or IS-IS in the global table. BGP on the PEs to exchange VRF routes and IGP to distribute labels and global routes for BGP peering.

level 6
CCNP R&SOriginal Poster2 points · 3 months ago

That makes sense I suppose. Is it a very common setup or do carriers still tend to run BGP on their backbone as well?

level 7

BGP-free core is extremely common. BGP is very slow to converge, so ECMP with an IGP and BFD make for quick failover while maintaining BGP peering across the network.

level 8
CCNP R&SOriginal Poster2 points · 3 months ago

I see. Thank you.

level 7

Its not common to run on the backbone. Usually, you'll run it on your customer agg and edge routers (your LERs). You need to make sure that the packets know where to go when they leave the LSP at the penultimate hop.

You want to use MPLS LSPs because you want to be able to traffic engineer in the core. BGP becomes redundant, if you have an IGP there.

level 7
3 points · 3 months ago

I think most carriers use IS-IS.

level 8
CCNA, MEF-CECP, OCSA2 points · 3 months ago

This is correct

level 7

IS-IS for the backbone.

level 8
CCNA, MEF-CECP, OCSA2 points · 3 months ago


level 3

MPLS can provide transit for traffic in the global routing table too. So effectively most carriers do what you say, just the “global” table is used not a specific VRF.

On some hardware platforms there are things you can do in the global table you can’t do in a VRF, which is the reason why.

level 1

No. Carriers have a public internet and private internet. If you have an MPLS circuit the VRF starts at the provider edge and goes through the carrier's private network back to their other edge and then to your other location where the VRF stops. Loosely speaking, the VRF contains BGP specific to you in this case. MPLS is the Carrier's "routing" protocol on their private network.

If you don't have an MPLS connection, then the traffic just stays on the public internet, all BGP.

Does that help?

level 2
CCNP R&SOriginal Poster3 points · 3 months ago

I mean... I don't know. Are you saying you believe carriers put MPLS on a completely separate network than their internet infrastructure? Because that doesn't sound correct to me.

level 3
CCNP | ISP Operations3 points · 3 months ago

We basically do exactly that.

level 4
CCNP R&SOriginal Poster1 point · 3 months ago

Large carrier?

level 5
CCNP | ISP Operations2 points · 3 months ago

Top five globally.

level 3

If you have the money you can do this for sure.

The upside is that any DDoS and other evilness that exists on the internet is physically segmented away from the enterprise/office networks being used by your WAN-service.

Another upside is if/when for example an internet related router goes poff this online affects your internet traffic and not your WAN-service (and the other way around, if something goes poff with your WAN-service either its a hardware malfunction, cablecut or just bad config then your internet-service is most likely unaffected (except for the cablecut scenario).

A possible upside can also be if you got physically separated networks you can then also upgrade the gear for your WAN-service without affecting the stuff used for internet. Today its not uncommon that other services than just pushing ip-packets between sites can be provided - for example VPLS that can push ethernet-frames. Or other specific requests from the customers such as being able to push jumboframes (9k) site to site where with internet you can just say "nope, its 1500 bytes MTU and thats it".

The downside is not only double the cost for gear, darkfiber and wavelengths (and power, cooling and space renting at the site) between sites but also double the amount of management/maintenance since you now have twice the amount of gear to take care of.

level 3

No. But also, kind of, yeah. It's a mix of the two. For MPLS, the customers connect directly to the MPLS routers. For public internet, the customers connect to edge routers, which then connect to MPLS routers. The MPLS connections are then routed back to the private side of the core, and the public connections are routed to the public side of the core.

So the access layer is often separate, then sometimes meet at a shared aggregation layer, then to the appropriate public/private core routers.

level 1

It depends on many factors, some technical, some business reasons. Some SPs float internet on L3VPN, some carriers use inet.0 (global routing table).

In our case, we operate two completely separate layers of networks. IP transit (single AS) division runs internet on global routing table, but our regional area networks (different ASNs) float internet on L3VPN.

Bottom line, there is no technical justification on which method is superior. You can just as easily protect your core from DDoS and outside attacks while running internet on inet.0 instead of on L3VPN (infrastructure ACLs, proper numbering schemes, etc). It really comes down to the service models you're offering and business case.

level 1

You doesnt have to but it has become some sort of "best practice" on how to run an ISP.

The downside is the complexity and the cost (the gear must support MPLS) but the upside is that you can separate internet (or whatever vrf) from any other networks you maintain - if not to at least separate MGMT from PROD where PROD in this case is INTERNET.

And once you have MPLS up and running its a small task to also start to sell WAN-services and not just internet-connections to your customers.

Once you go the WAN-path you will for sure end up with customers running the same ipranges (if not so at least RFC1918) and here is vrf very handy to be able to deal with this.

The other option would be that you always tunnel the customers traffic end to end (think of this as a traditional vpn), this way whatever the customer choose to do or run wont affect your core or distribution network because the customer traffic is encapsulated. I mean the customer can still choose to run vpn on their own to protect the traffic from eavesdropping but then wont have to do it if they just want to move packets from site A to B (and doesnt care about eavesdropping along the road).

level 1

I would imagine that the reason any carrier chooses to do this is because the lack of supernets for routing purposes. Currently, we don't have that problem, yet. But I don't think it will be long before there are serious issues with anyone not natively supporting V6 on their buildouts. We run a 100% dual-stack so not really an issue for V6, but IPv4, won't last forever.

level 1
DRINK-IE and LINKSYS-IE1 point · 3 months ago · edited 3 months ago

Because you generally can't resolve as much in VRF as you can in the master routing table.

level 1

It scales better to run your internet in a VRF instance, rather than running it in your GRT

level 1
Probably drunk CCIE-2 points · 3 months ago

Erythang is MPLS(or SR). Internet is just another VRF. The global table only contains the /32 loopbacks of the PE's

Community Details





###Enterprise Networking Routers, switches and firewalls. Network blogs, news and network management articles. Cisco, Juniper, Brocade and more all welcome.

Create Post
r/networking Rules
Rule #1: No Home Networking.
Rule #2: No Certification Brain Dumps / Cheating.
Rule #3: No BlogSpam / Traffic re-direction.
Rule #4: No Low Quality Posts.
Rule #5: No Early Career Advice.
Rule #6: Educational Questions must show effort.
Cookies help us deliver our Services. By using our Services or clicking I agree, you agree to our use of cookies. Learn More.