MPLS: Delivering Next-Generation Networking Services

Randy Zhang

Many business WAN services are currently provided by leased lines, frame relay, or ATM services. These services are popular because they provide a high level of quality guarantees and privacy, albeit at a high cost. For example, a frame relay-based network provides quality in terms of Committed Information Rate (CIR) and privacy in terms of permanent virtual circuits (PVC). Besides the high cost, these layer 2 technologies are commonly provisioned point-to-point, presenting a scalability problem. Success of the Web technology and the industry's convergence to a single protocol, IP, presents another alternative -- WAN services through the public IP network, the Internet. Because IP exists at layer 3 and is connectionless, it provides an any-to-any connectivity, with unmatched scalability by layer 2 technologies. However, IP services were not designed to be reliable and predictable. The majority of IP traffic is still delivered as a best-effort with little or no quality guarantees, yet such quality may become increasingly critical for many applications. Another increasingly common concern is data privacy and security, yet the current IP-level VPN solution involves building encryption tunnels. These tunnels, which are similar to WAN circuits, present a scalability problem.

There are two major forces pushing for new IP services: new applications and expanded infrastructure. IP applications have grown tremendously over the years; they now include voice, video, virtual private networks (VPN), extranet, and e-commerce. These applications demand better delivery service from IP. The expanded physical infrastructure is also ready to provide better services. High-speed access technologies such as Digital Subscriber Line (DSL) and cable are more available now, while the Internet backbone has been greatly upgraded.

To deliver better IP services at a scalable fashion, the IETF is moving to MPLS (Multi-Protocol Label Switching). MPLS is a switching technology that can be used by service providers and enterprises to provide layer 2 types of services at layer 3, or to provide connection-oriented services using connectionless infrastructure. Layer 2 in the Open System Interconnect (OSI) reference model refers to the data-link layer. An example of layer 2 WAN services is frame-relay PVC. Layer 3 in OSI is the network layer. IP is an example of layer 3 protocol.

To help you take advantage of these new services, this article will first explain the rationale behind MPLS, followed by a technical overview of how MPLS works. Two examples of MPLS services will be discussed in detail: VPN and traffic engineering (TE). A brief outlook for the future of MPLS and some industry resources are provided at the end of the article. Refer to the sidebar for abbreviations and acronyms.

Why MPLS?

Many Internet service providers (ISP) and enterprises construct their IP WAN networks using an overlay model (i.e., IP is running independently over circuit-based layer 2 technologies, such as ATM). As a connectionless protocol, IP has no concept of circuits in the connection-oriented world. As shown in Figure 1, the underlying WAN infrastructure is invisible to IP, forming a so-called WAN cloud. IP routers on the edge are connected to each other using virtual circuits. Many complicated schemes have been devised to map IP to the WAN topology, yet quality of services provided at layer 2 cannot be easily translated into layer 3.

There are two common designs: hub and spoke, and fully meshed. In the hub and spoke design, the hub router has circuits to all the spoke routers, but spokes do not have circuits among them (Figure 2). For one spoke router to talk to another spoke router, it must first go through the hub router. This design cuts down the number of circuits, and thus cost, but introduces an additional hop for spoke-to-spoke traffic and a single point of failure on the hub router.

The fully meshed design builds circuits between every router (Figure 3). This design provides any-to-any connectivity for IP but presents a scalability issue. As the number of routers increase, the number of circuits increase exponentially. For example, 6 circuits are needed to connect 4 routers, but 16 circuits are required to connect 2 more routers. Increasing the number of circuits also means more neighbors for routing protocols, which leads to network stability issues. The solution to these problems is to make WAN switches IP visible using MPLS, as discussed in the next section.

The forwarding technologies in the Internet are now destination-based (i.e., traffic is routed based on the destination address of the IP packets). This presents two problems. First, every router needs to do an address-based search to match the longest prefix, which is generally a slow process. Second, the IP address has to be unique on the entire network. Techniques such as network address translation (NAT) have to be used to translate illegal, private, or duplicate addresses. VPNs must be provided by building encryption tunnels using such as IPsec. Because these tunnels are point to point, a similar scalability issue also arises. Properly maintaining these tunnels can become a major operational headache. To avoid these problems, MPLS detaches the control function (route lookup, for example) from the forwarding function, as discussed later.

Although the IP version 4 header contains three bits for precedence, it is rarely used in forwarding. Additionally, circuit-based layer 2 quality of service (QoS) is difficult to map to layer 3. For most IP traffic, it is still best-effort. Currently, most routing protocols use a constant cost-based metric in calculating paths and are not aware of the congestion that might occur or has occurred on the links. Predefining a path through the network is in itself a complex task. Based on routing protocols that understand link constraints, MPLS is capable of providing class of service (CoS) and traffic engineering at the IP layer.

So what is MPLS? In short, MPLS is a layer 2 independent switching technology that uses layer 3 to provide layer 2 type of services. It is a switching technology based on label swapping. It is layer 2 independent because it can work on frames-based networks or cell-based networks. MPLS uses layer 3 functions such as IP routing to build the label forwarding table. It provides layer 2 type of services because the label switched path behaves like a virtual circuit through the network. The paths can be based on destination addresses, CoS, or any other requirements. The next section will go into detail on how MPLS operates.

Basic MPLS Operation

IP routing has two fundamental components: control and forwarding. The control component involves finding a path through the network. The forwarding component formulates data with the right headers and sends them out to an interface. Conventional routers do these processes on every hop on the way. MPLS takes the approach of separating the two components, thereby adding flexibility to the paradigm. MPLS still uses routing protocols such as Open Shortest Path First (OSPF) to build network reachability information, but forwarding is based on fixed-length labels and not destination IP addresses. How a forwarding component is implemented separates two types of MPLS devices.

The two types of devices are edge label switch router (LER or edge LSR) and (core) label switch router (LSR). LER encodes unlabeled IP packets with labels, whereas LSR swaps labels (i.e., LSR forwards labeled packets without consulting its routing table). A label is a numerical value that has local significance to the device. Its function is equivalent to that of VPI/VCI in ATM cells. An LER can insert a label into the header of a packet to convert a regular packet into a labeled packet. The label is often assigned based on a destination address and is exchanged dynamically among LSRs (discussed next). Thus, an LSR builds a label-forwarding table mapping an incoming label to an outgoing label.

When an LSR receives a labeled packet, it swaps the incoming label with the outgoing label and forwards the packet. An LSR operates like an ATM switch -- instead of switching cells based on VPI/VCI values, it switches labeled packets based on encoded label values. When the labeled packet reaches the other end of the MPLS network, another LER removes the label and forwards the packet as normal IP forwarding. The labeled path through the MPLS network is called the label switched path (LSP), a path that is connected through a series of labels. An LSP is like a virtual circuit in the ATM network, which is based on a series of VPI/VCI values.

MPLS constructs the LSP using an interior routing protocol (IGP) and the label distribution protocol (LDP). The IGP is used the same as on any other conventional routers to populate routes. Once the routing table is built, LSR uses a method such as downstream on-demand to build LSPs. The downstream is respective to routing updates. The LSR receiving routes will attempt to create labels for all those routes and then request the upstream LSRs to assign labels for the route using LDP. By such, a label forwarding table is created on each LSR. Note that LSPs are built before data traffic arrives. When a labeled packet arrives, an LSR looks up the outgoing label, puts it at the place of the incoming label, and forwards the packet, without looking up the routing table with packet's destination IP address.

Although MPLS is layer 2 independent, ATM-LSR deserves special attention because of the popularity of ATM switches in the service provider backbones. An ATM-LSR is an ATM switch that has added functionality of IP routing and MPLS. By converting a conventional ATM switch to an ATM-LSR, the previous ATM cloud becomes visible to IP. Instead of building many circuits end-to-end, now the edge routers only need to peer with one or more ATM-LSRs in the MPLS network (Figure 4). This architecture cuts down the circuits required and yet has any-to-any connectivity.

Examples of MPLS Services

VPN

A next-generation VPN service is now available from MPLS. A VPN by definition means that routing information is private over a public infrastructure, such as the Internet. WANs built on top of virtual circuits are by nature private. Address and content hiding using encryption over the public network is also VPN. MPLS VPN provides the same type of security as layer 2 circuits, but without the complexity of using encryption. Because IP addresses are not used to forward data on the MPLS network, the addresses can be made private.

In an MPLS-based VPN, LSRs are generally called Provider (P) routers and LERs are generally called Provider Edge (PE) routers (Figure 5). Only PE routers are VPN-aware, and P routers are generic LSRs. In contrast to layer 2 circuit-based networks, PE routers do not require meshed connections no matter how many VPNs are connected to them. Customer routers, including the customer edge (CE) routers, do not need to run anything special other than regular IP routing. Many PEs with various VPNs from many customers can share the same MPLS core network. Encryption tunnels are not required but can still be used to further protect the data. To distinguish VPN traffic, PE routers encode additional labels to the packets. P routers, however, only read and swap the top label. The remote PE router that understands the correct VPN pops the VPN label and delivers the packets to appropriate VPNs. To ensure route uniqueness across the MPLS backbone, a route distinguisher (RD) is prepended to every VPN route. To further increase the scalability, VPN routes are exchanged between PEs using Border Gateway Protocol (BGP) with multi-protocol extension, which aggregates VPN routes across the core.

MPLS brings a new type of VPN service. With the current IP VPN, point-to-point secure tunnels are built. With the number of VPN sites increasing, scalability becomes a problem. Additionally, enterprise customers generally have to manage these VPNs. With the MPLS-VPN, VPN service is provided by the service provider, and enterprise customers do not have to manage VPNs. Because the VPN is built on the architecture of MPLS, adding additional VPN sites does not create scalability problems.

The operation of MPLS-VPN is best explained with a simple example. Assume a small enterprise has two sites that need to be connected as one VPN, VPN1. We also assume the topology is as shown in Figure 5, with site 1 on the left and site 2 on the right. Both sites are subnetting the private address of 172.16.0.0, and OSPF is their routing protocol. The customer edge (CE) routers that directly connect to the PE routers are OSPF backbone routers. All customer routers, including the two CEs, do not need to run MPLS or VPN. All the VPN configurations are the responsibility of the provider, specifically on the PE routers. Routing updates from the remote CE router are passed by the MPLS-VPN backbone, and both CE routers become OSPF area border routers. Routes from the remote site show up as OSPF internal routes in the local site.

Packets generated from site 1 destined for site 2 are first forwarded from the CE router in site 1 to the connecting PE. The PE router identifies, by configuration, that traffic from the CE belongs to VPN1, so it associates all the routes from that CE with a preconfigured RD for that VPN. As indicated previously, MPLS labels and LSPs are already created before user traffic has arrived.

For VPN, there are two levels of labels: the top label directs the traffic through the MPLS backbone to the remote PE router, and the bottom label tells the remote PE how to handle the VPN traffic. So, the PE connecting to site 1 encodes two levels of labels to packets and forwards them to the next hop P router. The P router inspects the top label and forwards the labeled packet further along the way to the PE connecting to site 2. Note the P router has no knowledge of VPN and does not inspect the bottom label.

Once the labeled packets arrive at the last P router, it determines that the destination PE router is directly connected, so it pops the top label and forwards the packet that has the bottom label intact to the PE. The PE inspects the remaining VPN label and determines that it is associated with VPN1, by configuration, and removes the label and forwards the packet out to the CE. Between the two PEs, VPN traffic is never seen by devices other than the VPN. This level of security is equivalent to that provided by layer 2 virtual circuits.

Traffic Engineering

Current IGPs, such as OSPF, calculate network paths using an algorithm such as shorted path first (SPF) to find a least cost path, often with little or no consideration of the link load or congestion condition. Directing traffic to a specified path often involves manual configuration on every hop of the path. MPLS traffic engineering allows mapping of traffic flow to paths deviating from the IGP paths using a signaling protocol, such as the resource reservation protocol (RSVP). Traffic engineering allows a better load balance across the entire network, leading to better resource utilization.

For MPLS traffic engineering to work, several components are required. First, the IGP must be able to understand link resources and distribute such information. Link-state routing protocols such as OSPF and Intermediate System to Intermediate System (IS-IS) are intended to carry and advertise link utilization information. Second, a signaling protocol must exist to reserve needed resources and establish LSPs across the network. One such signaling protocol is the RSVP extended for MPLS. Third, there must be a way to define an explicit path or to use constraint-based routing. An explicit path is a pre-specified LSP that consists of a series of LSRs. LSPs for traffic engineering are often called traffic engineered tunnels or traffic trunks. Constraint-based routing is routing decision based on available resources. The constraint can include bandwidth and QoS requirements.

Generally, there are several steps in building MPLS traffic engineering tunnels. The IGP extended with traffic engineering (IGP-TE) floods the link states and a link-state database for traffic engineering (TE-LSDB) is created. An ingress LSR (tunnel headend) applies constraint to the database and produces a list of constrained shortest paths. The constraint can be created administratively or dynamically. A signaling protocol such as RSVP starts to exchange labels, and constraint-based LSPs are created from ingress LSRs to egress LSRs. Next, the ingress LSR assigns destinations to various LSPs based on predefined policies. Now the engineered LSPs are complete and traffic can flow. These tunnels remain operational, unless modified by administrators or link resources change. In the case of link failure, tunnels can be dynamically rerouted.

Looking to the Future

MPLS represents the future of Internet services. For ISPs, MPLS allows them to provide more value-added services at a more manageable level. For service users, MPLS can mean better services at a lower cost. In addition to the two MPLS services presented in this article, more services are being developed. Some of these services can replace current WAN services, such as leased line over MPLS, ATM over MPLS, and Frame Relay over MPLS.

Although some issues are still under discussion, the MPLS protocols are on track to soon become IETF standards. One salient effort is a proposal to put MPLS on wavelengths, appropriately named MPLambdaS. As the core of the Internet moves more to pure optical transport, it can be envisioned to be more efficient to have MPLS directly on top of optical cross-connects.

The progress of the IETF MPLS work group can be followed at:

http://cell.onecall.net/cell-relay/ \
  archives/mpls/mpls.index.html

About the Author

Randy Zhang, Ph.D. and CCIE, is a software engineer at Cisco Systems. He can be reached at randy.zhang@cisco.com.