IPQoS

Peter Baer Galvin

IP Quality of Service (IPQoS) is a part of the Solaris Resource Manager, and is included in Solaris 9. It purports to allow management of network traffic, to the point of dictating how much network bandwidth each application can use. This month, the Solaris Companion explores the theory and fact of IPQoS.

Theory

Sun's old Bandwidth Manager has been replaced in Solaris 9 9/02 with the IPQoS facility. This facility is compliant and interoperable with IETF Differentiated Services standards and specifications. (For more details see RFC 2474 and RFC 2475 at http://ietf.org.) It is also integrated with the other Solaris Resource Manager features (as described in the April and June 2003 Solaris Companion). But what exactly is IPQoS?

According to Sun's documentation, "IP Quality of Service (IPQoS) enables system administrators to provide different levels of network service to customers and to critical applications. By using IPQoS, the administrator can set up service-level agreements. These agreements provide an ISP's clients with varying levels of service that are based on a price structure. A company could also use IPQoS to prioritize among applications so that critical applications get a higher quality of service than less critical ones."

IPQoS is important to Sun's future. It will be a key component of data center virtualization, as systems take on multiple roles, and as automated deployment replaces hand-tuned architectures and implementations. It can also be helpful in capacity planning and chargeback via its integration with accounting. Furthermore, IPQoS is the networking core of the forthcoming "containers" technology found in Solaris Next (see the "Solaris Containers -- How Advances in Server Virtualization Will Simplify Service Manageability" whitepaper at: http://wwws.sun.com/software/whitepapers/wp-svccont/wp-svccont.pdf).

IPQoS is Sun's implementation of the IETF Differentiate Services (DiffServ) standard. It works by redefining the "type of service field" in the IP header. The DS codepoint (DSCP) field now indicates the class (or "aggregate") of the packet. Other DiffServ-aware networking devices and machines can then honor this information and give the packet the appropriate treatment.

Of course, this is a standard, and with standards come lots of detail and new jargon. Suffice it to say that there are various activities that can be performed based on this field, including measuring class performance and data flow, metering and dropping packets, and shaping packet flow using queuing and scheduling to conform to the priority of the class. Certain values of the DSCP have predefined meanings according to the RFPs, including four classes of service and low-, medium-, and high-drop precedence within the class. The "expedited forwarding" value is for high-priority, low-latency communications. The sending system can put a DSCP value in place to inform the rest of the network of the relative importance of the packet. This value can be modified by routers or nodes along the path as they see fit.

The net result is that if a link has too many packets to send on their way, it can make intelligent choices of what to send when, and what to drop. Videoconferencing may proceed at full speed, for instance, while an FTP session hesitates or stalls. Also, rate limiting can be used even when there is no congestion to ensure that a traffic flow is limited according to some external requirement (such as a service level agreement at an ASP).

Note that, as with many standards, this one has competing and conflicting standards to contend with. The Layer 2 802.1D Quality of Service standard for Ethernet has its own thoughts about traffic priorities, and it is undefined how those interoperate with DiffServ.

The Implementation

IPQoS is a full kernel component, included for free with Solaris 9 9/02. It integrates with kstat for monitoring, and with accounting for reporting. The ipqosconf command configures IPQoS on Solaris. It chains together modules that classify the packet and then acts on that classification to put a packet into a specified class.

Classification can be based on one or more of the following:

• Source/destination IP address and mask
• Source/destination port number
• Protocol
• IP version (IPv4, IPv6)
• Direction (local in, local out, forward in, forward out)
• Interface name, interface group name
• TOS/DSCP value
• User ID (uid); local outgoing only
• Project ID (projid); from the resource manager -- local outgoing only

There are several kstat options to report on current IPQoS activities, per class. These include kstat -m ipgpc, kstat -m tokenmt, and kstat -m dscpmk. The accounting details can be obtained via acctadm.

But what does this all add up to? How useful is the facility, and how can it be used? On a fully implemented IPQoS network, it could work quite well. For example, a Web services provider could dole out the services based on the priority of the client. IPQoS could also save network costs by reducing the amount of bandwidth that a given site needs to buy. If multiple functions are traversing the network, but most are low priority, then a more "narrow" pipe could be used if the low-priority traffic could be made to behave itself and allow the high-priority traffic to use as much as it needs.

Many sites implement multiple network connections. Sites performing replication for disaster recovery frequently have dedicated pipes for that traffic, while other traffic between the same sites courses over other paths. If the traffic could be managed, fewer lines could be used, or at least more efficient use could be made of these lines.

For all of this to work properly, both the sending and receiving host must have IPQoS implemented and enabled. Furthermore, the network devices between the hosts may also implement DiffServ. This is a problem over public networks (like the Internet), but not over a company's WAN, or a leased-line WAN between cooperating companies.

Without the support of the interposing network, there is still some function to IPQoS. It provides the classification and metering functions, but not the queuing functions. That is the packet can be classified, and the metering can monitor the amount of traffic in that class. It can be configured to limit the traffic based on that metering. Unfortunately, the only way it can implement that limit is to drop packets. UDP communications would likely be unhappy about having packets dropped. TCP, of course, expects some packet drop, and can recover from it.

Another use for IPQoS on Solaris is in the accounting and performance monitoring area. By classifying the traffic, the system is able to monitor network traffic according to the classification. Much more detailed information about traffic over time (accounting) or current network traffic (kstat) can be obtained.

Conclusions

If IPQoS is implemented on Solaris as part of a fully DiffServ network implementation, it can add a lot of functionality to the network. It can be the core of resource management, including shaping traffic, guaranteeing priority, and dropping packets that are exceeding the configured limits. If it's implemented strictly at the host level, the function is much more limited. It can classify traffic for accounting and kstat reporting, and it can drop packets if the metering determines they are over quota.

For more reading about IPQoS, see the Sun whitepaper at:

http://wwws.sun.com/software/whitepapers/solaris9/ipqoswp.pdf

http://docs.sun.com/db/doc/816-4094?q=ipqos

Sun is doing a very good job, these days, of keeping systems managers informed of the changes in each Solaris release. In case you didn't know, at http://docs.sun.com, you can select the operating system release you are interested in (i.e., Solaris 9 8/03). Then, select the "About What's New" subtree, and the "What's New in the..." document. There are quite a few details in the document about the new features of that particular release and all the releases since the major release. This is the best place to start in determining whether a release is right for your site, and how to test that release if you are thinking of implementing it.

For a quick overview of each release and what it includes, have a look at:

http://wwws.sun.com/software/solaris/fcc/index.html#solaris9

Peter Baer Galvin (http://www.petergalvin.info) is the Chief Technologist for Corporate Technologies (www.cptech.com), a premier systems integrator and VAR. Before that, Peter was the systems manager for Brown University's Computer Science Department. He has written articles for Byte and other magazines, and previously wrote Pete's Wicked World, the security column, and Pete's Super Systems, the systems management column for Unix Insider (http://www.unixinsider.com). Peter is coauthor of the Operating Systems Concepts and Applied Operating Systems Concepts textbooks. As a consultant and trainer, Peter has taught tutorials and given talks on security and systems administration worldwide.