Many programmers working with networking expect a significant measure of support from the network protocols used on their host systems to provide general & reliable service. Unfortunately, the expectation of full generality can get in the way of efficient use of resources in many distributed applications. Much of this waste has been traced to using a ``full'' protocol stack for all network traffic, even if not all is necessary for each transmission.
For distributed computing to be effective, the overhead of each call needs to be reduced as much as possible. When a client process initiates a bind operation with an RPC interface, a network address for the server is part of the binding resolution. In most cases, this address is on the network local to the client. This is a situation which may offer significant advantages where the networking layers of the host can offer dynamic configurability, allowing elements of the protocol stack to be added and removed to create the most efficient stack with the required transport qualities for the RPC protocol.
The x-kernel is an operating system kernel implemented on Sun-3 workstations. It supports the array of facilities common on UNIX-like systems, and adds support for constructing new network protocols and composing fundemental protocol layers in an object-oriented manner. Passing control from one layer of the protocol stack to another is less expensive than in traditional UNIX kernels, requiring only a procedure call overhead rather than a context switch into another process. The binding sequence of a protocol stack can be deferred until a call from a high-level protocol to a low-level protocol is executed.
The research discribed in the reference implemented the Sprite RPC protocol for the x-kernel. The Sprite Operating System is a distributed system running on Sun 3/75 workstations. It was developed by Dr. John Ousterhout at U.C. Berkeley. Brent Welch developed the Sprite RPC protocol, also at U.C. Berkeley.
The research team used two techniques to improve the performance of of the RPC protocol. They called these virtual protocols and layered protocols.
The easiest to understand is the virtual protocol, as it reflects design patterns often found in object-oriented programming. As discussed above, the network protocol layer in a traditional system maintains full generality for all transmissions, but most RPC services require only local transmissions most of the time. This implies that RPC transmissions typically do not require the services provided by the IP layer of the protocol stack. This is the layer that handles node addressing across network boundaries, and is clearly not needed when no network boundaries are involved. A common protocol stack for remote procedure calls consists of RPC/UDP/IP/ETH, where ETH is the ethernet layer that communicates directly with the hardware. The most significant software overhead in this stack occurs at the RPC and IP layers. Removing the IP layer when the client and server are on the same ethernet wire can achieve significant performance improvements.
The researchers created a new protocol which they called the VIP protocol, for Virtual IP. This protocol checks the connection when it is opened and, if the two endpoints are not on the same physical network (if they really require the IP layer), passes messages on to the IP protocol. When they destination is on the same network as the source, the IP protocol is bypassed and the message goes directly to the ETH layer. Since the check for needing the IP protocol can be made when the connection is opened and is invariate, the VIP layer can decide whether to use the IP protocol for, typically, one instruction execution per message. When IP is needed, the extra cost of the single instruction is immaterial next to the overhead of the IP protocol and the real time transmission lag. For those transmissions only over the local network, replacing an entire protocol with a single comparison provides a major improvement, especially since we also reduce the size of the data which must be transmitted.
While virtual protocols allow individual elements of the protocol stack to be skipped over if not needed, they simplify the compostion of a protocol stack only in one rudimentary way, but do not improve each protocols usability through abstraction. The use of layered protocols improves the useful scope of each protocol element.
Protocol layering is accomplished by decomposing high level protocols like HTTP or RPC into several distinct protocols, each of which implements only one function of the high level protocol being replaced. Several protocols handle some service aspects as well as their primary function: RPC protocols often include methods for handling and reconstructing large packets. Other protocols, including UDP, include this function as well. Breaking functional aspects into separate protocol layers allows better encapsulation of each capability, improved optimization, and improved compositional qualities. Since each new protocol handles exactly one function, these layers may be more effectively composed into fully functional protocol stacks. These stacks can be dynamically defined as appropriate for each process. Handling large-packet decomposition and recreation as a separate protocol allows less code to be used for that function, with better attention to reliability and performance factors. This also allows each layer in the protocol stack without concern for the effects of other layers, simplifying the needed computation. Removing unnecessay layers from the stack prevents wasted CPU time, allowing much better throughput in the system.
Go Back to the Operating Systems page.
Go Back to the RPC page.