XDR (RPC Marshalling) Overhead

The Problem

The process of marshaling data so that it can be exchanged between a server and a client with different hardware architectures is often implemented inefficiently. In a 1991 USENIX paper Richard Macklem reported significant improvements in NFS performance from replacing XDR data marshaling routines with macros. While we are primarily interested in the new methods of improving NFS throughput, we also attempted to bound the effect of replacing XDR routines with macros.

The Experiment

We compare user level programs that model NFS write exchanges, but with different data encapsulation mechanisms. One version encapsulates the data via XDR routines, the other by simpler network byte ordering routines. The simpler encapsulation is more efficient due to avoiding the overhead of the function calls to the XDR routines and extra data copies therein, as predicted by Macklem. The functionality of XDR is provided by both. The user level Sun RPC routines do not marshal their data to mbufs as the kernel routines do, which is one reason their throughput is higher than NFS protocol throughput.

Results

The graph above shows the result of an experiment in which the equivalent of a 64 MB file is transferred. Values are reported for multiprocess pipelined versions that mimic NFS biod processes. We compare process pipelining to process pipelining rather than using a single threaded implementation because we want to isolate XDR overhead. The lightweight encapsulation is capable of matching TCP data transfer speeds. In fact, the effect of multiprocess transfer makes the aggregate throughput of the 8 processes higher than that of a single process using TCP.

This experiment suggests that careful implementation of the data translation routines could boost throughput considerably. Although we did not rewrite the kernel as Macklem did, we have shown that user level processes can be optimized to deliver considerably better raw throughput than XDR provides without loss of functionality. The experiment is encouraging, but idealistic. The user-level processes need not interact with other kernel systems; implementations in the kernel may not show the same improvement.

Go back to the ATOMIC-2 home page. / Go back to the ISI home page.

This page written and maintained by the ATOMIC-2 group.
Please mail us any problems with or comments about this page.
Last modified July 9, 1997