X-Bone Multilayer Tunneling

X-Bone uses two-level tunnels. Each IP packet in an overlay is wrapped in two additional IP headers. The innermost (overlay) header is the endpoint in the overlay, i.e., it contains the overlay interface addresses. The next layer acts as a link layer in the overlay, and includes the source and destination addresses of the link interfaces. These link layer addresses are represented by a separate set of IP addresses, also internal to the overlay. The final IP header indicates the source and destination endpoints in the base network. Note that the base network can be another overlay, providing stacked or recursive overlays. The header order is shown in Figure 1.

Figure 1. X-Bone header format: double tunneling results in three layers of headers

The additional tunnels are required to allow multiple tunnels between two components, even within the same overlay. Such doubly-connected components are useful to emulate systems with larger numbers of components, i.e., 50-node rings simulated by using 10 router nodes. The additional layer also permits the use of multicast and dynamic routing algorithms inside the overlay, because such systems effectively operate on the link IP layer. Without that layer, it would be impossible to decouple intra-overlay routing from base-layer routing.

The two layers of the encapsulation change at every overlay hop, as shown in Figure 2. Hosts are indicated by their single overlay interface (A, D) and overlay link addresses (Q, T); the router is indicated by its pair of overlay interface (B and C) and overlay link addresses (R and S). Each component is shown as using a single, canonical base address for base-layer routing (X, Y, and Z); this can be relaxed for multihomed systems. X-Bone requires that routers are multihomed inside the overlay, according to the standard Internet practice

Figure 2. A single packet traverses the overlay – modifying both outer IP headers at each hop

In this figure, an application on overlay host A sends a packet to overlay host D. On the first hop (left), it is first wrapped with an IP header indicating its overlay link endpoint addresses QàR, and then wrapped with the base layer addresses of the endpoints of that tunnel, X and Y. On the second hop (right), both these outer encapsulation headers are removed and replaced with the overlay link and base layer tunnel endpoints of the next hop (S,T and Y, Z). IPSEC authentication or encryption occurs as a modification of the QàR or SàT overlay link headers. The X-Bone uses separate transport-mode IPSEC on these inner headers, i.e., first the IP in IP header is added, then IPSEC is performed to secure the packet Figure 3.

Figure 3. IPSEC’d X-Bone packet

X-Bone is currently implemented using separate IP address spaces both for the overlay endpoint addresses and the overlay link addresses. The use of separate address spaces effectively encodes the overlay identifier inside the IP addresses, allowing conventional dynamic routing and forwarding at the routers, and conventonal IP demultiplexing at the destination host. This can be relaxed to allow address reuse, provided the decapsulation steps in routers (for forwarding) and end hosts (for demultiplexing) keep sufficient context of the discarded layers of IP headers. Current overlay implementations discard this state, requiring global addresses. Note that overlay addresses can be reused among overlays that do not themselves overlap; this can easily be incorporated in the negotiation process. These issues are covered in more detail in the discussion of IPSEC.

X-Bone's dual-layer tunnels allow existing dynamic routing and network diagnostic tools to be used inside an overlay, transparent to the base network. This has been used to deploy dynamic routing across non-cooperating administrative domains, where only the hosts involved need participate in the routing algorithms. This has been demonstrated in the X-Bone system, and dynamic routing using RIP (via gated) and multicast (mrouted) are supported inside deployed overlays.

Figure 4. User views of a network mapping utility; different views in different windows