Graham, Bob, Jeff, and Ted July 14,1998 Valid for ASP 1.5 The ASP Active Extension Mechanism ---------------------------------- INTRODUCTION The Active Reservation Protocol (ARP) project is developing a facility for "active extension" of network signaling protocols, that is, for the dynamic composition of new or modified service features with an existing signaling service. The resulting mechanism is called Active Signaling Protocols (ASP). The ASP mechanism is designed to support multiple versions of each of multiple signaling protocols. We use the term "p/version" for a version of a particular signaling protocol with a particular set of service features. This terminology indicates that the same extension mechanism will support dynamic introduction of new protocols as well as the extensions to existing protocols. New p/versions might be defined and installed by network managers, or, ultimately, by any user of the signaling function. However, the latter will require solutions to hard protection and security problems. Some robust authorization mechanism will be required to control which users can use which versions. We add two important objectives for the ASP mechanism. First, the creation of new versions should be incremental, so that a version is composed of a "base version" and a set of "functional extensions". A new version is created by adding a new functional extension to an existing version. Second, the mechanism should allow sharing of common code among different versions. For each signaling activity, an appropriate version of the signaling software must be selected, and it must be installed on the node if it is not present. The granularity of a "signaling activity" depends upon the application; for RSVP, for example, the fundamental unit signaling activity is a session. Version selection might originate from an end user including explicit version information in each signaling packet. In principle, each signaling control message could be an active "capsule" containing the signaling algorithms to be applied. In practice, however, these algorithms are typically too large to be carried in individual signaling packets, which can only carry names and locations for fetching the version code out of band. An alternative selection approach is local control: the node could apply packet filters to signaling messages to implicitly map them to versions. For example, particular CIDR classes of source and/or destination addresses might be used for version selection. FUNCTIONAL EXTENSIONS The ASP facility will allow an initial or "base version" (BV) of the code that implements a particular signaling algorithm to be dynamically modified by one or more "functional extensions" (FEs). The base version BV is assumed to be self-contained and to implement a fully functional signaling protocol; it is the default when no FEs are in use. A version of signaling code for a particular signaling activity is defined by the base version BV plus the set of FEs in use. FEs are cumulative in effect, and in most cases, they must be applied in a pre-determined order. Thus, a version is described by the ordered sequence: {BV, FE1, FE2, ..., FEn}, where FE1, FE2, ... represent functional extensions. For explanatory purposes it is often useful to distinguish the base version from functional extensions; however, in many ways the BV plays the role of a default initial extension, i.e., of "FE0". Within the version shown above, the correct operation of a particular FEk requires that the "earlier" functional extensions FEj with j < k also be in place. We say that an FE is "a direct descendant of" the immediately preceding FE in the version sequence, and it is "descendant of" all FEs (and the BV) that precede it. The inverse relationship to "descendant of" is "ancestor of". The ASP scheme generally places a strong condition on compilation of a new FE: it may be necessary to have all relevant code of its ancestors available for the compilation to succeed. Starting from any version, multiple new versions can be created by adding different functional extensions; these new versions will be mutually exclusive. Hence, the "descendant of" relationship among functional extensions defines a "version tree" that is rooted at BV; see Figure 1 for an example. Here we have named the functional extensions with tree indices for clarity. A particular path from the root to a leaf of this extension tree specifies a sequence of FEs that defines a particular version. Thus, Figure 1 shows three alternative versions: {BV, FE.1, FE.1.1}, [BV, FE.2, FE.2.1}, and {BV, FE.2, FE.2.2}. Although versions and FEs are logically different, there is a one-to-one correspondence between leaf FEs and versions. BV --- | | / --- \ / \ --- --- FE.1 | | | | FE.2 --- / --- \ | / \ | / \ --- --- --- FE.1.1 | | | | | | --- --- --- FE.2.1 FE.2.2 Figure 1: Version Tree Example USING JAVA FOR FUNCTIONAL EXTENSIONS The ASP dynamic extension mechanism uses an object-oriented and platform-independent "byte code" for implementing signaling protocols. In particular, ASP uses Java byte code. The functional extension capability is constructed with the object-oriented mechanisms of Java. The BV and each FE consists of sets of Java classes, which we call "base version classes" and "extension classes", respectively. The Java class hierarchy forms a tree, the "class tree", among all the classes of the version. The root of this class tree is the Java "Object" class. The ancestor/descendant ordering of the version tree parallels that of the class tree. In particular, each extension class in a given FE must be an immediate sub-class of (1) a class in the same FE, (2) a class in an ancestor FE, or (3) a class in the BV. The inheritance mechanism in Java allows each such sub-class to extend its immediate super-class by overloading or adding methods and fields. These relationships are illustrated in Figure 2 below. This shows an example of a class hierarchy for version {BV, FE.1, FE.2}, formed of a base version and two functional extensions. Each node (circle) represents a class in the Java class hierarchy, and the lines denote the Java "extends" relationship. Thus, extension classes "d" and "e" of FE.1 respectively extend classes "b" and "c" of the BV. Appendix B shows an actual example of the class hierarchy comprising two active extensions to the specific signaling protocol RSVP. ................................. . Object . . -- O--- ----- . . / | \ \ . . a / b| c\ \ . . O O O | BV . ....|......|......|....|......... | | | | ....|......|......|....|......... . | d| e| | . . | O O | FE.1 . ....|......|...........|......... | | | ....|......|...........|......... . f| g| h| . . O O O FE.2 . ................................. Figure 2: Example of Java Extension Classes for Version {BV, FE.1, FE.2 } Since FEs may be added incrementally over time, each functional extension must be independent of any later descendant FEs, although it may depend upon any of its ancestors. Thus, the only names that may be assumed in an FE are those defined in the same FE or in an ancestor FE of the extension (including the BV). Methods in the BV must be written with some class names unbound to allow extension classes in FEs to extend the corresponding base classes. These unbound class names are dynamically bound at execution time in a manner consistent with the version, using a "dynamic extension" mechanism explained below. To instantiate one of these unbound base classes, the set of FEs in the version dynamically determines which base or extension class should be used. The class to be instantiated will always be a leaf in the class tree for the version. When it is instantiating the leaf class, the JVM will automatically load any classes above it in the class hierarchy that have not yet been loaded. In Figure 2, for example, the BV code to construct an object of class "a" must instead construct an object of class "f". Once the proper sub-class is constructed, a reference to it can be used in place of a reference to the super-class. We say that class "a" has been dynamically extended by class "f", and we write it as (a->f). Each new FE introduces an analogous mapping of one or more classes. For example, FE.1 in Figure 2 introduces the mappings {(b->d),(c->e)}, while FE.2 introduces {(a->f),(d->g)}. For a particular version, the overall mapping is the transitive closure of the individual dynamic extensions introduced by each FE in the version. For Figure 2, the version closure for dynamic extension is: {(a->f),(b->g),(c->e)}. Note that the mapping (c->e) in the version closure for FE.2 is necessary even though class "e" is not used in the leaf extension FE.2. For the version to execute correctly, class "e" must be used in place of class "c"; for example, "g" may access code in class "c" that in turn relies on code that was added or modified by class "e". A FE can introduce new classes that are not invoked by the dynamic extension mechanism, but use the normal Java static name definition. This allows the introduction of "helper classes", for example. Such statically-bound classes can be referenced only from within the same FE or a descendant FE. Class "h" in Figure 2 is an example. The programming rules for constructing a BV that can use the ASP dynamic extension mechanism are described in the next section. The choice of which classes in the BV may be dynamically extended, and which may not, must be *pre-planned*. In the future, we will consider avoiding the pre-planning by generating the dynamic extension machinery automatically by pre-processing the Java source or java byte code. Controlling the dynamic extension mechanism requires a small amount of additional state in the node; we call this state "version context". The exact nature of the version context will be discussed below. Finally, we note that common code is shared. For example, suppose that two versions {BV, FE.1} and {BV, FE.1, FE.2} of the same protocol are both loaded into a node. The code for the common classes (a, b, c, d, and e in Figure 2) will be shared by users of both versions. In summary: * Dynamic extensions in ASP are based upon two related trees: the version tree whose nodes are FEs, and the class tree whose nodes are Java classes. * A version is defined by a single path from root to leaf of the version tree, and different versions are mutually exclusive. * There is a distinct class tree for each version, and the leaves of the Java class tree are the points where extension is possible. * A new version is defined by adding an FE as a leaf in the version tree. This in turn addes additional leaves to the Java class tree. * For a given version, all the Java extension classes of all the FEs in the version must be loaded or loadable, for execution consistency. JAVA DYNAMIC EXTENSION MACHINERY This section describes the Java machinery for dynamic extensions in ASP. Suppose for simplicity that there is a base version, BV, and one functional extension, FE. Assume that the BV code constructs a new instances of each of the BV classes CC ... DD. Normally one would write, for example: CC cc = new CC(x, y, z); ... DD dd = new DD(p, q); Now suppose that the functional extension FE needs to dynamically extend CC, ... DD with extension classes CC1, ... DD1, respectively. We now illustrate two techniques to defer binding of the constructor until run time. TECHNIQUE A The Class API of Java can be used to construct classes by name. For example, one could write the following code to create set of references to "constructor objects": String name1 = "CC1"; // Dynamically mapped class name ... String namek = "DD1"; // Dynamically mapped class name // Load the classes using the ASP class loader. Class c1 = asp_class_loader.loadClass(name1, true); Class parm_type1[] = {A.class, B.class, C.class}; Constructor constCC = c1.getConstructor(parm_type1); ... Class ck = asp_class_loader.loadClass(namek, true); Class parm_typek[] = {P.class, Q.class}; Constructor constDD = ck.getConstructor(parm_typek); The names of these classes (e.g., "CC1", ... "DD1") are (pre-)computed from the version closure over for all FEs in the version. The ASP class loader, which extends the default class loader, knows how to load byte code over the network (using location specification machinery that is not shown). The resulting set {constCC, ... constDD} of references to constructors can be computed once, when the version is first referenced, and kept in the version context. The base version might then use the following code each time it needs to construct a (dynamically mapped) instance of CC, for example: A a; B b; C c; // Actual parameters Object parm_values[] = {a, b, c}; CC cc = (CC) constCC.newInstance(parm_values); // We now have a CC1 instance Technique A has two disadvantages: (1) it is awkward to explicitly create the formal parameter lists (signatures) and the actual parameter lists, as shown above; (2) it requires that a set of references be kept in the version context. Technique B overcomes both of these. TECHNIQUE B This approach introduces indirect constructor methods, which we call "constructor proxies", for each dynamically mapped class. All of these constructor proxies may be gathered into a single Java "Context" object, which is conceptually part of the version context state. Using the same example, the Context class would include: // This class is a local class class Context { // Define constructor proxy methods // CC newCC(A a, B b, C c) { return new CC(a,b,c); // base class ... DD newDD(P p, Q q) { return new DD(p,q); // base class } } Then this class can be extended in FE, overriding the constructor proxies: // The Context class is extended in FE and is // the extended class is potentially not available // locally. class Context1 extends Context { CC newCC(A a, B b, C c) { return new CC1(a,b,c); } ... DD newDD(P p, Q q) { return new DD1(p,q); } } Here Context1 implicitly defines the complete mapping closure for the FE version. Technique B ensures dynamic binding because the mapping from "Context" to "Context1" is known only at runtime. The following code might be used to load a new extension class: String name = "Context1"; // Dynamically mapped name // Load the class, using the ASP class loader Class c = asp_class_loader.loadClass(name, true); Object cntxt = (Context)c.newInstance(); The cntxt reference variable is then kept as part of the protocol's state. It must be accessible from every method that may need to instantiate a dynamically-extended class. Assuming that the correct value for cntxt has been retrieved for the current signaling activity, the following code is used to construct an instance of such a class: CC cc = cntxt.newCC(a,b,c); // We now have (e.g.,) a new CC1 instance ASP uses Technique B, which The "context" objects are realized by instances of (sub-classes of) the VersionContext class; we will refer to these as "version context objects". There will be an instance of a version context object for each distinct version in use. [Aside on forName versus loadClass: There are essentially two ways in Java to retrieve a Class object by its name. The first way using the method Class.forName() while the second method is using the method loadClass(, ) in the class loader. The loadClass method is more powerful because the caller can select a particular classloader to load the classes, possibly in a non-default manner. If the forName() method is used, the JVM will invoke a classloader to load the class if the class has not already been loaded. According to the Java language spec, the particular classloader to be invoked is the same classloader that loaded the class calling the particular forName() method. ] VERSION CONTEXT We can generally characterize the classes composing the BV as either "code-only" or "state-containing". State-containing classes are used to represent state blocks and other variable state, which are typically instantiated many times. Code-only classes, which basically contain only code (i.e., methods), would not require instantiation at all if method overloading were not required. However, as we will discuss later, extension is only possible for instance (i.e., non-static) methods, and therefore one instance of each code-only class must be instantiated. It is convenient to keep references to such code-only objects in the version context object. A "version context" object contains or implies the following version context state information: (1) The version name (character string) (2) The set of proxy constructors implicitly defining the closure mapping. (3) A set of locations from which all the class files may be loaded (4) References to code-only object instances When a class is to be loaded, the set of locations in (3) is searched. Our modelrequires that class files be uniquely named. The Java classes forming a version are loaded into memory on demand and are removed by garbage collection when they are no longer referenced. The lifetime of a "version" in the node is at least the lifetime of an RSVP session that uses that version. To reduce latency in loading versions, it may be useful to cache versions in memory or on disk. To cache it in memory, a version table would keep a reference to the VersionContext object, preventing its garbage collection. The class files comprising the "version" could also be saved in a disk cache. Global version control state is required to control this caching. CODING RULES FOR SIGNALING APPLICATIONS As we gain experience in writing extensions for Jrsvp and signaling protocols, we will be in a better position to give guidance to future developers. In the mean time, we have the following limited set of guidelines for Java developers: 1. Private methods or fields cannot be inherited by subclass. Therefore if for object oriented reasons, one uses private variables then one should provide public methods to access these variables. 2. The deferred binding required for polymorphism generally has an execution performance penalty. Normally a Java programmer can largely avoid this penalty by using the 'static' attribute, which allows binding at compile time. Unfortunately, runtime binding is essential for the ASP extension mechanism, so for maximum extensibility the programmer must avoid the 'static' attribute for all but the most primitive routines. 3. Overriding variables should in general be avoided because it makes the code difficult to read. Overriding variables actually introduces a new variable that must be maintained by methods in the class overriding the variable. 4. Programming would be easier if extension classes were given a different package than the BV. Having extension classes in a different package would generate more errors at compile time instead of at runtime. In more detail, if the BV and extension classes are in the same package, then no IllegalAccessErrors will be generated at compile time for methods (and classes) that have package scope. However, because local and remote classes are loaded with different class loaders they are treated as different packages by the JVM and consequently IllegalAccessErrors are thrown when extension code attempts to access non-public methods in the BV. IMPLEMENTATION Using a single classloader to demand-load class files: We use a *single* class loader to load all java classes regardless of location. The motivation for using a single classloader is that greater sharing (of memory) is achieved because all java classes are placed in a single name space. However, the single classloader introduces the problem that a mapping from class names to locations must be known, so that when a particular class is loaded it's location can be determined. In the case of the AppletClassLoader this problem is solved because the AppletClassLoader keeps a location object (actually an URL) as a private variable. When the AppletClassLoader is invoked at a later stage, is knows where to retrieve the class. [ The VersionSpec objects will typically be generated (because the name closures are encoded as proxy constructors. So we could generate a complete list of all the classes used by the FE and include it in the VersionSpec. The (class_name, search_path) mappings would then be made available to the classloader. The classloader might maintain two tables: a mapping of (class_name, extension_name) and a mapping of (extension_name, search_path). The separation of these tables would enable administrators to modify the an FE search_path independently of a "user" using a FE. Furthermore, one may wish to prevent users from specifying the location of their classes. This might be done by having a [local] configuration file that ensures that classes are loaded only from explicitly listed locations. Particular users would only be allowed to specify their own locations if they were explicitly listed in the configuration file. [No access would be the default, right?] This section describes an experimental implementation of the ASP mechanism for active extensions, which has been incorporated into Jrsvp. There is an instance of a version context class, i.e., of the VersionContext class or one of its sub-classes, for each active version. The version context classes are placed in a repository so that they can be loaded when required. A version context object itself is dynamically loaded on demand using the same mechanism that later loads extension classes. The version context object to be used for a given session is identified by a version selector called a VersionSpec, which is carried encoded within an RSVP Path message. The VersionSpec contains the information necessary to locate a version context object, items (1) and (3) listed above for version context objects, The VersionSpec is currently being encoded as a serialized Java object encapsulated as an RSVP object. Note that the extension context information must be injected by the Path message because path messages must arrive before Resv messages can be processed. Each RSVP session can select a version of the RSVP algorithms. Internally, each Session object will contain a reference to a version context object. Since the session is also the granularity of RSVP state, the important state blocks all contain references to the Session object. Thus, the extension context can be easily located from any method. The following steps must be taken to construct a new extension, i.e., to define a new version and initialize its state within the Jrsvp server: 1. The new version must be implemented by a Java class that extends the VersionContext class or appropriate sub-class. 2. All new class files that are required by the version must be placed in a location such that they can be retrieved by Jrsvp when they are demand loaded. The following options are currently available: an FTP server [but there is bug in Sun code], an HTTP server, or any Jrsvp-capable node. 3. A VersionSpec object, which contains the name of the new extension and the list of locations to search, must be included in every RSVP PATH message. The VersionSpec object might be initialized and then attached to the PktMap as follows: PktMap mapp = new PktMap(...); Vector v = new Vector(); v.addElement(new LocationFile(); v.addElement(new LocationHop(); v.addElement(new LocationFTP(); VersionSpec vcspec = new VersionSpec("rsvp.VersionContext1",v.elements()); mapp.map_vcontext_spec = vcspec; The PktMap is then sent to the RSVP next hop. The presence of a non-null VersionSpec in the PktMap informs the receiver that a new extension is requested. The name and path for the new extension is obtained from the VersionSpec object. How are are the contents of "version context" encoded by the VersionContext object? (1) Currently, the version name is encoded as the name of the class that extends the VersionContext class. An alternative approach might be add an additional level of indirection so that extension names might be independent of particular class file implementations. [ Where would this mapping reside? DNS server? local config file? ] (2) The class name mapping table is represented as a set of proxy constructors, as described above. (3) The references to code-only objects are encoded as references to base classes that can be overloaded, eg. RsvpPath, RsvpResv. Once the new extension has been sent in a RSVP Path message to a Jrsvp node, Jrsvp is responsible for loading the relevant classes over the network. We have overloaded the default class-loader in Java by extending the ClassLoader class with a new class loader, called 'AspClassLoader'. The AspClassLoader implements the method 'loadClass' to retrieve classes from a path of locations. The AspClassLoader essentially enumerates through the Location objects and calls the method findByteCode() to actually retrieve the class file. The findByteCode method has the following signature: byte[] findByteCode(String classname) throws ClassNotFoundException; The advantage of this relationship is that Location objects can be sub-classed (by an extension implementor to define new locations and compression/decompression algorithms for transferring class files across the network (without modifying the AspClassLoader). The active extension mechanism uses the following methods to construct an instance of the appropriate version context class: // first construct the new loader AspClassLoader loader = new AspClassLoader(); // set the path where the loader must search, for example, loader.setPath(); // then create an instance of the new extension VersionContext vc = (VersionContext)loader.loadClass( , true).newInstance(); OPEN ISSUES 1. Signatures on class files to prevent spoofing. 2. Config file for Locations. 3. Bootstrapping problem when no PATH message has arrived at a Jrsvp node. ------------------------------------------------------------------- Appendix A: In this appendix, we show our implementation of the loadClass() method in our classloader, called 'AspClassLoader'. The loadClass() method is called both explicitly by the BV on receiving an RSVP PATH message that has a non-null VersionSpec object and by the JVM when it attempts to load a particular class that is referenced by a class already loaded by the AspClassLoader. synchronized public Class loadClass(String name, boolean resolve) throws ClassNotFoundException { Class c = findLoadedClass(name); if (c == null) { c = (Class)cache.get(name); } if (c == null) { try { return findSystemClass(name); } catch (ClassNotFoundException e) {} c = findClassLocally(name); if (c == null && path != null) { Location loc; Enumeration n = path.elements(); while (n.hasMoreElements() && c==null) { loc = (Location)n.nextElement(); try { byte[] buf = loc.findByteCode(name); c = defineClass(null, buf, 0, buf.length); AspServer.save(name, buf, 0, buf.length); } catch (IOException e) {} } catch (ClassNotFoundException e) {} } } if (c == null) throw new ClassNotFoundException(); cache.put(c.getName(), c); } if (resolve) resolveClass(c); return c; } ------------------------------------------------------------------- APPENDIX B: Example: Adding SCOPE Object to RSVP [This section needs rewriting] Suppose for example that we want to add the SCOPE mechanism to RSVP. (The current Jrsvp does not have SCOPE implemented). The SCOPE object in included in wildcard filter (WF) Resv and ResvErr messages, to limit the forwarding of such messages in order to avoid loops. A SCOPE object contains a list of sender IP addresses. Adding SCOPE will require addition of a new Scope class, as well as updating the following classes and methods: RsvpPacket.map2byte(), RsvpPacket.byte2map() Send/receive SCOPE objects RSB Define rs_scope field. PktMap Define map_scope field RsvpPacket.byte2map(): Scan SCOPE field and create new PktMap RsvpPacket.map2byte(): Send SCOPE field RsvpResv.* To use scopes in generating Resv and ResvErr messages to be forwarded. To receive and process SCOPE objects RsvpPath.acceptPATH(): clear scope union Get new RsvpPsb RsvpPsb: Add InScope boolean ("scope union") As can be seen, this particular extension has implications across much of Jrsvp. Suppose that we develop a second functional extension, to create a version that implements the CONFIRM message and object as well as the SCOPE object. This results in the following class hierarchy for the functional extensions. ========================================================================== BASE VERSION Object | +-------+-------+--------+------+-----+-----+--... (and many | | | | | | | more that RsvpPacket | RsvpPath | RSB | Style are not | PktMap | RsvpResv | MRS | extended) | | | | | | | | | | | | | Style_WF | | | | | | | ========================================================================= "ScopeExt | | | | | | | FE | | | | | | | RsvpPacket_1 | RsvpPath_1 | RSB_1 | Style_WF_1 PktMap_1 RsvpResv_1 | | | | | | MRS_1 | | | | | =============================================================== "ConfirmExt"| | | | | FE | | | | | RsvpPacket_2 | | RSB_2 | PktMap_2 RsvpResv_2 MRS_2 =============================================================== Class name suffix "_1" => "ScopeExt" Class name suffix "_2" => "ConfirmExt"