John Heidemann

Stackable Design of File Systems

TitleStackable Design of File Systems
Publication TypeTechnical Report
Year of Publication1995
AuthorsJ. S. Heidemann
Pagesxvi + 105
Date Publishedsep

This dissertation presents the design, implementation and evaluation of file-system design with \emphstackable layers. Stackable layering addresses two significant problems in file-system development. First, existing services are difficult both to extend incrementally and to re-use in new work. Stacking addresses this problem by constructing sophisticated new services as a \emphstack of new and existing layers. Layers work together since each layer is bounded above and below by the same (\emphsymmetric) interface; layer configuration is limited only by semantic constraints. Layers can be independently developed and distributed as binary-only modules to protect the investment in their development. Incremental improvements to existing services can be provided through new, thin layers. Second, evolution of filing interfaces presents a problem to development and maintenance of services. In some respects, evolution is often too fast, as when vendor changes to interfaces invalidate existing third-party layers, greatly adding to their development and maintenance costs. At the same time, evolution is too limited and slow, as when developers and especially third parties cannot provide new services because of the constraints of old, centrally-managed interfaces. We address these problems by providing an \emphextensible layering interface which supports managed interface evolution by both vendors and third parties. When interface changes are too large or are incompatible with existing practice, a \emphcompatibility layer can smooth over the changes. With an extensible interface, a layer may be confronted by an operation it does not understand. A standard mechanism allows layers to handle these operations by sending the operation to a lower layer for processing. Stacking enables and simplifies several design techniques. A transport layer may move operations between machines and allows user-level layer development. Our stacking solution also includes a cache-coherence protocol to synchronize state across stack layers and a lightweight layering protocol allowing the benefits of independent development to extend even to very ``thin'' layers. We have constructed several layers using our stacking facilities. This dissertation describes both the implementation of these services and their measurement and evaluation. We examine the performance of the stacking framework, cache-coherence protocols, and lightweight layers, concluding that stacking often adds little or no cost to user-observed performance and minimal additional kernel overhead. Finally, our experiences using stacking to develop and deploy several layers suggest that new services can be provided significantly easier with stacking than with traditional methods.