# Copyright (c) 1999 University of Southern California. # All rights reserved. # # Redistribution and use in source and binary forms are permitted # provided that the above copyright notice and this paragraph are # duplicated in all such forms and that any documentation, advertising # materials, and other materials related to such distribution and use # acknowledge that the software was developed by the University of # Southern California, Information Sciences Institute. The name of the # University may not be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED # WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF # MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The LSAM Proxy Cache ==================== LSAM Authors: Lars Eggert, Amy S. Hughes, Stephen Suryaputra, Joe Touch, Ted Faber $Revision: 1.36 $ ($Date: 1999/06/18 20:02:15 $) More Information: http://www.isi.edu/lsam Introduction ------------ The LSAM project's primary deliverable is a modified version of the Apache proxy cache, enabled with LSAM's technology to support virtual distributed multicast caching. The LSAM proxy uses multicast push based on interest groups, to reduce server and network load, and increase client performance. The code is distributed as FreeBSD 2.2.6 compatible versions of: - modified Apache proxy cache 1.3b1 - port of AFDP 0.98, a multicast file transfer protocol Features of the LSAM Proxy architecture - Automatically converges at network aggregation points - Uses multicast to reduce network and server load - Uses self-configuring interest groups to direct the multicast For more information about the LSAM project, and the LSAM Proxy Cache current releases, see our web pages at: http://www.isi.edu/lsam/ Distribution Features --------------------- This distribution contains a FreeBSD 2.2.6 port of the following components: Apache v1.3b1 web proxy cache extended as indicated below AFDP v0.98 multicast file transfer system custom Makefile, but otherwise as-is CGI scripts, requiring Perl v5 used to manage the system these will be included into the Apache code in the next available version of the LSAM proxy cache The Apache proxy cache has been modified to support distributed multicast push of related web pages, including: - automatic configuration of a proxy hierarchy - support for prioritized web responses - a partitioned cache supporting multiple cache replacement policies - automatic multicast push of web pages over user created affinity channels - request analysis to suggest affinity channels to improve performance at the local server. You will download the file lsam-proxy-1.1.tar.gz Distribution Contents --------------------- lsam-proxy-1.1/ LSAM-installscript - a shell script which configures and compiles the full apache proxy and AFDP. README.lsam - this file README.lsam-install - installation overview for the LSAM proxy and all its components. README.lsam-api - documentation of LSAM API in the script form afdp/ - The source code for the AFDP multicast environment This directory should contain the source code and instructions to build the AFDP enviroment. README.lsam-afdp - Notes on Apache/AFDP integration. apache/ - the apache proxy server with LSAM enhancements .lsam-passwd - the password file used to control access to the configuration scripts in cgi-bin/lsam Valid users are: lsam-root:root guest: Passwords can be added or changed using apache's htpasswd utility. README.lsam - this file README.lsam-install - installation overview for the LSAM proxy and all its components cgi-bin/lsam/ - the directory containing CGI scripts needed to interact with the server and its configuration. LA_API.README - documentation of LSAM API in the script form LA_channel.README - documentation of the .channels file format and explanation of how channels work. README.lsam-scripts - explains the purpose of the files and how to interact with the scripts. demo-files/ - all files in this directory are used to simulate the real server configuration data and shield server control from guests. conf/ - templates for the configuration files. README.lsam-conf - describes the LSAM configuration files access.conf-LSAM-dist - the access.conf file which contains all of the directives needed to run the LSAM proxy. httpd.conf-LSAM-dist - the httpd.conf file which contains all of the LSAM configuration directives and activates all apache directives necessary to run the LSAM proxy. lsam_logdaemon.conf-LSAM-dist - this configuration file controls the executation of lsam_logdaemon. The daemon periodically evaluates the request logs and makes channel suggestions. srm.conf-LSAM-dist - the srm.conf file which contains all of the directives needed to run the LSAM proxy. htdocs/ - the apache documentation and HTML documents README.lsam-htdocs - describes the LSAM HTML documents index.html - this is a redirect to the main LSAM config page lsam/ - this contains a copy of index.html lsam-demo/ - this directory contains a standalone browser demo of the virtual distributed cache concept lsam-images/ - images used in LSAM pages manual/ - the Apache manual (standard Apache distribution) src/ - the apache source code and LSAM modifications. Configuration.LSAM.tmpl - a complete LSAM Configuration file Configure.lsam - the Configure script which calls LSAM files Makefile.LSAM.tmpl - a complete Makefile.tmpl with extra LSAM requirements added. Makefile.tmpl.dist - the apache distribution of the Makefile.tmpl renamed to work properly with other code. README.lsam - link to this file README.lsam-caching - describes the different caching algorithms available for use with the LSAM proxy. README.lsam-install - link to ../README.lsam-install. README.lsam-logdaemon - describes the LSAM server/proxy log analyzer. README.lsam-measure - describes the LSAM usage monitoring additions to apache. README.lsam-relocation - describes the LSAM proxy relocation mechanism. README.lsam-routing - describes the LSAM routing support provided. README.lsam-scheduling - describes the LSAM background traffic scheduling mechanism. README.lsam-tvgmulticast - describes the LSAM "TV Guide" channel announcements. main/ - the main apache source code lsam_config.h - the header file which contains definitions needed to compile LSAM features into apache. lsam-extras/ - other source code needed by LSAM README.lsam - description of contents. Limitations ----------- LSAM proxy must be run as non-root, therefore it cannot use ports less than 1024. Some remote sites return with the error "Document contain no data". We have not been able to discover the cause for this. One suggestion was that it was due to cookies, but other sites that use cookies are accessible. Changes ------- V1.1 - 2/24/99: Added a local demo that relies on Javascript and can be run independently of the LPC within any browser that supports Javascript. It illustrates the effect that a virtual distributed cache has on reducing the latency involved with retrieving files. The demo can be accessed via http://your-server:port/lsam-demo/lsam-demo.html A daemon management web page was added to allow the administrator to verify that all necessary components are running. LPC-lite, available separately, was developed. It performs a one-time, automated, multicast push of selected documents and their in-line images. This set of scripts can be added to any apache server without modifying the apache code. This can be accessed via http://your-server:port/cgi-bin/LPC-lite/LPC-lite-cgi.pl The automated join/leave utility was expanded to included automatic addition and removal of channels as their popularity warrants. V1.0 - 8/24/98: The relocation system has been heavily revamped, improved and documented. It is now possible to relocate caches based only on the geographic name. See lsam-docs/relocation for a description of the protocols and the lsam manual pages for descriptions of the relocation system components (geod(8), relod(8),reloc_xfer(1)) The background scheduling code has been improved; background connections will now always utilize the full rate limit. Multicast backgrounding is supported via a modified AFDP. The TV Guide Channel code is more mature now; all of its parameters are configurable from the web pages (esp. TTL of multicast pushes/announcements). The LPC does not run properly from the user 'root'; access by that user and configuration to ports requiring root permissions (<1024) is now prohibited. This is related to a conflict regarding process children and signals. The configuration using ports <1024 is probilited, and starting the system as 'root' is prohibited (only via the apachectl.lsam script). The system now auto-configures for both MD5 and DES-based 'crypt' libraries. This affects the default value of the /cgi-bin/lsam/.htaccess file. The API has been extended with a command-line function to call the API on another host, called LA_http.pl LSAM-installscript requires an explicit argument or prompted response, to tell it what to build and install. LSAM-installscript now leaves LSAM-installscript.log as a logfile of both STDOUT and STDERR Background cut-through request has been added. There is still possible to do foreground cut-through request, however it is strongly not recommended as it will hog the network. A mechanism has been added to probe whether the origin server supports differentiated services Run and tested under FreeBSD 2.2.6 instead of 2.2.5 All channel information is stored in one central file. Time information has been included for announced and received channels as well as for cached files. v0.8 - 7/24/98 (in original 0.8 release, but omitted from Changes list) Added shell script to coordinate installation, avoiding the need for manual configuration instructions. Unified DEFINEs - all LSAM features now enabled by default by a single _LSAM DEFINE. See src/main/lsam_config.h. All required parameters are on by default in the config files. Web control pages use simple password security (.htaccess-based) Web control pages default to demo-mode on dummy data when using no password, or 'guest' or 'anonymous' A running version of the demo control pages can be sampled at http://mul.isi.edu/ A command-line control API has been added. It is accessible remotely via the GET or POST method (provided the appropriate username/password is provided, in addition to the specified parameters). A tool to simplify remote application access will be provided in an upcoming release. Support for proxy relocation, including geographic proxy discovery and migration support. This is an intial implementation, and will significantly improve in the next release Automatic suggestion of both server and client actions, such as channel creation, deletion, join, and leave In a future release, actions themselves will be automatic, rather than only flagged. On-line log analysis to suggest server and client actions Extended facilities for resource utilization monitoring. V0.8 - 7/1/98: Added complete web interface. Multicast channels can now be created at the server. Individual files can be pushed out on a given multicast channel. Files are also multicast in response to requests at the server. Clients can tune in a desired channel and receive multicast updates from it. A daemon has been added to analyze the request logs and suggest possible channels to the server administrator. This same daemon makes suggestions about which channels to receive as well. The cache has been partitioned. Files from a single channel are stored in each partition. All non-channel cached files are stored in the generic partition. In response to a changed channel multicast address, the client receiver will be automatically updated. If a tuned channel disappears, the system will automatically detune that channel. Foreground cut-through request has been added. Background response scheduler now configurable from the web page; real-time live stats available via the web page, various fixes. Some instability in lsam_tree hierarchy protocol implementation has been fixed. Now using AFDP 0.98 which includes our FreeBSD patches by default. V0.71 - 4/6/98: Added full source code for AFDP v0.97b ported to FreeBSDv2.2.5 V0.7 - 3/98: This is the initial release of the LSAM Proxy Cache