tcp-over-satellite-digest Saturday, January 25 1997 Volume 01 : Number 001 In this issues: FWD>RFC 2001 on TCP See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: 23 Jan 1997 16:55:45 -0800 From: "Aaron Falk" Subject: FWD>RFC 2001 on TCP Mail*Link¨ SMTP FWD>RFC 2001 on TCP This may be of interest to you folks. Aaron - -------------------------------------- Date: 1/23/97 4:02 PM From: RFC Editor A new Request for Comments is now available in online RFC libraries. RFC 2001: Title: TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms Author: W. Stevens Date: January 1997 Mailbox: rstevens@noao.edu Pages: 6 Characters: 12975 Updates/Obsoletes: None URL: ftp://ds.internic.net/rfc/rfc2001.txt Modern implementations of TCP contain four intertwined algorithms that have never been fully documented as Internet standards: slow start, congestion avoidance, fast retransmit, and fast recovery. The purpose of this document is to document these four algorithms for the Internet. This is now considered a Proposed Standard Protocol. This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. This announcement is sent to the IETF list and the RFC-DIST list. Requests to be added to or deleted from the IETF distribution list should be sent to IETF-REQUEST@CNRI.RESTON.VA.US. Requests to be added to or deleted from the RFC-DIST distribution list should be sent to RFC-DIST-REQUEST@ISI.EDU. Details on obtaining RFCs via FTP or EMAIL may be obtained by sending an EMAIL message to rfc-info@ISI.EDU with the message body help: ways_to_get_rfcs. For example: To: rfc-info@ISI.EDU Subject: getting rfcs help: ways_to_get_rfcs Requests for special distribution should be addressed to either the author of the RFC in question, or to admin@DS.INTERNIC.NET. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. Submissions for Requests for Comments should be sent to RFC-EDITOR@ISI.EDU. Please consult RFC 1543, Instructions to RFC Authors, for further information. Joyce K. Reynolds and Mary Kennedy USC/Information Sciences Institute ... Below is the data which will enable a MIME compliant Mail Reader implementation to automatically retrieve the ASCII version of the RFCs. - --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ds.internic.net" Content-Type: text/plain Content-ID: <970123142601.RFC@ISI.EDU> SEND /rfc/rfc2001.txt - --OtherAccess Content-Type: Message/External-body; name="rfc2001.txt"; site="ds.internic.net"; access-type="anon-ftp"; directory="rfc" Content-Type: text/plain Content-ID: <970123142601.RFC@ISI.EDU> - --OtherAccess-- - ------------------ RFC822 Header Follows ------------------ Received: by qmgate.trw.com with ADMIN;23 Jan 1997 15:55:20 -0800 Received: from ietf.org by mailhub1.trw.com; Thu, 23 Jan 97 15:48:48 -0800 Received: from ietf.org by ietf.org id aa02584; 23 Jan 97 17:41 EST Received: from zephyr.isi.edu by ietf.org id aa02468; 23 Jan 97 17:36 EST Received: from zephyr.isi.edu by zephyr.isi.edu (5.65c/5.61+local-23) id ; Thu, 23 Jan 1997 14:34:11 -0800 Message-Id: <199701232234.AA19422@zephyr.isi.edu> To: IETF-Announce:; Subject: RFC 2001 on TCP Cc: rfc-ed@isi.edu Date: Thu, 23 Jan 97 14:34:11 PST Sender: ietf-announce-request@ietf.org From: RFC Editor ------------------------------ End of tcp-over-satellite-digest V1 #1 ************************************** tcp-over-satellite-digest Saturday, February 1 1997 Volume 01 : Number 002 In this issues: RFC 2001 Re: TCP/IP and spoofing Draft Agenda for Internet P Updated agenda for Internet See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- End of tcp-over-satellite-digest V1 #2 ************************************** tcp-over-satellite-digest Saturday, February 8 1997 Volume 01 : Number 003 In this issues: TCP Over Satellite Paper Re: TCP Over Satellite Paper Re: TCP Over Satellite Paper TIA-SCD-CIS-Internet Protoc ITU GII Report Available Spoofing in DirecPC A question on errors Re: A question on errors Re: A question on errors Re: A question on errors Re: satellite TCP performan Re: satellite TCP performance enhancement Re: satellite TCP performance enhancement See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 03 Feb 1997 14:07:13 -0500 From: Mark Allman Subject: TCP Over Satellite Paper At the NASA Lewis meeting last week, we discussed our TCP Over Satellite paper with a number of you, and I just wanted to let everyone know that it is now available. This is a performance comparison of various TCP extensions run over the ACTS system. The following is the reference and the web page... Mark Allman, Chris Hayes, Shawn Ostermann, Hans Kruse. TCP Performance Over Satellite Links. Proceedings of the Fifth International Conference on Telecommunications Systems, Nashville, TN, March, 1997. (To Appear). http://jarok.cs.ohiou.edu/papers If you have any problems obtaining the paper, please let me know. Also, we would appreciate any comments you might have on this work. Mark ------------------------------ Date: Mon, 3 Feb 1997 12:02:59 -0800 From: fred@cisco.com (Fred Baker) Subject: Re: TCP Over Satellite Paper At 2:07 PM 2/3/97, Mark Allman wrote: >At the NASA Lewis meeting last week, we discussed our TCP Over >Satellite paper with a number of you, and I just wanted to let >everyone know that it is now available. that's a real interesting paper. I assume that you are aware of how to configure your 2514 to have deep queues, and how to configure WFQ and RED on it. Right? ------------------------------ Date: Mon, 03 Feb 1997 15:15:11 -0500 From: Mark Allman Subject: Re: TCP Over Satellite Paper > that's a real interesting paper. I assume that you are aware of > how to configure your 2514 to have deep queues, and how to > configure WFQ and RED on it. Right? Fred- We do understand how this is done. We used the standard queuing mechanisms for this study, with one exception. The queue size and the TCP receive window size were carefully engineered to provide a congested or uncongested gateway, depending on the test. I am sure that WFQ and RED will play a part in our future experiments. Thanks for the note, allman ------------------------------ Date: 4 Feb 1997 17:11:34 -0800 From: "Aaron Falk" Subject: TIA-SCD-CIS-Internet Protoc Notes from the TIA-SCD-CIS-Internet Protocols Working Group January 30, 1997 NASA Lewis Research Center, Cleveland, OH 8:30am - 5:00pm 20 degrees F. Here are some notes I made regarding the meeting: Short term goals: - -Sort through the issues involving maximizing the performance of TCP over satellites, address which types of satellite systems are affected - -Collect those aspects that can be solved simply through the IETF-- i.e.solutions which use standards track implementations or those not requiring substantial research - -Work with IETF to form a BOF/WG to address these issues - -Identify a working plan to address issues requiring more substantial research - -Identify technology demonstrations that will address what is becoming known as the ÒLA Times article.Ó I.e. that TCP cannot provide high throughput over satellite. Long Term Goals- - -Address research aspects of TCP over satellite - -Address other Internet protocols that should be addressed for use over satellite (UDLR, RSVP, multicast, TCP over ATM) - ---- To address the short term goals the working group is going to produce several documents: - - A table of problem vs. domain. Domains include fixed vs. mobile, GEO vs. LEO, high data rates vs. low data rates, Òfiber-typeÓ error rates vs. typical Ka/C-band error rates, applicability to Gbit terrestrial networks. Problems do not have to have been demonstrated just suspected. This document is meant to capture some of the assumptions in the dialogs weÕve been having about what works and what doesnÕt. Lori Jeromin (MIT Lincoln Labs) has agreed to be the editor of this doc and take the first stab at it. - - A list of possible changes to TCP that are satellite friendly and why. This also is meant to be a baseline for future discussion. A list of existing or potential TCP options in one doc could become the begining of a IETF working group charter. ItÕll certainly be useful in demonstrating whether we are ready for a BOF yet. One scenario IÕve discussed with Allyn is that a collection of existing TCP options can be collected into an informational Internet Draft. Even if this doesnÕt solve all the problems this can be useful for vendors who want to support satellite services or folks who need ammunition to get certain options included in an implementation.Eric Travis (MITRE) and Shawn Osterman (OU) have agreed to be the editors of this doc. - - A list of implementations of TCP over satellite. This can be a list of pointers to sites with code and/or results so that those interested can keep from reinventing the wheel. The consensus was that this belongs on a web page and Scott Corson (corson@isr.umd.edu) has agreed to create the working group web page. Please send him pointers to any known implementations. Also we might include planned implementations. Benefits of these documents include becoming a basis for creating a IETF BOF/WG, to collect the the various efforts of the working group members in one place, and to develop a strategy for solving research problems. Several WG members have volunteered resources for developers to use. Panamsat has offered DirecPC links (12Mbps) and MCI links (6Mbps) as well as T1 to E1 rates. Also NASA has offered access to the ACTS satellite T1. NASA, Hughes, UMd, and Ohio Univ. have ACTS terminals. The question came up regarding our (TIA-SCD-CIS-Internet Protocols Working Group) interface with the IETF. For the moment there are two aspects to that interface. First, I am in contact with Allyn Romanow a Transport Area Director. We have been discussing the optimal way to draw upon the wider Internet community. This may include a BOF at the Memphis meeting of the IETF, April 7-11. Second, the existance of the tcp-over-satellite LISTSERV has been announced to IETF members who have an interest in wireless communication and many have already subscribed. My hope is that we can conduct much of the business of the working group over email. However, I am planning on calling another working group meeting at the Memphis IETF meeting. It may take a little while to coordinate with facilities but IÕll get the announcement out as soon as possible. It may be that the next working group meeting will be the IETF BOF for TCP over satellite. Whether that BOF gets created is really a function of how well we can create drafts of the documents described above. Thanks for your support. See you on the net... Aaron Attendees: Aaron Falk, TRW Allyn Romanow, IETF/Sun Micro Kent Price, Loral Space & Comm Dan Shell, Cisco Randy Tabor, PanAmSat Chris Schram, PanAmSat Victor Barajas, Hughes Spaceway Brian Kachmar, Analex Corp./NASA LeRC Paul Mallasch, Analex Corp/NASA LeRC Anil Agrawal, COMSAT Labs Daniel Kohn, Teledesic Eric Bobinsky, Terasphere Lori Jeromin, MIT Lincoln Labs Tim Shepard, BBN Eric Travis, MITRE William Kissick, NTIA/ITS (Dept. of Commerce) Raj Jain, Ohio State Univ. Karen Hansen, Ohio Univ. Shawn Osterman, Ohio Univ. Mark Allman, Ohio Univ. Chris Hayes, Ohio Univ. Scott Corson, Univ. of Md Adrian Hooke, NASA/JPL Polly Estabrook, JPL Keith Scott, JPL Pat Gary, NASA GSFC Cindy Tran, NASA LeRC Will Ivancic, NASA LeRC Satellite Networks & Architectures Dan Glover, NASA LeRC Duc Ngo, NASA LeRC Rich Kurak, NASA LeRC Telecom & Networking Thomas Wallett, NASA LeRC/5610 Robert Kerczewski, NASA LeRC/5610 Kul Bhasin, NASA LeRC ------------------------------ Date: Wed, 05 Feb 1997 16:50:43 -0500 From: "Eric A. Bobinsky" Subject: ITU GII Report Available As I mentioned at the NASA Lewis meeting last week, the report of the fourth meeting of the ITU-T Joint Rapporteurs Group on the GII is now available on our website. The URL is www.terasphere.com, and the report itself can be found under "White Papers and Reports". I'm using a new server, so if you have any problems I'd appreciate it if you'd let me know. The same report can be obtained directly from the ITU server in Geneva (when it feels like responding) at www.itu.ch. Cheers! Eric PS - yeah, I know the website isn't fancy; maybe one of these days I'll get ambitious! >------------------------------< Eric A. Bobinsky Terasphere 343 W. Bagley Road, Suite 405 Post Office Box 10 Berea, OH 44017-0010 Tel +1 216 243 2992 Fax +1 216 243 2934 Internet eric@terasphere.com >------------------------------< ------------------------------ Date: 5 Feb 1997 22:15:49 -0800 From: "Aaron Falk" Subject: Spoofing in DirecPC Narin- I'm involved in a technical working group for the satellite industry that is addressing optimizing TCP performance over satellite. I've just been re-reading my thesis trying to recall what kind of spoofing was implemented in DirecPC. The prototype that we produced did not drop ACKs in the user terminal nor did the satellite gateway spoof for the end user but these ideas were suggested as ways to increase throughput. Since we were able to demonstrate only 140kbps compared to the 400-3000kbps that DirecPC is advertising, I'd say they made some mods in that general direction. Are you aware of what was implemented? The reason I'm asking is that as our working group is getting to know the IETF (& I'm getting a better understanding of the TCP protocol), the benefits of using ACKs to clock new packets into the network are becoming clearer. This is used for congestion control for the network as a whole not just to limit retransmissions, as I stated in my thesis. Therefore, by spoofing the server at the satellite uplink there is no congestion control for whatever is on the downlink end of the satellite connection. This wasn't an issue when there was only one terminal per TCP over satellite connection but I understand that PanAmSat is going to use DirecPC to provide Internet service to South America, Central America (and Africa, I think). There is no reason, other than congestion control, why you wouldn't put a network at the downlink end of the connection. This network would then be subject to the congestion that be prevented by the proper flow of ACKs. In fact, the arrangement could be turned around so that the downlink end was connected to the "US" Internet and the server was offshore (perhaps a major archive of alt.binaries.erotica.redheads, perhaps) submitting the larger Internet to that congestion. If this is the case, I think that we should let the HNS folks know that this wouldn't be a neighborly way to behave on the Internet. Can you see what you can find out? Thanks, Aaron PS. I'll be at UMd for the Center for Satellite and Hybrid Communication Networks meeting on Feb. 19. See you there. Aaron PPS. I'm posting this note to the tcp-over-satellite LISTSERV to see if anyone else is familiar with the way that DirecPC is being implemented. ------------------------------ Date: Thu, 6 Feb 1997 18:00:08 -0400 From: hkruse1@ohiou.edu (Hans Kruse) Subject: A question on errors Here is a question for (hopefully) productive discussion: What will the reasonably expected error performance of future satellite systems be? (*) We know that the raw satellite link has non-zero error performance. (*) We know (from ACTS among others) that the satellite link _can_ be made error-free. (*) I know from my work with the ACTS T1-VSAT that there is a bandwidth penalty for using the error correction code. My expertise is in protocols and network management. Could some of you knowledgable in the RF and coding stuff comment on whether the FEC coding to make satellite channels "error free" will continue to carry a significant bandwidth penalty? Hans Kruse, Associate Professor McClure School of Communication Systems Management, Ohio University 9 S. College Street Athens, OH 45701 614-593-4891 voice, 614-593-4889 fax, hkruse1@ohiou.edu ------------------------------ Date: Thu, 6 Feb 1997 15:13:54 -0800 From: touch@ISI.EDU Subject: Re: A question on errors > From: hkruse1@ohiou.edu (Hans Kruse) > Subject: A question on errors > > Here is a question for (hopefully) productive discussion: > > What will the reasonably expected error performance of future satellite > systems be? > > (*) We know that the raw satellite link has non-zero error performance. > > (*) We know (from ACTS among others) that the satellite link _can_ be made > error-free. Given enough FEC, this is true for any link, provided you burn the BW and increase the latency. Known since 1948, which predates ACTS, I think :-) The BW penalty isn't a big issue, it's whether you're left with a usable channel in the end, i.e., whether the BW remaining and latency incurred in the encoding is acceptable. Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: 6 Feb 1997 18:22:19 -0800 From: "Aaron Falk" Subject: Re: A question on errors Reply to: RE>>A question on errors The question is really an economic one. There is a cost-performance trade that each system will make of how much can be charged for the usable bandwidth and how good does good have to be. So the question can be turned around: what kind of error rates are necessary for TCP? At what probability of error does congestion control start to cause significant problems? Aaron - -------------------------------------- Date: 2/6/97 3:15 PM To: Aaron Falk From: touch@ISI.EDU > From: hkruse1@ohiou.edu (Hans Kruse) > Subject: A question on errors > > Here is a question for (hopefully) productive discussion: > > What will the reasonably expected error performance of future satellite > systems be? > > (*) We know that the raw satellite link has non-zero error performance. > > (*) We know (from ACTS among others) that the satellite link _can_ be made > error-free. Given enough FEC, this is true for any link, provided you burn the BW and increase the latency. Known since 1948, which predates ACTS, I think :-) The BW penalty isn't a big issue, it's whether you're left with a usable channel in the end, i.e., whether the BW remaining and latency incurred in the encoding is acceptable. Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: Fri, 07 Feb 1997 09:31:44 -0500 From: "Shawn Ostermann" Subject: Re: A question on errors >> [...] >> what kind of error rates are necessary for TCP? :-) never quite heard it phrased that way. >> At what probability of error does congestion >> control start to cause significant problems? >> >> Aaron The brief studies that we did for the paper that we mentioned at NASA a couple weeks ago showed that For bit error rates of 1 times 10^{-7} (the hardware emulator was programmed to insert 4000 bit errors every 10,000,000 bits), the addition of SACK to the standard TCP Reno code is shown to greatly improve throughput relative to TCP Reno without SACK. At this error rate, we're destroying approximately one or two segments out of 2400, or injecting 3 loss events into a 5 Mbyte file transfer. Optimal throughput for this line speed would be approximately 165 Kbytes/second. TCP Reno achieved approximately 22\% of this rate, while TCP SACK was able to achieve approximately 41\% efficiency. For bit error rates of 1 times 10^{-6}, we're destroying approximately one or two segments out of 240, or injecting 30 loss events into a 5 Mbyte file transfer. TCP Reno only achieved approximately 4\% of the optimal rate, while TCP SACK was able to achieve approximately 9\% efficiency. At an error rate of 1 times 10^{-5}, affecting one or two segments out of 24, both of the protocols exhibits very poor performance. Not particularly comprehensive, but even with a "damaged segment" rate of 1 or 2 in 2400, TCP's throughput was only 41% of optimal even for the best TCP options we could find to test. - --sdo - ------------------------------------------------------------------------- Dr. Shawn Ostermann - Assistant Professor - Ohio University 140 Morton Hall, Ohio University, Athens, Ohio 45701-2979 ostermann@cs.ohiou.edu -- FAX: (614)593-0406 -- Voice: (614)593-1242 http://www.cs.ohiou.edu/people/faculty/ostermann.html ------------------------------ Date: 7 Feb 1997 09:15:50 -0800 From: "Aaron Falk" Subject: Re: satellite TCP performan - -------------------------------------- Date: 2/6/97 9:05 PM To: Aaron Falk From: Eric Travis Hopefully Tim Shepard, Aaron Falk or someone else will correct me if I misunderstood, but: At the TIA meeting last week Tim took the opportunity to have the attendees revalidate the above assumption. I *thought* the answer that came back was mixed, leaning toward "Maybe" and "It depends". The point is, there didn't seem to be consensus within the group. - ------------------------- The lack of consensus is due to the fact that satellite systems are far from homogeneous. And are becoming less so. In very rough terms, satellite systems are being referred these days as having three generations. The first generation is what is on orbit these days. PanAmSat is planning on using existing C-band or Ku-band satellites to offer Internet services using DirecPC to countries in South and Central America (& Africa, I think). These satellites don't have great error performance (I'll try to get a number). The second generation of satellites are the global voice satellite systems such as Motorolla's Iridium and TRW's Odessey. These satellites are designed primarily for voice and very low rate data (9600 bps fax). The third generation of satellites are the Ka-band systems that are proposed for deployment in the 2000-2010 era. These systems are being specifically designed for personal communications services and are being advertised to offer 64kbps - T1 rate data to the ! end user. There are 11 systems t hat have filed with the FCC. You'll hear a lot more about these systems at the IETF Plenary in Memphis. Since the Ka-band systems are designed to offer PCS services they need to be able to support real time multimedia traffic and, hence, need excellent error performance. Whether the error rates will vary depending on the type of traffic carried is TBD for each system. A couple of problems in getting data on all these systems is that they are all competing for a similar pool of investors who are waiting for a shakeout among providers. Additionally, the systems are still in the design phase and all the aspects of the design are being carefully guarded. One of the short term goals for the TIA Internet Protocols over Satellite working group is to document as much as we can about the assumptions we can all agree upon for all the systems. One way to look at it is that the Ka-band systems will have to offer error rates that will let the protocols work. If we know what those rates are they will influence the design. I am cross posting this response (& I suggest others do also) to tcp-over-satellite@listserv.trw.com to get the input from the satellite crowd. Aaron - ------------------ RFC822 Header Follows ------------------ Received: by qmail4.sp.trw.com with ADMIN;6 Feb 1997 21:02:56 -0700 Received: by zephyr.isi.edu (5.65c/5.61+local-23) id ; Thu, 6 Feb 1997 19:55:06 -0800 Received: from venera.isi.edu by zephyr.isi.edu (5.65c/5.61+local-23) id ; Thu, 6 Feb 1997 19:55:05 -0800 Received: from mail.clark.net by venera.isi.edu (5.65c/5.61+local-26) id ; Thu, 6 Feb 1997 19:55:03 -0800 Received: from clark.net (root@explorer.clark.net [168.143.0.7]) by mail.clark.net (8.7.3/8.6.5) with ESMTP id WAB05488; Thu, 6 Feb 1997 22:53:12 -0500 (EST) Received: from localhost (travis@localhost [127.0.0.1]) by clark.net (8.8.4/8.7.1) with SMTP id WAA12493; Thu, 6 Feb 1997 22:54:56 -0500 (EST) Date: Thu, 6 Feb 1997 22:54:55 -0500 (EST) From: Eric Travis Reply-To: Eric Travis To: esCraig Partridge Cc: touch@ISI.EDU, perry@piermont.com, halsem@smtpgate.adm.intelsat.int, end2end-interest@ISI.EDU Subject: Re: satellite TCP performance enhancement In-Reply-To: <199702062112.NAA23106@aland.bbn.com> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-end2end-interest@ISI.EDU Precedence: bulk ------------------------------ Date: Fri, 07 Feb 1997 16:10:10 -0500 From: Tim Shepard Subject: Re: satellite TCP performance enhancement > >Don't do that yet -- Tim Shepard has found some cases in simulation where > >starting with 4096 actually appears to hurt, not help. We need some time > >to puzzle through this stuff. > > Can you send a description of the situation? > What I showed at the meeting in cleveland last week was all pretty simple stuff. All simulations where done with the "tcp" module in the LBL ns simulator version 1.0. My purpose was to use my TCP plotting method to look in detail at the traces to illustrate how TCP works today. I believe this "tcp" module in ns-1.0 corresponds most closely to the TCP in the tahoe release (which includes the algorithms described in Van Jacobson's 1988 paper, but not later improvements that appeared in the reno BSD release (fast recovery)). All simulations used a single tcp connection which had an infinite source of data (the "ftp" source as it is called in ns-1.0). All simulations had a queue limit of 6 packets, and packets which were sent by the TCP when there were already six packets in the queue were dropped. I had two different network configurations, one called "terrestrial" and one called "satellite". Both had 1.5 megabit per second information rates. The "terrestrial" configuration had a 25 ms one-way delay, while the "satellite" configuration had a 250 ms one-way delay. I ran TCP with a 16kb end-to-end window over both scenarios. Works fine on "terrestrial", but bumps up against the end of the window on "satellite" and hence goes at only about one-eighth speed. I believe this is no surprise to anyone. I ran TCP with a 500kb end-to-end window over both scenarios. The plots get much more interesting. Over the "terrestrial" configuration, the trace shows that the TCP does manage to go at about 1.5 megabits per second, but it does not get up against the end of the window. This one TCP has managed to congest the network all by itself, and then it responded gracefully. It beautifly illustrates how the 1988 Van Jacobson algorithms manage to find the right rate to send at (and a workable window to stay within) even though it cannot get up against the end-to-end TCP window. Over the satellite, things are not so good. Because the TCP starts out sending at rate which is twice what the link can handle (within each slow-start spurt, once-per round-trip time) and it gets no feedback for a whole RTT (which is one-half second here), many more packets get lost at the end of the first slow-start episode, and ssthresh then gets halved multiple times in the ensuing confusion. It turns out, that with the 500kb end-to-end window, things go just as poorly over the "satellite" configuration as they would have if the window had been left at 16kb. Making the window bigger alone did not help. (Some have pointed out that hand-tuning the end-to-end window for the particular delay-bandwidth product of this particular link would help much here. But hand-tuning is not a general solution.) To illustrate the 4k-start proposals that some have been talking about, I tweaked the ns-1.0 tcp module "window-init" up to a value of four (packets). This began all slow-start episodes (not just the connection start) with a 4k window. (This knob was set to one in previous simulations, so in those simulations slow-start began each time with a single one-kilobyte packets. This is the normal behavior of TCP-tahoe.) Over the "satellite", this 4k-start behaved even more poorly than the "satellite" experiment described in the previous paragraph. The 4k-starts that happen in the slow start episodes after the initial loss just lead to more losses which causes even more reductions in ssthresh, leading eventually to an even lower congestion window in the sender. SACK probably would have helped here. This simulation was useful to illustrate at the start how it could move a small file (of just a few kilobytes) quite a bit faster over a satellite than the usual one-packet start. I think it also points out how subtle the effects of tuning these algorithms can be. I should also point out that _none_ of my simulations match the suggested changes that Sally just sent out, so I don't believe any conclusions can be drawn from my simulations about her proposal in particular. I believe I have said nothing yet in this message that would be news to others who are actively thinking about and experimenting with TCPs congestion-control-related algorithms. My purpose of presenting these simulations at the meeting in Cleveland last week was to make sure that all present where aware of the details of how the Van Jacobson 1988 algorithms work, and about their importance to the health of the Internet. (I showed showed some of the traces that were in my 1990 SM thesis which showed how badly the 4.3BSD TCP was behaving in the Internet circa 1988 when the net was apparently suffering from congestion collapse.) I do believe that caution is called for in evaluating these 4k-start proposals. 4k-start will clearly improve performance of short transfers in uncongested situations, but it increases the impulsiveness of the traffic offered to the net, which may increase the congestion, which may overall be counter-productive. Some have said that that is no worse than opening four TCP connections in parallel to try to go faster. That's probably true. But I don't believe it would be good idea to be opening 4 connections in parallel, each of which is doing the 4k-start. So perhaps a TCP stack should only do a 4k-start only if it only has one connection open to that destination. Or perhaps multiple TCP connections to the same destionation should be treated as one meta-connection for congestion-control related algorithms. There's an Internet Draft (already expired but still on the ftp server !!) by Joe Touch which presents this idea with a good bit of detail. Perhaps it should be married to the 4k start proposal so as to properly handle things higher in the stacks which open multiple connections in parallel. I do expect that starting with 4k may be valuable when coupled with something Vegas-like because it may allow the sender to learn more about the available bandwidth on the path during the first round trip time and hence do a better job of not exceeding the available capacity for too long (which can lead to its own demise). But this is getting researchy. (We all have much work to do!) For those who want to see the pictures I had in cleveland... First, type these commands on your favorite Unix box (assuming you've got all the latest gnu tools like "wget" and the gnu version of "tar".) wget ftp://ftp.lcs.mit.edu/pub/lcs-pubs/tr.outbox/MIT-LCS-TR-494.ps.gz gunzip MIT-LCS-TR-494.ps.gz lpr MIT-LCS-TR-494.ps wget ftp://mercury.lcs.mit.edu/pub/shep/xplot-0.89.tar.gz tar xfz xplot-0.89.tar.gz cd xplot-0.89 ./configure make That much will get you a copy of my 1990 SM thesis printed, and will also get you a copy of my xplot program compiled. Then get ns-1.0 from LBL and compile it. See the stuff at http://www-nrg.ee.lbl.gov/ http://www-nrg.ee.lbl.gov/ns/ ftp://ftp.ee.lbl.gov/ns-1.0.tar.gz to get the LBL simulator. (There are later versions available for ftp. I haven't upgraded yet, so I don't yet know if the script attached below will work with them, but I expect it well work in all the ns-1.* versions.) Then, use the cleveland.tcl script which I'll attached below (gzipped and uuencoded) to generate the six out.*.xplot files with ns: ns cleveland.tcl terrestrial ns cleveland.tcl satellite ns cleveland.tcl terrestrial_bigwin ns cleveland.tcl satellite_bigwin ns cleveland.tcl terrestrial_start4k ns cleveland.tcl satellite_start4k Click the right button on the xplot window each time to go on to the next simulation. Then look at them all at once with synchronized zooming by running: xplot -x -y out.*.xplot See the xplot-0.89/README file for instructions on how to zoom in and out with the mouse in xplot. -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 -- - -- - -- -- - -- - -- -- - -- - -- -- - -- - -- -- - -- - -- begin 644 cleveland.tcl.gz M'XL("$Z$^S(``V-L979E;&%N9"YT8VP`Y5?=;]LV$'^6_HJKH!8):AM2UO2A M@8=AP#8,Z/KD-]<(9(FVB="41E%Q`$/_>^](6M:'ZP(-DF:=`5OF\;[O^#NJ M4'D*J6*)9K>:*<5*K7@B8`\U['W?6XM\BX`P@7`)\>3ZGR5<76]+R%1>C'7"Q:)CIRNT(;DXBE`H MZLIX<\%E1CP?(5H8T7\K5K&QX%NNX7V'(3[%4/M^T.Q?12TK& MBDM>;O`A,`^-+RX3>L-N=Z@GW_DVJA7,\X))R"L]"4EF\E"(7,,.72HJ74*X M@B#+JR5J*_E:LBQH;6BN!0L:(PW=:`K0!'M@*22[._3$(X[?__CK[T^T\)+T M[IZK-9/,&+L)KL]1I$5Q>&BYC57&I MR-Z:=6=ZA1$\4)>OSKF/4 M;=>)\U63"4OWSAK^3!R>2$J-8IIOV0C<@ERP_K0(-X]4B-]CRPRT=W:-*1=Z M#*^F!T87[K=S:CTAMIZ908'@S.?[3)R(\V!O$&,-3)1L&%2F.2H[TU7=)CS# MW3;I1&O7,8=G^_A$AP9O51')X56/3K0X=EWJE-4M4].P9?8("%K!K[_AL7:' M,Q4Y1H]+LS('W$)&#T(.^PA.$8&312<"FYE*4L+H4N?%C'S5.,,^)?B'4-MB MAMK">&55'G;)$8KI2$<-;`,"GISF<3W,F)-`1G;SC@ M_YE"S5/)>T1=,''^.2RSL:7A$%8T1(UNEJ=4#' MW$J3:L>>5PKKAZ3CJ+,ICB8183_Q8L,I'1Q]'CAODF>K9OOA%Y)NQ;O`J"CQ MPOA+\W9ABV"=-'\;K1W&S0E&TP6JDL=.[!?H=LG7&/G3U>DZBG[*0KG$/7N] MC.?O[O[#!>LSC1&`-+S[X05UF7V6BK8N[=]3QT;\?P&/3;3/6YI'0>.W*_13 M`6,_:4]7J5.%>A0FOH!*O21$'&3U"0\=OC?MD5.M4WISBLU%F+28-^I29PC. ML*_*9,T^4'RI8/=,X#5XHE,!<]BWT'O4^%WO1ZX)1X<0:EC4=,/&I,9HV)@- MYERNC$GD`U@9-'+4 ,X/M?`"*37':]$@`` ` end ------------------------------ Date: Fri, 7 Feb 1997 17:11:41 -0500 (EST) From: Matt Mathis Subject: Re: satellite TCP performance enhancement >All simulations used a single tcp connection which had an infinite >source of data (the "ftp" source as it is called in ns-1.0). All >simulations had a queue limit of 6 packets, and packets which were >sent by the TCP when there were already six packets in the queue were >dropped. This is what I suspected. In the recent past some routers had as few as a dozen packet buffers. The fiascos at the NAPs and other places finally convinced everybody that this is just plain not enough buffering. (everybody = ISPs and router vendors) As far as I know most of the current and much of the previous generation of routers/bridges/switches had at least 100ms of buffer at full line rate, with average sized packets (1.g. 200 byte packets). So for example at OC-3 linecard would have to hold about 10k packets. This is roughly 100 mS if they are 200 bytes each, or 2 MB if they are full FDDI sized. (I am assuming packet queues) There is still some gear around that only holds 6 packets, but for the most part I would consider it to be broken. Consider the relative cost/value of speeding up WWW vs replacing/retiring the busted gear...... My one concern is the behavior of data over voice modems. However, I also don't trust simulations to accurately model all of the foibles of modem compression, etc. The experiment which we ("other person singular") should do is measure how the modem/slip/ppp routers currently in the field respond to 4kB initial windows. I will never trust a simulation to answer this question. So here is the experiment: Take a large multiserver WWW site. Run 4kB initial windows on *some* of the servers and collect packet traces from ALL connections. Inspect traces for statistics on the relative position of the first packet loss.... Can you estimate the population of slip/ppp routers with 6 packet buffers? How much does the 4kB initial window it really help WWW? How easy is it to make things worse? This sounds like a project for somebody at Altavista, Netscape or Microsoft. - --MM-- ------------------------------ End of tcp-over-satellite-digest V1 #3 ************************************** tcp-over-satellite-digest Thursday, February 13 1997 Volume 01 : Number 004 In this issues: Re: satellite TCP performance enhancement Re: satellite TCP performance enhancement realistic simulations; router buffer space ; solar system distances TCP experiments over ACTS re: TCP experiments over ACTS Re: realistic simulations; router buffer space ; solar system distances FWD>RE>satellite TCP perfor Another news article on TCP over satellites Re: Another news article on TCP over satellites Internet Protocols over Satellite (IPoS) Home Page Re: Internet Protocols over Satellite (IPoS) Home Page Re: Internet Protocols over Satellite (IPoS) Home Page Re: realistic simulations; router buffer space ; solar system distances See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Sat, 08 Feb 1997 22:36:57 +0100 From: Havard.Eidnes@runit.sintef.no Subject: Re: satellite TCP performance enhancement [CC list trimmed] Hi, although I wouldn't classify myself as an expert in this field I will nevertheless make a comment regarding your message describing the simulation setups for the "terrestrial" and "satellite" scenarios. Reading your "ns" script I see that you used the the same queue limit in the bottleneck in both setups, i.e. 6 packets. I think the erratic TCP behaviour (essentially way too low throughput) you observed for the "satellite_bigwin" scenario is mostly due to insufficient buffering at the bottleneck in this case. I think that it is (or at least should be!) common practice these days to operate with queue limits approximately proportional to the Delay x Bandwidth product of the path involved, and that what your simulation shows is that TCP (with huge windows) doesn't perform too well in a setup with too little buffering relative to the Delay x Bandwidth of the path, which shouldn't come as a large surprise to anyone (?). I think that recommending more than the 6 packets initially used is in line with the observations done by Curtis Villamizar in http://engr.ans.net/published.html#curtis9410 quoted earlier in this thread. I even took the time to download "ns" and tweak the create_satellite procedure in your simulation script to use a queue limit of 60 instead of 6, and it had a markedly positive effect (approximately 4X as far as I can see) on the "sattelite_bigwin" setup. It would also seem that in the underbuffered satellite case, starting with a 4K window (and big windows) actually hurts performance relative to simply using big windows. - - H=E5vard ------------------------------ Date: Sat, 08 Feb 1997 23:50:24 -0500 From: Curtis Villamizar Subject: Re: satellite TCP performance enhancement In message <199702072119.AA02682@venera.isi.edu>, Tim Shepard writes: > > > >Don't do that yet -- Tim Shepard has found some cases in simulation where > > >starting with 4096 actually appears to hurt, not help. We need some time > > >to puzzle through this stuff. > > > > Can you send a description of the situation? > > > > What I showed at the meeting in cleveland last week was all pretty > simple stuff. All simulations where done with the "tcp" module in the > LBL ns simulator version 1.0. My purpose was to use my TCP plotting > method to look in detail at the traces to illustrate how TCP works > today. I believe this "tcp" module in ns-1.0 corresponds most closely > to the TCP in the tahoe release (which includes the algorithms > described in Van Jacobson's 1988 paper, but not later improvements > that appeared in the reno BSD release (fast recovery)). The default is tahoe, but reno, scak, and fack are also offered. > All simulations used a single tcp connection which had an infinite > source of data (the "ftp" source as it is called in ns-1.0). All > simulations had a queue limit of 6 packets, and packets which were > sent by the TCP when there were already six packets in the queue were > dropped. It would be real nice if we could realistic whe doing simulations. Real routers in WANs have queues ranging from hundred to over a thousand, per output port. Our backbone routers are set to about 1500. This would be a lot full (FDDI) MTU, but with tinygrams it works well. A large MTU flow gets mixed in with the small MTU gunk and doesn't push the delay very high. (The links are DS3 and ATM VCs at over DS3). WAN links are often shared by many hundreds of simultaneous active connections. Multiple congested gateways exist. The connections are bandwidth limited elsewhere, but I don't know what the mix is. I've seen reports ranging from 35-75% HTTP in the traffic mix with average HTTP transfer size a bimodal, 3KB (probably HTML) and larger (probably mostly inline images). With an average transfer being 6KB. NNTP, another contributor aveages around 3KB per transfer and SMTP is even smaller. These are off the top of my head. (For better number find a more reliable source:). I'm not sure it will be easy to do a realistic simulation. I've tried to do some larger simulations using ns, quite recently. It seems ns doesn't scale very well. I did a quick hack to replace a linked list in the scheduler and its faster but still too slow. I ended up scaling back the simulation considerably. > I ran TCP with a 500kb end-to-end window over both scenarios. The > plots get much more interesting. Over the "terrestrial" > configuration, the trace shows that the TCP does manage to go at > about 1.5 megabits per second, but it does not get up against the end > of the window. This one TCP has managed to congest the network all > by itself, and then it responded gracefully. It beautifly > illustrates how the 1988 Van Jacobson algorithms manage to find the > right rate to send at (and a workable window to stay within) even > though it cannot get up against the end-to-end TCP window. Over the > satellite, things are not so good. Because the TCP starts out > sending at rate which is twice what the link can handle (within each > slow-start spurt, once-per round-trip time) and it gets no feedback > for a whole RTT (which is one-half second here), many more packets > get lost at the end of the first slow-start episode, and ssthresh > then gets halved multiple times in the ensuing confusion. It turns > out, that with the 500kb end-to-end window, things go just as poorly > over the "satellite" configuration as they would have if the window > had been left at 16kb. Making the window bigger alone did not help. > (Some have pointed out that hand-tuning the end-to-end window for the > particular delay-bandwidth product of this particular link would help > much here. But hand-tuning is not a general solution.) Start out by using a reasonable buffer in the router. Also try using RED. ns supports a version of red, though I've not had good luck tuning it (probably my ignorance - haven't played long enough). We also tried this same sort of 1 TCP flow experiment, plus used 4 and 8 flows with real real hosts, real routers, an 2000 miles of real DS3 ciscuits. Some of the resuts are similar to yours. If buffering was too small we could hand tune to reach a peak but then if the window was raised any, performance dropped considerably. Adding buffering helped. Using RED helped (the version of RED used was later found to be quite badly tuned, but it still helped). > To illustrate the 4k-start proposals that some have been talking > about, I tweaked the ns-1.0 tcp module "window-init" up to a value of > four (packets). This began all slow-start episodes (not just the > connection start) with a 4k window. (This knob was set to one in > previous simulations, so in those simulations slow-start began each > time with a single one-kilobyte packets. This is the normal behavior > of TCP-tahoe.) Over the "satellite", this 4k-start behaved even more > poorly than the "satellite" experiment described in the previous > paragraph. The 4k-starts that happen in the slow start episodes > after the initial loss just lead to more losses which causes even > more reductions in ssthresh, leading eventually to an even lower > congestion window in the sender. SACK probably would have helped > here. This simulation was useful to illustrate at the start how it > could move a small file (of just a few kilobytes) quite a bit faster > over a satellite than the usual one-packet start. I think it also > points out how subtle the effects of tuning these algorithms can be. What's the point if the same change would wreak havoc on the Internet and there are other solutions that would not wreak havoc. > I should also point out that _none_ of my simulations match the > suggested changes that Sally just sent out, so I don't believe any > conclusions can be drawn from my simulations about her proposal in > particular. > > I believe I have said nothing yet in this message that would be news > to others who are actively thinking about and experimenting with TCPs > congestion-control-related algorithms. My purpose of presenting > these simulations at the meeting in Cleveland last week was to make > sure that all present where aware of the details of how the Van > Jacobson 1988 algorithms work, and about their importance to the > health of the Internet. (I showed showed some of the traces that were > in my 1990 SM thesis which showed how badly the 4.3BSD TCP was > behaving in the Internet circa 1988 when the net was apparently > suffering from congestion collapse.) That's certainly a useful exercise. Unfortunately, due to the error in selecting a queue size, you've not represented the way TCP works in a well configured network. > For those who want to see the pictures I had in cleveland... Thanks. btw- it runs on ns2.09a. The D*BW product for T1 and 250 msec is about 180 packets at 4096 packet size. I set the queue in the simulator to 200 and satellite_bigwin ran about 60% faster than satellite_start4k. Curtis ------------------------------ Date: Mon, 10 Feb 1997 00:02:29 -0500 From: Tim Shepard Subject: realistic simulations; router buffer space ; solar system distances >>> It would be real nice if we could realistic when doing simulations. Yes. It would be nice. I don't believe I made any claims for the realism of the simulations I presented in Cleveland. My goal was to demonstrate the algorithms in action so that we could then talk about the different phases of the algorithms with something in mind. I was able to do so in a simulation of a single hop network (with just one congestion point) with a single TCP stream running over it (and no other source of traffic). I expect many things would be different in reality. The router queues would be bigger. There would also be other sources of traffic contributing to the queue length, so when a new TCP started up, it alone would not get all the buffers to itself. So I still believe that slow-start's tendency to send at twice the available rate (within each slow-start spurt) is a source of problems. (I think I could demonstrate that for any amount of buffering one might put in a router, even with an isolated connection.) I think simulations will always fall far short of realism. But they can aid thinking, if the thinking is well-informed of the limitations of the simulation. (Simulations can be very useful. They can reveal important behavior we might not immediately spot in a real network, e.g. ack compression was first found in simulation.) I think that it may often be best to keep raw simulation results to oneself because it's harder to convey all the thinking that went with them than it is to convey the raw results. Raw results can be misleading without all the thinking. But I had spoken publically in Cleveland about these particular simulations, and others were starting to talk about them, so I thought it better to speak up quickly and bare all rather than be misquoted in smaller sound-bite pieces. Regarding the packet queue limits.... What is the right size for the queue limit in a router? Enough to hold a full RTT's worth of data has been suggested. But in general, a router is not aware of what the RTT is for any particular stream of traffic. (The bottleneck link may be in a part of the network managed by someone who is unaware that there are connections involving (multiple?) GEO satellite hops passing through their routers.) I don't believe that all we need to do to allow seemless integration of satellite links into the fabric that makes up the Internet is to quintuple the amount of buffer memory in every IP router in the Internet. Some have also explained to me that putting more buffering in the router is not always advisable because it just adds to delay creep. In a fully loaded mesh network of similar bitrate links with a radius of 20 routers, with local sources at each, each of which had queues which were mostly full, and each had a full RTT's worth of buffers, users would see RTT's as high as 41 times bigger than the actual (sans queuing delay) RTT. No? And now, FYA... someone asked about distance to GEO satellites. An astronomer friend of mine prepared this table of solar-system mean distances for me. It's in units of light-time. 0.021 sec - earth radius 0.14 sec - earth (center) to geosynchronous orbit 1.3 sec - earth to moon 3.2 min - sun to mercury 6.0 min - sun to venus 8.3 min - sun to earth 13 min - sun to mars 23 min - sun to main-belt asteroid 158 Koronis 43 min - sun to jupiter 1.3 hr - sun to saturn 2.7 hr - sun to uranus 4.1 hr - sun to neptune 5.5 hr - sun to pluto Those numbers are one-way delays. RTT's would be double. Earth to planet distances vary with time (they are shortest at opposition), but can be bounded by adding and substracting numbers the table above. We've got a bit of thinking to do if we want the Internet to span the solar system! RTT over a GEO satellite is 4 times 0.14 because the round trip path is up, down, up, down. (Actually it is a little less since if you are on the earth's surface and can see the GEOS above the horizon, then you are closer to it than the center of the earth is.) -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Mon, 10 Feb 1997 01:43:31 -0800 From: Bo Ryu Subject: TCP experiments over ACTS I read Tim Shepard's comments on TCP over satellite wight great interest. I am not in the end2end-interest mailing list *yet* (shame on me :-), so I haven't followed up the discussion closely, but we (Hughes Research Lab, HRL) have done TCP experiments with NASA LeRC over ACTS closely related to Tim's experiments and comments. We wanted to wait until full results are obtained, but realize that putting some results on this list may help us get more valuable feedback for future experiments. One of Tim's comment was that the use of 500KB window size was as bad as 16K. We tried netperf bulk transfer throughput test over ACTS T1 link between HRL and NASA LeRC, with window size varying from 8k to 256k. Theoretically, the use of window size around 100KB would fill up the entire T1 bandwidth over GEO link, and we confirmed that with our experiments. In fact, the throughput stayed at maximum at the window size 256KB, the maximum we tried. We used Solaris 2.5 OS (I think its TCP version is Reno, I will have to check, though.) running on Sun Ultra 1 with large window option on. WE didn't try 500 KB window size, but we assumed that it would give the same maximum throughput. IT might be that the netperf test was long enough that the slow-start period has relatively small impact on the overall throughput, but still, Tim's comment was kind of a surprise to us. We believe that once you pass the slow start stage, it makes little difference whether one uses 128KB or 500 KB as long as the window size is greater than 100KB to fill up the T1 over GEO link. I guess the question is how long does it take to find the right window size with large window size option, which is one if the issues we are currently looking at. IN addition to this simple, straightforward single connection test, we ran multiple-connection tests. We experimented two- and four simultaneous connections with large end-to-end window using netperf, and measured the throughput of individual connection for each case and studied fairness, network volatility., and congestion due to larger-than-standard window size. We don't have a lot of numbers yet, but initial results are quite interesting. Let me briefly explain the four-connection test. Each of four connection equally shared the T1 bandwidth up to 64K end-to-end window (of each connection). Note that in this case the effective window size is 64K * 4 = 256K, much larger than what is required to fill up the full T1 bandwidth over GEO link. HOwever, with 128KB window, we were unable to achieve statistically confident throughput value for any of four connections. (all the other throughput values have 95% confidence interval.) It seems that there is a lot of traffic fluctuation, making each TCP source (connection) very difficult to figure out how much bandwidth is available. The detailed analysis of traces will tell us more about what's going on in that case. Most likely the above results are no surprise to most of you. It is well known that the use of large window will be able to fully fill up high-speed GEO links. (By Ohio U. and Bellcore/NASA, I think.) OUr interest was whether the use of large window will be truly effective when there are multiple users present (and under very severe congestion). We hope our results are interesting to all of you, and hope to receive your opinions and feedback. Bo ps: BTW, please don't forget to cc to ryu@isl.hrl.hac.com when you post your comments regarding our experiments since I am not in the mailing list yet. ------------------------------ Date: Mon, 10 Feb 1997 16:42:00 -0500 From: matthew halsey Subject: re: TCP experiments over ACTS This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-0002CDFF Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Reply Text: (12 lines follow) Bo, I was interested to read this, although I was able to discern your conclusions for multiple connections. Did you say the individual performance was reduced? How did the large window perform with errors? If a large (say 100 kB) window gets hit by an error, does the whole window need to be retransmitted? Or is there a mechanism for determining an individual segment that has been hit (e.g. if SACK is employed?) Any more detail on this would be greatly appreciated. Thanks Matt Original text: (68 lines follow) From RYU@SMTPGATE (Bo Ryu) {ryu%isl.hrl.hac.com.@intelsat1.intelsat.int}, on 10/2/97 4:43 AM: To: END2END-@SMTPGATE {end2end-interest@ISI.EDU}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.TRW.COM} Cc: RYU@SMTPGATE {ryu@isl.hrl.hac.com}, SON@SMTPGATE {son@isl.hrl.hac.com}, DANTE@SMTPGATE {dante@usc.edu}, LYAN@SMTPGATE {lyan@isl.hrl.hac.com}, STHOMAS@SMTPGATE {sthomas@isl.hrl.hac.com} I read Tim Shepard's comments on TCP over satellite wight great interest. I am not in the end2end-interest mailing list *yet* (shame on me :-), so I haven't followed up the discussion closely, but we (Hughes Research Lab, HRL) have done TCP experiments with NASA LeRC over ACTS closely related to Tim's experiments and comments. We wanted to wait until full results are obtained, but realize that putting some results on this list may help us get more valuable feedback for future experiments. One of Tim's comment was that the use of 500KB window size was as bad as 16K. We tried netperf bulk transfer throughput test over ACTS T1 link between HRL and NASA LeRC, with window size varying from 8k to 256k. Theoretically, the use of window size around 100KB would fill up the entire T1 bandwidth over GEO link, and we confirmed that with our experiments. In fact, the throughput stayed at maximum at the window size 256KB, the maximum we tried. We used Solaris 2.5 OS (I think its TCP version is Reno, I will have to check, though.) running on Sun Ultra 1 with large window option on. WE didn't try 500 KB window size, but we assumed that it would give the same maximum throughput. IT might be that the netperf test was long enough that the slow-start period has relatively small impact on the overall throughput, but still, Tim's comment was kind of a surprise to us. We believe that once you pass the slow start stage, it makes little difference whether one uses 128KB or 500 KB as long as the window size is greater than 100KB to fill up the T1 over GEO link. I guess the question is how long does it take to find the right window size with large window size option, which is one if the issues we are currently looking at. IN addition to this simple, straightforward single connection test, we ran multiple-connection tests. We experimented two- and four simultaneous connections with large end-to-end window using netperf, and measured the throughput of individual connection for each case and studied fairness, network volatility., and congestion due to larger-than-standard window size. We don't have a lot of numbers yet, but initial results are quite interesting. Let me briefly explain the four-connection test. Each of four connection equally shared the T1 bandwidth up to 64K end-to-end window (of each connection). Note that in this case the effective window size is 64K * 4 = 256K, much larger than what is required to fill up the full T1 bandwidth over GEO link. HOwever, with 128KB window, we were unable to achieve statistically confident throughput value for any of four connections. (all the other throughput values have 95% confidence interval.) It seems that there is a lot of traffic fluctuation, making each TCP source (connection) very difficult to figure out how much bandwidth is available. The detailed analysis of traces will tell us more about what's going on in that case. Most likely the above results are no surprise to most of you. It is well known that the use of large window will be able to fully fill up high-speed GEO links. (By Ohio U. and Bellcore/NASA, I think.) OUr interest was whether the use of large window will be truly effective when there are multiple users present (and under very severe congestion). We hope our results are interesting to all of you, and hope to receive your opinions and feedback. Bo ps: BTW, please don't forget to cc to ryu@isl.hrl.hac.com when you post your comments regarding our experiments since I am not in the mailing list yet. Use Proportional Font: true Previous From: RYU@SMTPGATE (Bo Ryu) {ryu%isl.hrl.hac.com.@intelsat1.intelsat.int} Previous To: END2END-@SMTPGATE {end2end-interest@ISI.EDU}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.TRW.COM} Original to: END2END-@SMTPGATE {end2end-interest@ISI.EDU}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.TRW.COM} Previous Cc: DANTE@SMTPGATE {dante@usc.edu}, LYAN@SMTPGATE {lyan@isl.hrl.hac.com},RYU@SMTPGATE {ryu@isl.hrl.hac.com}, SON@SMTPGATE {son@isl.hrl.hac.com}, STHOMAS@SMTPGATE {sthomas@isl.hrl.hac.com} Original cc: DANTE@SMTPGATE {dante@usc.edu}, LYAN@SMTPGATE {lyan@isl.hrl.hac.com},RYU@SMTPGATE {ryu@isl.hrl.hac.com}, SON@SMTPGATE {son@isl.hrl.hac.com}, STHOMAS@SMTPGATE {sthomas@isl.hrl.hac.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-0002CDFF Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjTc1CgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQADQCQAgAAAAAAAAAAAAAAAAAA AAAAAAAAT3JpZ2luYWwgdGV4dEEPRgoAAAAAAAAAAAAAfgIDAEEPeAICAAIA AAA/AAIAAQABAIgBAAAAAAAAAgCJAbkNAAAAAAAAOP8AAAAAAACQAQAAAAAA AAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8AAAAA AACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAF8AAQBfAMsAAQDLAIgBAQCIAYkBAgCJAfd/AgDVAfd/AgAeAvd/ AgBnAvd/AgCyAvd/AgD7Avd/AgBIA/d/AgB3A/d/AgB4A/d/AgDHA/d/AgAU BPd/AgBgBPd/AgCrBPd/AgDuBPd/AgA7Bfd/AgCDBfd/AgDKBfd/AgAUBvd/ AgBfBvd/AgCsBvd/AgDjBvd/AgAnB/d/AgBxB/d/AgC9B/d/AgAICPd/AgBI CPd/AgBJCPd/AgCUCPd/AgDaCPd/AgAjCfd/AgBrCfd/AgC4Cfd/AgD6Cfd/ AgA4Cvd/AgCFCvd/AgDOCvd/AgAXC/d/AgA4C/d/AgB8C/d/AgDDC/d/AgAN DPd/AgBSDPd/AgCYDPd/AgDiDPd/AgDjDPd/AgAsDfd/AgB4Dfd/AgDFDfd/ AgANDvd/AgBADvd/AgBBDvd/AgCNDvd/AgCkDvd/AgClDvd/AgCoDvd/AgCp Dvd/AgD2Dvd/AgBBD/d/AgBCD/d/AAAAAAAAAAAAAAAAZAABgAEBAAMBgAQB AAYBgAcBAAkBgAoBAAwBgA0BAA8BgBABABIBgBMBABUAAAAAAAAAAAAAAABk AAGwAQFgAwEQBQHABgFwCAEgCgHQCwGADQEwDwHgEAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-0002CDFF-- ------------------------------ Date: Tue, 11 Feb 1997 02:17:34 -0800 (PST) From: Phil Karn Subject: Re: realistic simulations; router buffer space ; solar system distances >Regarding the packet queue limits.... What is the right size >for the queue limit in a router? Enough to hold a full RTT's >worth of data has been suggested. But in general, a router is I consider this probably the most important issue that should be addressed in the design of internets with large path b*w products. Too much of the TCP research to date has focused on making a small number of infinite TCP streams fairly share their use of a single bottleneck link. The real Internet is very unlike this. You have thousands of little HTTP transactions from thousands of different sources. Each transaction might as well be modeled as a small burst of uncontrolled connectionless packets, because the typical HTTP transaction is only a few packets and the usual TCP congestion control mechanisms hardly have time to get started. In this situation I just don't see much of an alternative to the provision of router buffering sufficient to avoid losing too many packets. The challenge is to figure out how to predict "sufficient". Phil ------------------------------ Date: 11 Feb 1997 10:34:19 -0800 From: "Aaron Falk" Subject: FWD>RE>satellite TCP perfor Mail*Link¨ SMTP FWD>RE>satellite TCP performan tcp-over-satellite'ers- See the note below from Matt Halsey. He's having trouble getting on the email list so contact him directly (halsem@smtpgate.adm.intelsat.int) if you are interested. Thanks, Aaron - -------------------------------------- Date: 2/10/97 2:50 PM From: Eric Travis On Mon, 10 Feb 1997, matthew halsey wrote: > Eric, > > Please consider myself very interested in getting together on this topic. > I work at INTELSAT and have been looking at some new IP protocols recently > and would very much like to be part of a industry-wide team effort. > I have conveyed this desire to Aaron also. > > Is there anyone else I should contact? > > Thanks Matt > Original text: (61 lines follow) > >From TRAVIS@SMTPGATE (Eric Travis) {travis@clark.net}, on 9/2/97 6:29 PM: > To: AARON$FA@SMTPGATE (Aaron Falk) {Aaron_Falk@qmail4.nba.TRW.COM} > Cc: CRAIG@SMTPGATE {craig@aland.bbn.com}, HALSEM@INTELSAT > ("halsem@smtpgate.adm.intelsat.i"), END2END-@SMTPGATE > {end2end-interest@ISI.EDU}, PERRY@SMTPGATE {perry@piermont.com}, > TOUCH@SMTPGATE {touch@ISI.EDU} > > Aaron, > > If you end up on the fence regarding a BOF, I'd like to offer an > additional motivation for trying to hold a BOF (or maybe even > something *less* formal, a "Stellar Lunch", perhaps?); > > There were some generous offers of satellite time for use in testing. > Given the convergence of knowledge, experience (and opinion) that will > be in Memphis, it seems to be that it would be an excellent time to > *begin* the task of designing meaningful experiments, as well as getting > a feel for the potential participants and available resources. Even if > this accomplishes nothing more than identification of participants and > advisors, you are that much ahead. Once the process is started, it > can be continued by a much smaller, more focused group (or groups). > > > [ much text trimmed] > > > > Shepard and Craig Partridge, and others). My personal opinion is that, > unless > > the satellite working group can sort things out to the point where we can > > make some clear, unambiguous statements about what helps TCP over > > satellite and under what conditions these statements apply, we are not > ready > > to support a BOF. I don't want to waste the IETF's time or bring up issues > > > that we (the satellite industry) don't understand. > > > I want to encourage anyone familliar with both satellites and Internet > > protocols to participate in this effort. > > > > Aaron > > ------------------------------ Date: Wed, 12 Feb 1997 08:07:24 -0500 (EST) From: J Patrick Gary 301-286-9539 GSFC Subject: Another news article on TCP over satellites Fyi, another news article on TCP over satellites is available at http://techweb.cmp.com/nc/803/803hrb.html . Pat ------------------------------ Date: Wed, 12 Feb 1997 07:04:00 -0800 From: fred@cisco.com (Fred Baker) Subject: Re: Another news article on TCP over satellites At 8:07 AM 2/12/97, J Patrick Gary 301-286-9539 GSFC wrote: >http://techweb.cmp.com/nc/803/803hrb.html I'm glad to see that Christy picked up on > Partridge says there's a sense of urgency about this TCP work, since > many issues facing satellite use today will confront higher-speed > terrestrial networks in the near future. The issues, he says, need to be > solved within the next two years or so. This is a point I've made many times. TCP's Large Windows Option was built to make co-located Crays run on multi-megabit LANs, not to make PCs work on satellites. ------------------------------ Date: Thu, 13 Feb 1997 12:39:44 -0500 (EST) From: "M. Scott Corson" Subject: Internet Protocols over Satellite (IPoS) Home Page Folks, Just got the web page up for the IPoS working group. http://www.isr.umd.edu/CCDS/IPoS/ Feel free to send me comments, suggested additions, etc. - -Scott ******************************************************************************* M. Scott Corson Institute for Systems Research corson@isr.umd.edu A.V. Williams Bldg. (115) Phone: 301-405-6630 (Rm. 2205 AVW) University of Maryland Fax: 301-314-8586 College Park, MD 20742 ******************************************************************************* ------------------------------ Date: Thu, 13 Feb 1997 11:32:43 -0800 From: Dan Molinelli Subject: Re: Internet Protocols over Satellite (IPoS) Home Page scott the tcp-over-satellite list is digested. send 'index' to: tcp-over-satellite-digest@listserv.trw.com for the past weeks digests. let me know if problems occur. later dan ------------------------------ Date: Thu, 13 Feb 1997 14:58:26 -0500 From: Tim Shepard Subject: Re: Internet Protocols over Satellite (IPoS) Home Page Dan, In Cleveland, we discussed making the archives of the mailing list be available via ftp (just like most of the IETF working groups lists are archived and made available). It is far easier to snarf stuff from an FTP archive than it is to have to send email to some email daemon, and wait, and then have your inbox polluted with bulk transfers. Perhaps you could put them up in an FTP archive at TRW? If not (because of firewalls or whatever), perhaps the mailing list could be moved to UMD where this stuff might be more easily done? See the end2end email archive at ftp://www.isi.edu/end2end for an example. -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Fri, 14 Feb 1997 00:07:02 -0500 From: Curtis Villamizar Subject: Re: realistic simulations; router buffer space ; solar system distances This is a response referencing a number of messages. I tried to wait for this thread to die down and just send once. I'll start a separate thread incuding only end2end-interest on possible ways to better tune TCP for both slow connections and fast where tuning for one doesn't hurt the other. Curtis In message <199702100507.AA02492@venera.isi.edu>, Tim Shepard writes: > > Regarding the packet queue limits.... What is the right size > for the queue limit in a router? Enough to hold a full RTT's > worth of data has been suggested. But in general, a router is > not aware of what the RTT is for any particular stream of > traffic. (The bottleneck link may be in a part of the network > managed by someone who is unaware that there are connections > involving (multiple?) GEO satellite hops passing through their > routers.) I don't believe that all we need to do to allow > seemless integration of satellite links into the fabric that > makes up the Internet is to quintuple the amount of buffer > memory in every IP router in the Internet. Internet service providers are strongly encouraging router (and switch) vendors to put in buffering greater than twice the bandwidth of the interface times the propogation delay of the previders high speed infrastructure. Most US ISPs are interested in DS3, OC3, and OC12 product with a design target of about 70 msec. The request is generally for > 2*D*BW. The routers we use have 8MB per interface and in practice we have not seen this as a contributor to excessive delays but do beleive it is a contributor to reduced loss. If we add to the problem the need to accomodate a T1 at 250 msec, then we are not increasing the buffer requirements. If it is a satelite DS3 or OC3 at 250 msec, then it represents an increase but an expected increase given that the US and Europe are already connected via an OC3 (for which buffering has already been identified as a problem). > Some have also explained to me that putting more buffering in > the router is not always advisable because it just adds to > delay creep. In a fully loaded mesh network of similar bitrate > links with a radius of 20 routers, with local sources at each, > each of which had queues which were mostly full, and each had a > full RTT's worth of buffers, users would see RTT's as high as > 41 times bigger than the actual (sans queuing delay) RTT. No? Routers do have a full RTT of buffering or more and users don't see 41 times the delay. While there are often multiple bottlenecks, there are rarely more than a few. As to whether there is a situation in which this could happen, I suspect it could but in the Internet as architected today, we are not seeing that sort of problem. The delay might be doubled but I don't have any figures to back that up. > And now, FYA... someone asked about distance to GEO satellites. > An astronomer friend of mine prepared this table of solar-system > mean distances for me. It's in units of light-time. > > 0.021 sec - earth radius > 0.14 sec - earth (center) to geosynchronous orbit > 1.3 sec - earth to moon So surface to to geosynchronous orbit would be 0.07 up, 0.07 down, and another 0.07+0.07 for the reverse path, plus lateral distance. An upper bound might be to add 280 msec to the terrestial RTT, which seems to agree with numbers people have mentioned. Thats assuming that there are enough satellites that you don't have to form a very obtuse angle. (ie: one is approximately overhead rather than just about over the horrizon). > 3.2 min - sun to mercury > 6.0 min - sun to venus > 8.3 min - sun to earth > 13 min - sun to mars > 23 min - sun to main-belt asteroid 158 Koronis > 43 min - sun to jupiter > > 1.3 hr - sun to saturn > 2.7 hr - sun to uranus > 4.1 hr - sun to neptune > 5.5 hr - sun to pluto Milo Medin (formerly with NASA) used to remind us of these interesting facts on a regular basis. I strongly suggest you disable the 75 second limit on TCP connection time, the 64 second backoff limit on exponential timer backoff, and do some other tuning before your next trip to pluto. :-) Even if collocation space on Voyager was available, its probably a poor choice for a web hosting facility. > RTT over a GEO satellite is 4 times 0.14 because the round trip > path is up, down, up, down. (Actually it is a little less since > if you are on the earth's surface and can see the GEOS above the > horizon, then you are closer to it than the center of the earth is.) I think that would apply if you are at the center of the earth. ps- I'm sorry if my comments on realism came out seeming to be critical. They weren't intended that way. The buffering parameters were poorly chosen but your effort is appreciated and your work well respected. I'm not sure any of us can speak for what reality is, since there are a variety of network, some private or for restricted uses and some public and inundated. A variety of cases have to be considered before changing host (TCP) or router responses. I can speak for what is in most major ISPs. To be truthful less buffering is available in most routers deployed by ISPs (512KB-2MB per interface being more common, with 512KB being replaced due to observed performance improvement with increased buffering). Thanks for helping calibrate my notion of reality wrt satellite delays. In message <199702100943.BAA04938@isl.hrl.hac.com>, Bo Ryu writes: > > We tried netperf bulk transfer throughput test over ACTS T1 link > between HRL > and NASA LeRC, with window size varying from 8k to > 256k. Theoretically, the ... > > IN addition to this simple, straightforward single connection test, > we ran > multiple-connection tests. We experimented two- and four > simultaneous ... > ... OUr interest was whether > the use of large window will be truly effective when there are > multiple > users present (and under very severe congestion). We ran very very similar tests only we used68 msec of DS3 (nominal 44.2 Mb/s). We found using a single TCP flow that with buffering less than D*BW (ie: less buffering than the end to end delay), if we exceeded the optimal window size by a significant margin, performance dropped substantially. If we had close to (or over) D*BW (delay times bandwidth, if not obvious), we saw far less drop in performance (on the order of 5-10%). RED (Random Early Detection Gateways, Floyd & Jacobson) made further improvements. With multiple flows we found performance did not fall off radically when buffering was , braden@ISI.EDU writes: > ... > > Can a credible case be made that some new frob should be added to > TCP to handle the case of a satellite hop that drops bits in addition to/ > instead of getting congested? Sally Floyd brought up a proposal to start slow start at max(4096, MSS) or max(4096, 2*MSS). I asked that we also consider impact on low speed performance and in particular already congested high speed links dominated by lots of low speed connections (lets not completely kill the already ailing portions of the Internet for a marginal gain in satelite performance). I think there is still an open issue as to whether there is a better way to start slow-start. I'll continue on another thread on end2end-interest. In message <9702101921.aa13792@huey.udel.edu>, Mills@huey.udel.edu writes: > ... > feed from the NWS radar and find the stormclouds before they rain on > my dish and divert traffic. We might even get source routing back in > all the routers by calling it Raincloud Avoidance Protocol (RAP). RAP may have be a possibility. It is almost April. :-) :-) ------------------------------ End of tcp-over-satellite-digest V1 #4 ************************************** tcp-over-satellite-digest Saturday, February 15 1997 Volume 01 : Number 005 In this issues: Re: realistic simulations; router buffer space ; solar system distances FYI: WG ACTION: TCP Implementation (tcpimpl) FYI: Report on 1st Trans-Pacific HDR end-to-end test See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Fri, 14 Feb 1997 11:31:10 -0500 From: Tim Shepard Subject: Re: realistic simulations; router buffer space ; solar system distances > > And now, FYA... someone asked about distance to GEO satellites. > > An astronomer friend of mine prepared this table of solar-system > > mean distances for me. It's in units of light-time. > > > > 0.021 sec - earth radius > > 0.14 sec - earth (center) to geosynchronous orbit > > 1.3 sec - earth to moon > > So surface to to geosynchronous orbit would be 0.07 up, 0.07 down, and > another 0.07+0.07 for the reverse path, plus lateral distance. An > upper bound might be to add 280 msec to the terrestial RTT, which > seems to agree with numbers people have mentioned. > > Thats assuming that there are enough satellites that you don't have to > form a very obtuse angle. (ie: one is approximately overhead rather > than just about over the horrizon). > I have a minor quibble with your 0.07 and 280 numbers (not with your other main points). An earth surface to geosynchronous earth orbit delay ranges from about 0.119 seconds to about 0.14 seconds, assuming the GEOS is above the horizon. The minimum RTT for an up-down-up-down would be 4 * 0.119 = 0.476 seconds, if the GEOS is directly overhead of both stations. A geosynchronous orbit satellite could only be directly overhead if you were on the equator. (If you are at 42 deg north latitude, then the GEOS could be at most 48 deg above the horizon.) If GEOS satellite is sitting on the horizon, then distance to satellite is sqrt((0.14)**2 - (0.021)**2), or about 0.138 seconds. (GEOS sitting on the horizon to surface point is tangent to the earth's surface, so it forms a right angle with an earth radius at that point. Earth center to GEOS is the hypotenuse.) Summary: One-way distance from surface to GEOS is 130 ms plus or minus 10 ms. RTT over GEOS is 520 ms plus or minus 40 ms. Further thoughts: In a real network involving a GEOS interconnected with the Internet, there'd be another 100 ms of RTT in terrestrial delays in some paths. Some situations might involve two hops over GEOSes. So think about 1.04 sec plus or minus 80 ms. Add in another 100 ms of terrestrial delay and we're getting close to 1.25 seconds RTT. We probably don't have to worry about anything longer than that. -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Fri, 14 Feb 1997 20:03:10 -0500 (EST) From: J Patrick Gary 301-286-9539 GSFC Subject: FYI: WG ACTION: TCP Implementation (tcpimpl) >To: ;;@IETF-Announce.gsfc.nasa.gov >Subject: WG ACTION: TCP Implementation (tcpimpl) >Date: Wed, 12 Feb 1997 17:02:02 -0500 >Sender: ietf-announce-request@ietf.org >From: Cynthia Clark > > >A new working group has been formed in the Transport Area >of the IETF. For additional information, contact the Area Directors >or the WG Chair. > >TCP Implementation (tcpimpl) >---------------------------- > > Chair(s): > Steve Alexander > Vern Paxson > > Transport Area Director(s): > Allison Mankin > Allyn Romanow > > Area Advisor > Allyson Romanow > > Mailing lists: > General Discussion:tcp-impl@engr.sgi.com > To Subscribe: majordomo@engr.sgi.com > In Body: "subscribe tcp-impl" in the message body > Archive: ftp://ftp.sgi.com/other/tcp-impl/mail.archive > >Description of Working Group: > >The objective of this group is to decide how to best address known >problems in existing implementations of the current TCP standard(s) and >practices. The overall goal is to improve conditions in the existing >Internet by enhancing the quality of current TCP/IP implementations. It >is hoped that both performance and correctness issues can be resolved >by making implementors aware of the problems and their solutions. In >the long term, it is felt that this will provide a reduction in >unnecessary traffic on the network, the rate of connection failures due >to protocol errors, and load on network servers due to time spent >processing both unsuccessful connections and retransmitted data. This >will help to ensure the stability of the global Internet. > >Examples of detected problems: > >o TCPs that retransmit all unacknowledged data at a single time. > This behavior greatly adds to Internet load, at a time when > the network is already under stress. The combination can > lead to congestion collapse. > >o TCPs that misinitialize the congestion window, leading to > potentially enormous bursts of traffic when new connections > begin. Such burstiness can greatly stress Internet routers. > >In the BOF, it was generally agreed that problems of this class need >to be fixed. > >Scope: > >The scope of this group must be carefully defined in order to ensure >timely progress. To this end, TCP issues that still remain areas of >research are considered out of scope for the WG. For example new >improvements in congestion control algorithms are not within the WG >scope. The WG will solicit input from the End-To-End research group of >the IRTF on questions of whether a TCP implementation issue is >considered research. > >The major objectives of this group will be to : > >Produce a compilation of known problems and their solutions. This will >raise awareness of these issues. > >Determine if any problems found are the result of ambiguities in the >TCP specification. If necessary, produce a document which clarifies >the specification. > >Catalog existing TCP test suites, diagnostic tools, testing >organizations, and procedures that can be used by implementors to >improve their code, and produce a document enumerating them. > > > Goals and Milestones: > > Jan 97 Working group formation. Decide on document editors. > > Apr 97 First Internet-Draft of problems and fixes, and very rough > first draft of catalogue of test suites. > > Apr 97 Define schedule for producing the test suite catalog > > Jul 97 Issue revised Internet-Draft documents. > > Sep 97 Cut-off for determining whether clarification document is > needed. If necessary, have interim meeting to focus effort on > clarification document. > > Dec 97 Submit Internet-Draft of problems and fixes to IESG for > publication as an RFC. > > Dec 97 Submit Internet-Draft of test catalogue to IESG for publication > as an RFC. > > Mar 98 Submit Internet-Draft clarifying RFCs 793, 1122, and 1323 to > IESG for publication as an RFC. > > Mar 98 CLose WG down. > > > > > * * * * * * Hal Folts | Tel (local): (301) 614-5229 (new 8/96) * (24 hours) (703) 905-0367 * | Fax: (301) 614--0267 (new 8/96) * | Email: folts@eos.nasa.gov * | Bldg 32, Room N230C * Distributed Systems and Networks Manager * Earth Science Data and Information System Project (ESDIS) - Code 505 * NASA Goddard Space Flight Center * Greenbelt, MD 20771 USA * * * * * ------------------------------ Date: Fri, 14 Feb 1997 20:29:55 -0500 (EST) From: J Patrick Gary 301-286-9539 GSFC Subject: FYI: Report on 1st Trans-Pacific HDR end-to-end test From: IN%"edelson@seas.gwu.edu" "Burt Edelson" 13-FEB-1997 13:24:16.32 To: IN%"jeffcoatc@comsat.com", IN%"millers@comsat.com", IN%"claude.burgio@intelsat.int", IN%"john.stevenson@intelsat.int" CC: IN%"pat.gary@gsfc.nasa.gov", IN%"helm@seas.gwu.edu", IN%"edelson@seas.gwu.edu", IN%"iasr@seas.gwu.edu", IN%"rdepaula@hq.nasa.gov", IN%"sgoldste@nsf.gov", IN%"bill.santos@telop" Subj: Report on 1st Trans-Pacific HDR end-to-end test Return-path: Received: from gsfc.nasa.gov by dftnic.gsfc.nasa.gov (PMDF V4.3-7 #8) id <01IFDAEKQTQO0009N9@dftnic.gsfc.nasa.gov>; Thu, 13 Feb 1997 13:24:04 EST Received: (from root@localhost) by gsfc.nasa.gov (8.8.5/8.8.4) with X.500 id NAA27502 for pgary@dftnic.gsfc.nasa.gov; Thu, 13 Feb 1997 13:23:53 -0500 (EST) Received: from franklin.seas.gwu.edu (franklin.seas.gwu.edu [128.164.9.2]) by gsfc.nasa.gov (8.8.5/8.8.4) with ESMTP id NAA27477 for ; Thu, 13 Feb 1997 13:23:48 -0500 (EST) Received: from seas.gwu.edu (edelson@felix [128.164.9.3]) by franklin.seas.gwu.edu (v8) with ESMTP id NAA06097; Thu, 13 Feb 1997 13:20:32 -0500 (EST) Received: (from edelson@localhost) by seas.gwu.edu (8.7.1/8.7.1) id NAA01212; Thu, 13 Feb 1997 13:20:28 -0500 (EST) Date: Thu, 13 Feb 1997 13:20:27 -0500 (EST) From: Burt Edelson Subject: Report on 1st Trans-Pacific HDR end-to-end test To: jeffcoatc@comsat.com (Carl Jeffcoat), millers@comsat.com (Susan Miller), claude.burgio@intelsat.int (Claude Burgio), john.stevenson@intelsat.int (John Stevenson) Cc: pat.gary@gsfc.nasa.gov (Pat Gary), helm@seas.gwu.edu (Neil Helm), edelson@seas.gwu.edu (Burt Edelson), iasr@seas.gwu.edu (Institute for Applied Space Research), rdepaula@hq.nasa.gov (Ramon DePaula), sgoldste@nsf.gov (Steve Goldstein), bill.santos@telops.gte.com (Bill Santos) Message-id: <199702131820.NAA01212@seas.gwu.edu> X-Envelope-to: pgary X-Mailer: ELM [version 2.4 PL23] Content-type: text Content-transfer-encoding: 7BIT TRANS-PACIFIC HIGH DEFINITION VIDEO EXPERIMENT Status Report Prepared by Charles Wang, JPL* On February 6, the Trans-Pacific High Definition Video (HDV) Experiment team established an end-to-end ATM (Asynchronous Transfer Mode) link from Sony Pictures High Definition Center (SPHDC), Culver City, California to Sony HQ, Japan. The connection consisted of Pacific Bell CalREN network (from SPHDC to JPL), NASA ACTS satellite (from JPL to Hawaii), GTE/Hawaiian Tel fiber network (between the earth stations of ACTS and Intelsat), Intelsat (from Hawaii to Japan), and NTT fiber network (from Intelsat ES to Sony HQ). ATM Test cells of 40 Mbps (bi-directional) were sent and received at SPHDC in loop back checks to Japan without any error. ATM test cells were sent at 140 Mbps in westward link segments up to the ACTS earth station in Hawaii. Although the ACTS satellite and the ACTS HDR (high data rate) terminals can support OC-3 (155 Mbps) connection, the GTE/Hawaiian Tel fiber network and the QPSK Intelsat modem only support 40 Mbps connections. The round-trip delay was 1.069 seconds. A Mitsubishi High-Definition MPEG-2 codec was used to encode analog production video clips. The video encoded ATM cells were transmitted and received at SPHDC with a loop back at Sony HQ. There were no observable artifacts of the network. The total delay including MPEG-2 encoding/decoding and the network delay of the loop back to Japan was approximately two seconds. The Trans-Pacific High Definition Video Experiment is part of the Japan-US Science, Technology and Space Applications Project (JUSTSAP). Its goal is to demonstrate the role of satellites in the global information broadband network (GIBN) and its capability in providing a link with quality comparable to that of fiber optics. Future runs of the experiment will be used to characterize the end-to-end link, including the effects of satellite channel errors on ATM cells and the resulting video stream, as well as the effects of using Internet file transfer protocols such as FTP over the satellite links with a large delay-bandwidth product. A public demonstration will be held in Sony HQ, Japan during which SPHDC (Culver City) will play a production quality video clip and perform file transfer of video masters through modified FTP in March. Similar demonstration will be held at SPHDC with video source coming from Japan. A demonstration of post-production activities will also be made. The post-production activity will include the transmission of remotely-shot clips to the production center, the merging and processing of studio-produced materials with the remotely-shot clips, and the viewing of the processed materials at the remote site. Trans-Pacific High Definition Video Experiment Team - JPL - ---------------------------------------------------------------- * Charles C. Wang Jet Propulsion Laboratory Email: Charles.C.Wang@jpl.nasa.gov 4800 Oak Grove Drive Phone: (818)354-8041 M/S: 161-260 Fax: (818)393-4643 Pasadena, CA 91109-8099 - ----------------------------------------------------------------- ------------------------------ End of tcp-over-satellite-digest V1 #5 ************************************** tcp-over-satellite-digest Saturday, February 22 1997 Volume 01 : Number 006 In this issues: Article re: TCP over Sat TCP Slow Start Tests HTTP Performance bug in my TCL code Re: bug in my TCL code Re: bug in my TCL code Re: bug in my TCL code Hello All See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Sun, 16 Feb 1997 21:46:33 -0500 (EST) From: Karen Lisa Hansen Subject: Article re: TCP over Sat I found a related article on the Web: http://nsn.net/debate.html Bill Sepmeier, V.P., NSN Network Services "Worldwide TCP/IP Using Satellites -- The Great Debate" ------------------------------ Date: Tue, 18 Feb 1997 14:45:03 -0500 From: Mark Allman Subject: TCP Slow Start Tests As I reported a week or so ago, we have been testing TCP with larger initial windows. We have a draft of the results prepared. It should be noted that this is a DRAFT and we are not completely happy with the testing method yet. We also tested another slow start mechanism, which very well may change slightly as well. This is work in progress. Having said all that, we believe that these tests are a good inidication that these slow start changes will not hurt the network. Furthermore, we wanted to get these tests out since there has been much discussion about this in recent weeks. We would appreciate any comments anyone might have. The note is available at: http://jarok.cs.ohiou.edu/papers under "drafts". allman ------------------------------ Date: Tue, 18 Feb 1997 21:36:39 -0500 (EST) From: Eric Travis Subject: HTTP Performance Folks, I'm taking the liberty of forwarding the following message to this group, as it has some applicability to the problem being addressed here; For those who have seen the original posting of this message, my apologies. The paper referenced below: http://www.w3.org/pub/WWW/Protocols/HTTP/Performance/Pipeline.html was quite an interesting read, although I haven't quite finished digesting it all. There might be some merit in attempting to recreate their high-latency tests using a geo-hop delay (real or emulated). Regards... - ---------- Forwarded message ---------- Date: Tue, 18 Feb 97 16:48:44 -0500 From: jg@zorch.w3.org To: tcp-impl@relay.engr.SGI.COM Subject: Re: HTTP and RFC1122 half duplex close I'm just joining this mailing list, and have been scanning (as opposed to carefully reading) the archive... at 300K, it is a bit much to catch up on. When discussing HTTP behavior, I'd like to remind people that HTTP behavior will be evolving greatly over the next year or two, with the deployment of HTTP/1.1. How long it will take to get rid of HTTP/1.0 and the problems it causes is the next interesting question... HTTP/1.1, if correctly implemented, will dramatically change HTTP's behavior and should end its abuse of TCP. Any suggestions about where to go from here with HTTP should be rethought in this light. Extrapolating much from current data on HTTP should be looked at very carefully; your presumptions are likely all wrong... We've recently put together a paper outlining our experiences and took quite a bit of data of running HTTP/1.1 implementations (both Jigsaw and Apache), which you will find in the paper. (including large amounts of tcpdump traces of our test site; if you are on UNIX and install Tim Shepherd's xplot, you can go from the tabular results in the paper, to the summary of all of our runs of data, to xplots of each run of data we took.) And certainly we saw bugs in TCP (in our particular case, Solaris had problems; Sun has since been kind enough to get us patches...). Thumbnail sketch: HTTP/1.1 over a single, buffered, pipelined TCP connection outperformed HTTP/1.0, with or without keep-alives, using N connections (N was 6 in our tests) under all the tests we ran... Note that we found that servers may need to close the incoming side of its connection; details are in the paper... We also believe that by judicious use of range requests in HTTP, a browser really ought not to be throwing away connections very much at all... Also note that HTTP/1.0 PUT has had various problems dealing with scenarios having to do with resets getting sent causing data to be discarded before being delivered to applications; hopefully HTTP/1.1 gets these right.... Henrik Frystyk Nielsen spent quite a while understanding the problem (and talked to Dave Clark to confirm) all that is going on. Jim Gettys Digital Equipment Corporation Visiting Scientist, W3C Editor of the HTTP/1.1 specification for the HTTP working group... Here's the abstract of the paper.... The paper can be found at: http://www.w3.org/pub/WWW/Protocols/HTTP/Performance/Pipeline.html Abstract We describe our investigation of the effect of persistent connections, pipelining and link level document compression on our client and server HTTP implementations. A simple test setup is used to verify HTTP/1.1's design and understand HTTP/1.1 implementation strategies. We present TCP and real time performance data between the libwww robot and both the Jigsaw and Apache HTTP servers using HTTP/1.0, HTTP/1.1 with persistent connections, HTTP/1.1 with pipelined requests, and HTTP/1.1 with pipelined requests and deflate data compression [22]. We also investigate whether the TCP Nagle algorithm has an effect on HTTP/1.1 performance. While somewhat artificial and possibly overstating the benefits of HTTP/1.1, we believe the tests and results approximate some common behavior seen in browsers. The results confirm that HTTP/1.1 is meeting its major design goals. Our experience has been that implementation details are very important to achieve all of the benefits of HTTP/1.1. For all our tests, a pipelined HTTP/1.1 implementation outperformed HTTP/1.0, even when the HTTP/1.0 implementation used multiple connections in parallel, under all network environments tested. The savings were at least a factor of two, and sometimes as much as than a factor of ten, in terms of packets transmitted. Elapsed time improvement is less dramatic, and strongly depends on your network connection. Note that the savings in network traffic and performance shown in this document are solely due to the effects of pipelining, persistent connections and transport compression. Some data is presented showing further savings possible by the use of CSS1 style sheets [10], and the more compact PNG [20] image representation that are enabled by recent recommendations at higher levels than the base protocol. Time did not allow full end to end data collection on these cases. The results show that HTTP/1.1 and changes in Web content will have dramatic results in Internet and Web performance as HTTP/1.1 and related technologies deploy over the near future. Universal use of style sheets, even without deployment of HTTP/1.1, would cause a very significant reduction in network traffic. This paper does not investigate further performance and network savings enabled by the improved caching facilities provided by the HTTP/1.1 protocol, or by sophisticated use of range requests. ------------------------------ Date: Wed, 19 Feb 1997 16:59:16 -0500 From: Tim Shepard Subject: bug in my TCL code (To the same list of recipients as my message two weeks ago.) Two weeks ago I posted a TCL script (gzipped and uuencoded) at the end of my long message about what I showed at the meeting in Cleveland. Today I discovered a bug in my TCL code that creates the topology for the simulations. The result is that queue lengths over the satellite and terrestrial T1 links were not 6 packets (as I had thought), but rather were the ns-1.* default of 50. Only the queue limits for the 100 Mbps stub net at the sender (included to have a way of getting log entries for the returning acks) were set to 6 in each direction. I do not believe anything I said at the meeting in Cleveland depended critically on this (6 vs. 50), but I know I should speak up quickly lest my bug may creep into other people's simulations and thinking. It is interesting to note that in some cases (like satellite_bigwin) the TCP manages to perform better with 6 packet buffers on the long-haul queue than with 50 packet buffers on the long-haul queue (with only 6 packet buffers on the local fast link in both cases). Again remember this was only a tahoe-era TCP being simulated, so no fast recovery, etc. I'm sorry if this bug caused anyone any confusion. A patch is attached below. -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - *** cleveland.tcl~ Tue Feb 4 15:26:15 1997 - --- cleveland.tcl Wed Feb 19 16:50:59 1997 *************** *** 4,13 **** set h [ns node] set a [ns node] set b [ns node] ! set L [ns_duplex $a $b 1.5Mb 25ms drop-tail] ! set L [ns_duplex $h $a 100Mb 0ms drop-tail] ! [lindex $L 0] set queue-limit 6 ! [lindex $L 1] set queue-limit 6 } proc create_satellite { } { - --- 4,15 ---- set h [ns node] set a [ns node] set b [ns node] ! set L1 [ns_duplex $a $b 1.5Mb 25ms drop-tail] ! set L2 [ns_duplex $h $a 100Mb 0ms drop-tail] ! [lindex $L1 0] set queue-limit 6 ! [lindex $L1 1] set queue-limit 6 ! [lindex $L2 0] set queue-limit 6 ! [lindex $L2 1] set queue-limit 6 } proc create_satellite { } { *************** *** 16,25 **** set h [ns node] set a [ns node] set b [ns node] ! set L [ns_duplex $a $b 1.5Mb 250ms drop-tail] ! set L [ns_duplex $h $a 100Mb 0ms drop-tail] ! [lindex $L 0] set queue-limit 6 ! [lindex $L 1] set queue-limit 6 } proc finish file { - --- 18,29 ---- set h [ns node] set a [ns node] set b [ns node] ! set L1 [ns_duplex $a $b 1.5Mb 250ms drop-tail] ! set L2 [ns_duplex $h $a 100Mb 0ms drop-tail] ! [lindex $L1 0] set queue-limit 6 ! [lindex $L1 1] set queue-limit 6 ! [lindex $L2 0] set queue-limit 6 ! [lindex $L2 1] set queue-limit 6 } proc finish file { ------------------------------ Date: Wed, 19 Feb 1997 23:22:54 -0500 From: Curtis Villamizar Subject: Re: bug in my TCL code In message <199702192209.AA13765@venera.isi.edu>, Tim Shepard writes: > > (To the same list of recipients as my message two weeks ago.) > > Two weeks ago I posted a TCL script (gzipped and uuencoded) at the > end of my long message about what I showed at the meeting in > Cleveland. Today I discovered a bug in my TCL code that creates the > topology for the simulations. The result is that queue lengths over > the satellite and terrestrial T1 links were not 6 packets (as I had > thought), but rather were the ns-1.* default of 50. Only the queue > limits for the 100 Mbps stub net at the sender (included to have a > way of getting log entries for the returning acks) were set to 6 in > each direction. > > I do not believe anything I said at the meeting in Cleveland depended > critically on this (6 vs. 50), but I know I should speak up quickly lest > my bug may creep into other people's simulations and thinking. > > It is interesting to note that in some cases (like satellite_bigwin) the > TCP manages to perform better with 6 packet buffers on the long-haul queue > than with 50 packet buffers on the long-haul queue (with only 6 packet > buffers on the local fast link in both cases). Again remember this was > only a tahoe-era TCP being simulated, so no fast recovery, etc. > > I'm sorry if this bug caused anyone any confusion. > A patch is attached below. > > -Tim Shepard > BBN Systems and Technologies > shep@bbn.com > +1 617 873 2013 Compatibility mode in ns-2 is broken for statements of the form: [lindex $L1 0] set queue-limit $queue_limit So I cheated in using ns-2 and changed the default using: Queue set limit_ $queue_limit and set queue_limit to 6 and changed it to 200. The difference was substantial. Of course, doing it this way I changed all the queues. You should probably see the same thing with ns-1. I also tried changing both links to 50 to see what effect that had (given you comment above). Performance was substantially better with both at 50 than with both at 6. Of course performance is even better with the queues set to 500, since that is the window limit. Curtis btw- another form that works in ns-2 is: set L1 [ns link $a $b] $L1 set queue-limit $queue_limit The challenge of using ns-2 is having to resort to rtfc when things don't work. ------------------------------ Date: Thu, 20 Feb 1997 10:09:58 PST From: Sally Floyd Subject: Re: bug in my TCL code Curtis - >Compatibility mode in ns-2 is broken for statements of the form: ... Unfortunately, ns-2 is still in an experimental, use-at-your-own-risk condition. We (the UCB/ISI/LBL/XEROX ns/VINT projects) plan to validate and extend the test suites for ns-2 as soon as possible, with a priority goal of getting it in shape to be used by the TCP community. We will report on the current status on the ns-2 web page (http://www-mash.cs.berkeley.edu/ns/). - - Sally and Steve (I am not subscribed to the tcp-over-satellite mailing list, so I won't see any ns feedback that is posted only there.) ------------------------------ Date: Thu, 20 Feb 1997 21:44:17 -0500 From: Curtis Villamizar Subject: Re: bug in my TCL code In message <199702201809.KAA08584@owl.ee.lbl.gov>, Sally Floyd writes: > Curtis - > > >Compatibility mode in ns-2 is broken for statements of the form: > ... > > Unfortunately, ns-2 is still in an experimental, use-at-your-own-risk > condition. We (the UCB/ISI/LBL/XEROX ns/VINT projects) plan to validate > and extend the test suites for ns-2 as soon as possible, with a priority > goal of getting it in shape to be used by the TCP community. We will > report on the current status on the ns-2 web page > (http://www-mash.cs.berkeley.edu/ns/). > > - Sally and Steve > > (I am not subscribed to the tcp-over-satellite mailing list, so > I won't see any ns feedback that is posted only there.) Sally and Steve, No complaint intended. ns-2 is great stuff. I'm aware of the status and I'll very happily live with this. ns-2 is cleaner internally than ns-1 which only matters if you need to look under the covers. I was just trying to explain why I saw a different result (changed the amount of buffering and it did something different). I had to do things differently and avoided the bug in the cleaveland.tcl file by accident. Curtis ------------------------------ Date: Thu, 20 Feb 1997 19:54:52 -7 From: "Bill Sepmeier" Subject: Hello All >I recently learned of this list, and have signed up ... I'm VP Engineering >at NSN Network Services in Colorado (nsn.net). We provide Internet by >satellite to several countries around the world, with more coming online >weekly these days. Obviously, we're working on the TCP over satellite issue >in order to improve throughput ... > >I'm also speaking on this topic tomorrow at INTELSAT, and again at Satellite >97 in March. (Actually, this is a re-post, so the talk's been done. A very interesting set of sessions, actually.) > >I haven't seen a lot of traffic here, but hope that I can add something to >whatever discussions take place. > >Please feel free to visit our web site -- let's solve this problem! > >Regards - > >Bill *********************************************** Bill Sepmeier VP Satellite Engineering NSN bill@nsn.net http://nsn.net "It doesn't matter if you're rich or poor .... .... it's always nice to have a little money." Milton Berle ------------------------------ End of tcp-over-satellite-digest V1 #6 ************************************** tcp-over-satellite-digest Saturday, March 8 1997 Volume 01 : Number 007 In this issues: Article See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 3 Mar 1997 09:30:01 -0500 (EST) From: Karen Lisa Hansen Subject: Article FYI- Here is an article re: Telstra deal to provide Internet service to Cambodia via satellite. Does anyone have any tech details eg satellite, bandwidth,etc. on this deal? Copyright 1996 The Cambodia Daily Telstra Signs Deal, Expects Internet Within Three Months By Kimsan Chantara and Kay Johnson The Cambodia Daily, Feb 26, 1997 Telstra, the Australian telecommunications company, signed a contract with the government Tuesday to bring the Internet computer service to Phnom Penh. Within three months, computer owners will be able to dial a local call and browse through the international information and communication network, Telstra country representative Chris Maloy said Tuesday. Now, the only way to access the Internet here is to dial a long-distance call to Singapore or another country with the service, he said. "We expect to have basic service within three months," Maloy said. "I can't guarantee that everyone in Phnom Penh will have perfect service from day one, but if they've got a good land-line connection they will have access like everyone else." Telstra will spend $300,000 to install a satellite Internet connection, Minister of Posts and Telecommunications So Khun said Tuesday at the signing ceremony. The contract is the first signed to bring the Internet to Cambodia. So Khun said his ministry is also negotiating with another company, which he did not name, to provide free Internet access for government offices and schools. The Telstra service will not be free. Maloy said prices have not yet been set but will be comparable to those in other countries. Last year, Posts and Telecommunications Undersecretary of State Koy Sim Sea said consumers could expect to pay about 50,000 riel ($18) per month for eight hours of on-line time. The ministry was poised to sign an exclusive Internet deal with US Sprint last March, but pulled out of the deal later, saying it wanted to encourage competition between at least two providers. Telstra is well-known around Phnom Penh for helping install phone-card booths and providing international long-distance connections, Maloy said. CamNews is for news and announcements about Cambodia ONLY. General posts can be sent to camdisc@cambodia.org, to unsubscribe from the list send mail to camnews-request@cambodia.org with a single line: unsubscribe If you have any questions or problems please feel free to send a message to lists@camboida.org ------------------------------ End of tcp-over-satellite-digest V1 #7 ************************************** tcp-over-satellite-digest Saturday, March 15 1997 Volume 01 : Number 008 In this issues: Catching Up and Test Availability Re: Catching Up and Test Availability TCP over Satellite BOF -- T See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Sun, 9 Mar 1997 15:44:00 +0000 From: "Schram, Chris (Greenwich)" Subject: Catching Up and Test Availability I was not added to the listserv following the January meeting and Aaron Falk suggested I broadcast to the group to ask for some of the relevant correspondence which has been exchanged during the last few weeks. We (PanAmSat) very much appreciated the opportunity to participate in the Cleveland meeting and would like to thank Aaron and Kul for chairing and getting us involved respectively. We also exonerate Dan for the sins of the Press. We continue to have increased interest in a TCP-over-satellite service called "SPOTbytes" (SPOT is a cartoon dog which has been a corporate mascot of sorts for years), with a corresponding increase in questions concerning TCP-over-satellite and asymmetric links. This service is used by overseas ISPs who do not have good terrestrial connectivity or want a dedicated link to the US (see www.panamsat.com, not a promo but for info). It is an extension of sub-T1 links we have done during the last 4 years for .edu and .org users in Latin America (subsidized by NSF) and later for .com users. Information from the meeting and the IPoS site has been helpful, but the underlying practical concern is the effective data rate for typical web traffic to multiple users (normal ISP traffic). Tim Shepard nicely described how large flows of traffic could be ratcheted down to a slow rate, but part of the discussion in Cleveland was whether mixed traffic would have slower throughput after the satellite delays are taken out. I am curious if anyone has since looked into mixed traffic scenarios. A second point covered in Cleveland was assymetric links. This is attractive to remote ISPs where the ratio of traffic from the US and traffic from the ISP runs between 4:1 to 8:1. Dan Shell was going to look at finding a Cisco contact who can address the implementation part of this, and I would also be interested in any other discussion about this during the intervening weeks. Also discussed in Cleveland was the possibility of testing either mixed traffic loads ("typical ISP traffic") or different traffic types over satellite (with and without enhancements), and in particular testing over commercially available satellite link equipment and at C- and Ku-band. We are still interested in supporting this type of testing, and would like to hear from the testing groups to discuss what we have available and how to quantify some of the effects of our BER regime (10-8 to 10-6, as opposed to typically higher rates for ACTS) on throughput. ------------------------------ Date: Sun, 9 Mar 1997 09:05:57 -7 From: "Bill Sepmeier" Subject: Re: Catching Up and Test Availability On 9 Mar 97 at 15:44, Schram, Chris (Greenwich) wrote: > We continue to have increased interest in a TCP-over-satellite service > called "SPOTbytes" (SPOT is a cartoon dog which has been a corporate > mascot of sorts for years), with a corresponding increase in questions > concerning TCP-over-satellite and asymmetric links. This service is used > by overseas ISPs who do not have good terrestrial connectivity or want a > dedicated link to the US (see www.panamsat.com, not a promo but for > info). It is an extension of sub-T1 links we have done during the last 4 > years for .edu and .org users in Latin America (subsidized by NSF) and > later for .com users. Chris, Welcome the the 'net over satellite world. As you may know, we've been delivering Internet via satellite - in one case, PAS-2 - for two years. > Information from the meeting and the IPoS site has been helpful, but the > underlying practical concern is the effective data rate for typical web > traffic to multiple users (normal ISP traffic). Tim Shepard nicely > described how large flows of traffic could be ratcheted down to a slow > rate, but part of the discussion in Cleveland was whether mixed traffic > would have slower throughput after the satellite delays are taken out. I > am curious if anyone has since looked into mixed traffic scenarios. In our experience, satellite TCP latency has no noticeable effect on dial up users, even with low speed links. We are getting ready to release a new router, however, that will all but eliminate TCP latency, which will be a great advantage for single high-speed users, such as corporations using TCP/IP in WAN service. > A second point covered in Cleveland was assymetric links. This is > attractive to remote ISPs where the ratio of traffic from the US and > traffic from the ISP runs between 4:1 to 8:1. Dan Shell was going to > look at finding a Cisco contact who can address the implementation part > of this, and I would also be interested in any other discussion about > this during the intervening weeks. We've had a solution to provide assymetric speeds and multi-drop routing in place for some months. > Also discussed in Cleveland was the possibility of testing either mixed > traffic loads ("typical ISP traffic") or different traffic types over > satellite (with and without enhancements), and in particular testing over > commercially available satellite link equipment and at C- and Ku-band. This testing has been conducted throughly be INTELSAT, and was discussed during their INTELSAT Technical Workshop in Washington DC two weeks ago. INTELSAT operates a large frame relay network over their fleet, and is incorporating Internet traffic, as well as telemedicine and other applications, into it now. In fact, the biggest problem we've found in our two years of delivering turnkey ISP services over satellite hasn't been technical, but political. If I had to make my profits from this aspect of our business alone, you'd have a monopoly on the market - small as it is due to licensing contraints, monopoly PTT regulations, etc. Best regards to you, and Mike and the gang in Atlanta - Bill Sepmeier *********************************************** Bill Sepmeier VP Satellite Engineering NSN bill@nsn.net http://nsn.net "It doesn't matter if you're rich or poor .... .... it's always nice to have a little money." Milton Berle ------------------------------ Date: 14 Mar 1997 16:19:03 -0800 From: "Aaron Falk" Subject: TCP over Satellite BOF -- T There will be a BOF on TCP over Satellite at the IETF Meeting in Memphis. The date of the BOF is still TBD but we are trying to schedule it for Thursday, April 10. Since many members of the TIA-SCD-CIS-Internet Protocols over Satellite working group should be in attendance, I am going to schedule a short working group meeting during the IETF as well. We'll probably do it after the BOF to discuss the next steps of the working group based on the BOF outcome. Meeting rooms will be hard to come by before that Friday, so you'll have to bear with me in trying to find a location to meet. BOF stands for Birds of a Feather. It is type of meeting with two possible purposes. Either it is informational in nature and can consist of a number of presentations or it is the first step involved in creating a working group. The IETF Transport Area Directorate is reviewing my draft BOF description and agenda. After I have received their feedback I will forward it this mail list for discussion and feedback. Those of you who are unfamiliar with the IETF should check out http://www.ietf.org and follow the links to Meetings and also to "Tao of the IETF". I also highly recommend attending the Introduction to the IETF scheduled for Sunday, April 6. Hotel accommodations for the meeting are dicey and if you are planning on attending, you should hurry. Aaron ------------------------------ End of tcp-over-satellite-digest V1 #8 ************************************** tcp-over-satellite-digest Saturday, March 29 1997 Volume 01 : Number 009 In this issues: TCPSAT BOF at Memphis IETF FWD>RE>Revised Thoughts on FWD>RE>Revised Thoughts on See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: 24 Mar 1997 13:47:46 -0800 From: "Aaron Falk" Subject: TCPSAT BOF at Memphis IETF Mail*Link=AE SMTP TCPSAT BOF at Memphis IETF TCP over Satellite BOF (TCPSAT) BOF Chair: Aaron Falk, TRW, Inc. (Aaron.Falk@trw.com) Document Editors: Eric Travis, JPL (travis@clark.net) Curtis Villamizar, ANS (curtis@ans.net) BOF Description: Satellites are being used to extend the Global Interenet geographically and will be more ubiquitous in the next decade with the deployment of several proposed services capable of providing greater than T1 access to individual users anywhere in the world. Yet, satellite links have a unique combination of characteristics that can affect the throughput of TCP traffic. Because of the high-bandwidth delay product, slow start and congestion control algorithms behave much differently when the path includes a satellite link than exclusively terrestrial ones. This BOF will present some of the research that has been done to characterize this behavior and suggest creation of a short lived working group to sort out the issues into those which require more research and those which can be addressed through a combination of implementation and protocol solutions. BOF Goals: Build a consensus to create a working group that will have a very short lifetime (1-2 meetings). This working group will have as its charter to produce an informational draft (ID). This ID shall describe the issues affecting improved TCP throughput over satellite links. Issues that should be addressed are the high delay-bandwidth product and the high propagation delay between routers because they are the primary issues affecting throughput. Other issues identified by the working group may be included as well. The document should identify implementation factors that affect throughput as well as protocol factors. Standards track protocol changes or options as well as acknowledged best practices should be identified where they improve throughput. The ID should have a section on additional requirements and open problems. This section could include pointers to organizations/companies that are willing to donate satellite resources to collaborate in this research. The BOF will meet twice during the Memphis IETF. The first meeting will address the items below and the second meeting will be to start work on the I-D. AGENDA INTRODUCTION (10 min) ISSUES SUMMARY (1-2 presentations) (one hour) Briefing One: Current view of the problem domain Tim Shepard, BBN - This is our environment(s) - These are the symptoms of our problem - We think these are the biggest contributing factors - Identification of similar communities of interest - These are some solutions that have been proposed to date - Rhetorical question (until discussion time): What have we = overlooking? Briefing Two: Phased Solutions Shawn Osterman, Ohio Univ. - We need an "environmentally friendly" solution now - This is what we can do today (mostly architectural) - This is what we think is possible in the near term (IETF timespan of a Working Group) to influence existing and standards track protocols - Ultimately, it would be nice if... - What are we missing/overlooking here? DISCUSSION OF THE PROPOSED CHARTER (45 min) DRAFT CHARTER PROPOSAL The TCP over Satellite Working Group is chartered to produce an informational draft which describes the issues affecting TCP througput over satellite links. It identifies the domains in which each issue = applies; fixes, both protocol and implementation, that ameliorate reduced throughput; and areas for further research. This group will meet at two IETFs and will present the final ID for = review at [the IETF meeting following Germany]. DISCUSSION TOPICS: - Feedback on the problem space o Factors that have been overlooked o Identified factors that might not be relevant - Discussion on proposed solutions & identification of other possibilities o How feasible are they (i.e, are there "server-side" only solutions) o How civic minded are the fixes (do they work and play well with others?) - Discussion on how to demonstrate that a candidate solution(s) will not be detrimental to the larger community o I don't think that a simulation/emulation only approach will cut it. ADJOURN If you would like to participate in online discussion, register for tcp-over-satellite by doing the following: Send mail to: majordomo@listserv.trw.com with the following in the body of the message: subscribe tcp-over-satellite To send mail to the list, send mail to: tcp-over-satellite@listserv.trw.com To get a list of the archives available via email, send mail to: majordomo@listserv.trw.com With the following in the body of the message: index tcp-over-satellite Also, visit: http://www.isr.umd.edu/CCDS/IPoS/ ------------------------------ Date: 26 Mar 1997 13:45:54 -0800 From: "Aaron Falk" Subject: FWD>RE>Revised Thoughts on Mail*Link=AE SMTP FWD>RE>Revised Thoughts on the TCP In message <199703201700.JAA17486@offshore.eng.sun.com>, Allyn Romanow = writes: > > GREAT descr. and goals I agree. > > Introduction (10 min) > > > > Issues Summary (1-2 presentations) (one hour) > > (very precise showing the problems) > > Osterman, OSU/Shepard, BBN > I think you want to include titles of what each will talk about, some > additional detail like that. > Eric has some good ideas about content In order to make simulation conditions increasingly more realistic, some or all of the following should be included. 1. traffic with multiple flows 2. 2-way traffic (actually improves TCP performance in many cases) 3. traffic with feeding branches of differing delay (avoids complete synchronization) 4. multiple bottlenecks 5. short duration flows as well as bulk transfers I'm currently working on code for ns to allow experimentation with these sorts of conditions. I don't mind sharing this code with others doing similar work (but I don't think you want it quite yet). What I'm running now is not focused on long delay paths though tests are easy enough to add. I'll send a preliminary sample. If I get something worthwhile together, someone can present it at the BOF if they like or Osterman and Shepard can borrow a few slides. > > Discussion of the proposed charter (45 min) > > > How about putting the proposed charter here, even a rough first cut. > The form for the charter would follow others which you can get thru > the web page, but I can send you examples if you like. > if you can't do it by deadline, ok, but should go out on the mailing > list for sure. I'd say goals are: 1. Characterize the current performance of TCP (Reno) over long delay paths using existing packet treatment (fifo, tail drop). Also test using TCP Tahoe (lack of fast recovery). 2. Determine how TCP implementations on the horizon (SACK) will behave, in the presense of existing TCP if sparsely deployed and if widely deployed. 3. Determine if anything can be done to the packet treatment, such as alternate queueing or drop strategy, without changing TCP. This would involve changing the router implementation at either side of the satelite link rather than millions of hosts. Test using both TCP Reno and SACK. 4. Determine if any changes can be made to TCP host implementations that will improve long delay path performance *without* adversely affecting TCP in mixed environments or in other environments. Non-goals are: 1. Make modifications to TCP or propose another protocol that will behave better over long delay paths but interoperate poorly with existing TCP or existing TCP congestion avoidance. 2. Intercept IP or TCP and encapsulate in some other protocol. 3. Invent a new link layer with its own congestion avoidance and reliable packet delivery. The reasons for the non-goals are 1) without the ability to interoperate with the rest of the Internet, satelite link TCP/IP solutions aren't viable, 2) intercepting and encapsulating traffic at even T1 speed has proven difficult, though doable, but at higher speed is not currently feasible and does not represent a scalable long term solution to the large delay problem, 3) a new link layer with improved forward error detection or correction is welcome, but should be out of scope for this group and a new link layer that does congestion avoidance and reliable packet delivery is probably not a good idea due to interaction between it's feedback loop and the TCP end-to-end feedback loop. Anyway, that's my 2 cents on the matter. Sorry I can't make the meeting. Wave if you're on the mbone. :-) Curtis - ------------------ RFC822 Header Follows ------------------ Received: by qmgate.trw.com with ADMIN;24 Mar 1997 20:54:40 -0800 Received: from brookfield-ef0.brookfield.ans.net by mailhub1.trw.com; = Mon, 24 Mar 97 20:52:16 -0800 Received: from brookfield.ans.net (localhost.brookfield.ans.net = [127.0.0.1]) by brookfield.ans.net (8.7.5/8.7.3) with ESMTP id XAA08441; = Mon, 24 Mar 1997 23:51:32 -0500 (EST) Message-Id: <199703250451.XAA08441@brookfield.ans.net> To: allyn@pacific-86.Eng.Sun.COM (Allyn Romanow) cc: Aaron.Falk@trw.com, travis@clark.net, mankin@isi.edu, curtis@ans.net Reply-To: curtis@ans.net Subject: Re: Revised Thoughts on the TCP In-reply-to: Your message of "Thu, 20 Mar 1997 09:00:17 PST." <199703201700.JAA17486@offshore.eng.sun.com> Date: Mon, 24 Mar 1997 23:51:31 -0500 From: Curtis Villamizar ------------------------------ Date: Fri, 28 Mar 1997 10:04:43 -0400 From: hkruse1@ohiou.edu (Hans Kruse) Subject: FWD>RE>Revised Thoughts on Some small comments on the draft charter >I'd say goals are: I agree with all of them. >Non-goals are: I agree with these, but only in the context of this working group charter, for the reasons below: >The reasons for the non-goals are 1) without the ability to >interoperate with the rest of the Internet, satelite link TCP/IP >solutions aren't viable, I totallu agree here >2) intercepting and encapsulating traffic at >even T1 speed has proven difficult, though doable, but at higher speed >is not currently feasible and does not represent a scalable long term >solution to the large delay problem, The statement above makes an implict assumption about the reference model used, in this case the location of the satellite link with respect to the user and the global network. I also do not see "spoofing" as a scalable solution, but there are and will be situatations where it solves a particular problem. So, lets keep it out of the WG, maybe declare it non-scalable if there is consensus, but lets not make a value statement beyond that. > 3) a new link layer with improved >forward error detection or correction is welcome, but should be out of >scope for this group and a new link layer that does congestion >avoidance and reliable packet delivery is probably not a good idea due >to interaction between it's feedback loop and the TCP end-to-end >feedback loop. Here I have a bigger problem. I just don't see enough research having been done to declare this "not a good idea". It may turn out to be a very bad idea; but we don't yet know. For example, ATM ABR flow control algorithms are just now being widely tested, and there is not a large body of experience with TCP over these types of links. Having said this, I would agree that this WG is not the place to decide this issue. At the very least this is a research problem that will not progress far enough during the life-time of the WG; it may also be somewhat outside the realm of the IETF since the "tuning" would need to happen at the link layer, not in TCP or IP. Hans Kruse, Associate Professor McClure School of Communication Systems Management, Ohio University 9 S. College Street Athens, OH 45701 614-593-4891 voice, 614-593-4889 fax, hkruse1@ohiou.edu ------------------------------ End of tcp-over-satellite-digest V1 #9 ************************************** tcp-over-satellite-digest Saturday, April 5 1997 Volume 01 : Number 010 In this issues: TCPSAT and Plenary on MBONE Re: TCP over Satellite BOF See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: 3 Apr 1997 09:34:08 -0800 From: "Aaron Falk" Subject: TCPSAT and Plenary on MBONE Both meetings of the TCPSAT BOF and the plenary presentations (including "The Role of Space in Cyberspace") will be available on the MBONE. If you can't make it to the IETF meeting in Memphis next week, check with your system administrator to see if MBONE access is available to you. I was surprised to find that my company had it. The plenary presentation will be made by Dr. Bob Bonemetti; now with Bell Atlantic, formerly with DARPA and the White House Office of Science and Technology Policy. He will be presenting, on behalf of the satellite industry, a description of the future convergence of the Internet and satellite services. I expect it will be an excellent presentation and encourage everyone to watch it if at all possible. IETF multicast information is available at: ftp://ftp.ietf.org/ietf/0mtg-multicast-guide-97apr.txt Aaron Falk ------------------------------ Date: Thu, 03 Apr 1997 12:17:58 -0800 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Re: TCP over Satellite BOF Aaron: My proposed presentation is to help raising some problems and issues. I'll definitely post on the e-mail list after we finish the experiment, but won't be able to do that before this weekend (our scheduled NASA ACTS time only ends this coming Monday). But I can show some interesting results in IETF. We've been doing TCP over ACTS over a month. What we do is to set up three dedicated machines at echo site (Hughes Research Labs and NASA Lewis Res. Cent.) and dump TCP traffic in both directions. The traffic is replay of traces from tcplib (statistics collected from USC, Bell-labs, ... over a month). We also rerun the experiments over a LAN with 50ms delay, as a comparison. We found some interesting results. For example, we found that optimal network utilization (defined as the total amount of useful user data carried through by TCPs, divided by the channel bandwith minus framing, TCP/IP headers, and ACKs.) is much less when run over ACTS. In LAN, we can achieve around 85% utilization using TCP Reno/SACK. The 15% overhead is congestion loss/retransmission. In ACTS case (285ms delay), the maximum utilization of Reno/SACK is only 70%, no matter how much more or less traffic we dump into the network. This shows that longer delay increases the overhead associated with congestion under the current TCP algorithms. I don't know if similar results have been found in the gigabit network cases. How about I only put up 1-2 slides? At this moment I am not sure I will be able to make it Friday as I have pre-scheduled activity back in the company. Thanks, Yongguang ============================================================================== Yongguang Zhang, Ph.D., Research Staff Member (Computer Science) Hughes Research Laboratories, RL-96, 3011 Malibu Canyon Road, Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com phone: 310-317-5147 URL: http://www.wins.hrl.com/people/ygz fax: 310-317-5695 ------------------------------ End of tcp-over-satellite-digest V1 #10 *************************************** tcp-over-satellite-digest Saturday, April 12 1997 Volume 01 : Number 011 In this issues: Workshop Announcement and CFP Re: TCP over Satellite BOF TCPSAT WG charter See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 7 Apr 1997 11:08:42 -0400 (EDT) From: romaniak@woohoo.lerc.nasa.gov (Greg Romaniak) Subject: Workshop Announcement and CFP WORKSHOP ANNOUNCEMENT A one-day workshop entitled "Transport Protocols for Long-Delay Broadband Networks" will be held at Globecom '97, in Phoenix, Arizona, USA, Monday, November 3, 1997 from 8:00 am to 5:00 pm. The location is the Hyatt Regency Hotel, Phoenix Civic Plaza. The workshop is sponsored by the IEEE Communications Society Technical Committee on Gigabit Networking and Technical Committee on Satellite and Space Communications. CALL FOR PAPERS The full performance of emerging high-speed satellite and optical networks is often unavailable to applications running on hosts connected to networks with such extremely large bandwidth-delay characterisitics. Although large bandwidth-delay transport-layer issues have been previously dealt with, and solutions to transport-layer bottlenecks have been demonstrated, these solutions have typically been environment-specific. Furthermore, both the individual and synergistic effects of operating system and protocol changes interacting with the computing and network environment are still not well understood. This workshop will examine the state of the art in deliverable network application and performance possible in modern long-delay high speed networks, identify and quantify performance limiters, and investigate possible solutions. Work toward a unified approach to optimization is of particular interest. The workshop will focus on work with networks with delay-bandwidth products greater than 50 Mb, and delays greater than those typically found in land networks (LEO satellite systems are NOT excluded). Papers on the wide range of application and protocol performance implications are of interest but papers investigating solutions that unify these approaches with those of current gigabit transport (high bw*delay) are of particular interest are especially sought. In particular, papers dealing with the following are solicited: - --a broad focus to TCP optimization, - --networked applications over wideband, high-delay networks, - --applications using native ATM, and - --applications using multiple, geographically separate hosts Experimental results observed on ATM testbeds and satellite systems are of interest, along with prototyping results of integrated systems. Focus should be on current research and implementation issues pertaining to data transmission over high bandwidth-latency paths. - ---------------------------------------------------------------------------- Workshop Chair: Douglas J. Hoder National Aeronautics & Space Administration Lewis Research Center ms 54-6 21000 Brookpark Road, Cleveland, OH 44135 216-433-3438 (voice) 216-433-8705 (fax) dhoder@lerc.nasa.gov Program Chair: Patrick Dowd Department of Defense 9800 Savage Road, MS R53 Ft. Meade, MD 20755 301-688-0347 (voice) 301-688-0588 (fax) p.dowd@ieee.org Workshop Secretary: Gregory W. Romaniak Sterling Software NASA Lewis Research Center ms 142-1 21000 Brookpark Road Cleveland, OH 44135 216-433-8289 (voice) 216-433-8000 (fax) romaniak@lerc.nasa.gov Program Committee: Andrew Campbell, Columbia University, USA Geoff Coulson, Lancaster University, UK Daniel Daly, Bellcore, USA Patrick Dowd, DOD, USA Serge Fdida, Laboratoire MASI, France Dan Glover, NASA Lewis Research Center, USA Vijay Konangi, Cleveland State University, USA Hans Kruse, Ohio University, USA Craig Partridge, BBN, USA Ira Richer, CNRI, USA Isil Sebuktekin, Bellcore, USA Saragur Srinidhi, Fore Systems, India James Sterbenz, GTE Laboratories, USA Joe Touch, ISI, USA Arun Welch, Compuserve, USA Important dates: Paper Submission: July 1, 1997* Notification of Acceptance: August 15, 1997 Camera Ready Copy: September 15, 1997 *Papers will be accepted for submission until July 15, and although every attempt will be made to properly review these submissions, there is no guarantee. Submission Policy: To speed up the reviewing process an electronic version of the paper should be submitted by e-mail to . The preferred formats for electronic submission are: Adobe Acrobat Microsoft Word Postscript If electronic version cannot be generated then four copies should be mailed to the workshop secretary. The submission of double-spaced full papers is preferred, but extended abstracts up to four pages will be considered by the program committee as well. The length of double-spaced papers should not exceed 20 pages, including figures and references. Publication Policy: All submitted papers will be reviewed by at least three members of the program committee. WWW: or ------------------------------ Date: Mon, 07 Apr 1997 13:21:34 -0400 From: Curtis Villamizar Subject: Re: TCP over Satellite BOF In message <199704032017.MAA02843@isl.hrl.hac.com>, Yongguang Zhang writes: > > We've been doing TCP over ACTS over a month. What we do is to set up > three dedicated machines at echo site (Hughes Research Labs and NASA > Lewis Res. Cent.) and dump TCP traffic in both directions. The traffic > is replay of traces from tcplib (statistics collected from USC, Bell-labs, > ... over a month). We also rerun the experiments over a LAN with 50ms > delay, as a comparison. We found some interesting results. For example, > we found that optimal network utilization (defined as the total amount of > useful user data carried through by TCPs, divided by the channel bandwith > minus framing, TCP/IP headers, and ACKs.) is much less when run over ACTS. > In LAN, we can achieve around 85% utilization using TCP Reno/SACK. The 15% > overhead is congestion loss/retransmission. In ACTS case (285ms delay), > the maximum utilization of Reno/SACK is only 70%, no matter how much more > or less traffic we dump into the network. This shows that longer delay > increases the overhead associated with congestion under the current TCP > algorithms. Depending on the router vendor you may be able to make an improvement by changing the queueing parameters in the router on each side of the satelite link. Perhaps we can discuss this offline. > I don't know if similar results have been found in the gigabit network cases. We're not running gigabits, but delay bandwidth products that may be similar to yours (greater if you are running over long delay T1). We've found that with tail drop a buffer size of more than one times the delay bandwidth product is needed. It appears that 2-3 times delay bandwidth product or more might be a good idea but we don't have any sort of measurement or simulation result to share yet. I have some simulation results for 4 and 16 TCP flows over T1 and 250 msec. I still regard these as preliminary results, but here are some observations. If you create a queue of 6-7 times the delay bandwidth product you can get a consistent 100% goodput in steady state with 4 flows. With 16 flows, the point where 100% goodput is acheived in steady state is somewhere over 10 times the delay bandwidth product for smaller MSS (512, 1024), but doesn't occur for 4096 MSS. I'm not advocating 10 times delay bandwidth product. The real problem seems to be that with tail drop, there is a strong tendency to incur multiple drops, though the drops are typically distributed over the flows. The result is the flows get synchronized, in that they all cut back at the same time and begin increasing the traffic in flight. RED has a a great deal of potential in this case. RED introduces an occasional isolated drop at a drop rate somewhat "tuned" to the offerred load and responsiveness to drop. TCP flows can be expected to back off one at a time, avoiding draining the queue. Although I have not been able to complete simulations to verify this, I expect that with RED the very large queue sizes (and very large delays) can be avoided while still achieving 100% goodput through the bottleneck. One problem that you may encounter with satelite links is a delay variability. If there is retransmission at the link layer, TCP RTO may get fooled. If there is FEC at the link layer this will not occur. I don't know which is used (I'm actually rather ignorant of satellite characteristics, but I know how TCP works). In any case the WG, should it form, would be charged with characterizing TCP performance over satellite links, and making suggestions to improve performance so I'm sure your results would be most welcome. [Note that none of the suggestions I've made involve changing TCP in any way. Of course, you'd do better with SACK, but I'd prefer to concentrate on what can be done at the routers on either end of the satellite link as this is the easiest to change. If you use SACK, all the better. There may be other compatible, non-aggresive tweaks to TCP that make further improvements.] Curtis ------------------------------ Date: Thu, 10 Apr 1997 09:40:03 -0700 (PDT) From: tom henderson Subject: TCPSAT WG charter Dear Mr. Braden, Ms. Mankin, and Ms. Romanow, I watched the MBone broadcast of the TCPSAT meeting but unfortunately was unable to participate due to OS problems on my machine here at Berkeley. I would like to comment that I believe that an IETF WG on TCP over satellite would be a good thing if the charter of the WG were made more clear and productive. The current proposal (as I understand it) is to create a repository of issues concerning TCP over satellite. This scope, unfortunately, is not very helpful since it provides weak guidance to implementors and users as to specific action items that would be beneficial. Such information could just as easily be found in the technical literature. A more useful document would be one which recommends to implementors which TCP options should be on standard implementations (e.g., SACK, 4K slow start, etc.), or what kind of end system modifications (e.g., kernel modifications to provide larger socket buffers for long RTT connections) can be employed by users. This type of document does require a consensus building approach and hence would be appropriate for WG status. The performance problems with TCP over satellite are vexing enough to warrant a careful look by the IETF. Tom Henderson UC Berkeley ------------------------------ End of tcp-over-satellite-digest V1 #11 *************************************** tcp-over-satellite-digest Monday, April 14 1997 Volume 01 : Number 012 In this issues: Stray thought on mitigation of latency Karen Hansen's paper Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency draft text TCPSAT BOF Minutes See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Fri, 11 Apr 1997 11:31:19 -0800 From: fred@cisco.com (Fred Baker) Subject: Stray thought on mitigation of latency Dan Shell and I had a thought, which I shared with Aaron and Lisa during the recent meeting, and I wonder what anyone thought about it. For the record, I should tell you that I have discussed it with some of the TCP gurus at Cisco, and they told me >I think the principal problem with this idea is that it breaks the >end-to-end model for everyone who would pass through that part of >this network. > >Some intermediate node is keeping state about a connection, and if >that information is lost, the connection is likely to be silently >damaged. Consider doing a long FTP from Europe and having several >segments worth of data missing from the middle because the spoofer >ACKed them to the server and then died without being able to reliably >send them to the client. There are also some interesting problems in routing; if the route is load shared with another route, the mechanism I'm about to describe wouldn't work properly. It would be better if the application could set up the same structure and manage it - but it would now have to be built into the applications. So what I am proposing is a hack with a limited applicability domain. That said... Ze network picture: satellite / \ / \ / \ host --/-- ??? --/-- router router --/-- ??? --/-- host Presume a TCP implementation in the router that implements SACK and RFC 1323, and which has an augmented API that would allow the application to inspect and set some of TCP's inner variables. Suppose that (somehow) when host left wants to TCP to host right, the routers in the middle intercept the TCP SYN and turn it into three TCP connections: host left <-> router left router left <-> router right router right <-> host right The two router-host TCP connections can now transit traffic in a manner consistent with the limits of the potentially non-wonderful TCPs in the hosts, including slow start, fast recovery, and all that. This gives them an opportunity for something that amounts to a rate-shaped rather than a slow-start approach on the long delay link. The rate is the standard one-window-per-rtt, but since the long hop's rtt is presumably some multiple of the terrestrial rtts, it becomes a series of traffic bursts and ultimately a stream of traffic feeding the link. Suppose that the trans-avian-carrier TCP sets its initial permitted window not to 1 MSS, but to the initial credit from the far side, which somehow set the offered window equal to twice the delay-bandwidth product (twice because we need to fill the pipe for the round-trip interval, not just the one-way-transmission interval)? As data arrives, it immediately heads for the bird, and if that's the only data in question, will use the entire facility. When badness sets in, fast-retransmit/fast-recovery will reduce the permitted window and start slowly increasing it again, adapting the rate to ambient capacity. In addition, SACK will make sure lost data gets resent reasonably quickly. One could also play with window sizes. As noted, the window over the bird wants to initially approximate double the delay-bandwidth product - perhaps with some excess in order to assure continuous transmission. But one might make it window originally = 2*delay*bandwidth max(0, min(window as adjusted by rfc-2001 window modification algorithms, 64K - size of untransmitted data queue in next TCP)) Should the right-hand transmission get interrupted, or the rate turn out to be less than the capacity of the bird, this should apply backpressure across the bird to minimize the buffer utilization in the router. Similary, the session from the sender to the first hop router could be limited in this way: window originally = probably one MSS max(0, min(window as adjusted by rfc-2001 window modification algorithms, 64K - size of untransmitted data queue in next TCP)) There are a couple of possibilties for the router-router TCP - one could imagine there being one TCP chared by all the flows doing this, with packet tags as in RFC 1006 plus a sub-session identifier. There are fairness questions, but this would eliminate the SYN/SYN-ACK interval on the first transmission. One could also imagine a TCP session per flow going through, which would suffer one RTT on setup. There are also a couple of possibilities on how to garner the sessions - one could imagine snooping the stream for all TCP SYN or SYN-ACKS, or using a protocol such as SOCKS to officieally inform the router. I would not want to use SOCKS across the bird, as in set up a control and a data connection before transferring data. I dunno - enough rambling. What-cha think? ------------------------------ Date: Sun, 13 Apr 1997 17:20:58 -0400 From: "Eric A. Bobinsky" Subject: Karen Hansen's paper Kudos to Karen for her excellent paper "Making the Internet Satellite-Ready" that appears in April issue of Business Communication Review. I've just ordered a bunch of reprints for my satellite-skeptical clients. Way to go, Karen! ------------------------------ Date: Sun, 13 Apr 1997 22:01:46 -0400 From: Mark Allman Subject: Re: Stray thought on mitigation of latency Fred- > I dunno - enough rambling. What-cha think? I see a number of problems with this approach... The main reason I don't like it is the reason outlined by your TCP gurus. It changes TCP into a "semi-reliable" protocol (that is, even if the sender receives all the ACKs for a given transfer, the receiver can still end up missing pieces of the data). But, even if we are using high quality cisco routers that rarely crash there are still problems... It seems like this solution does not scale well. Lets use the following example network... HostA ==== RouterA ---- RouterB ==== HostB == is a 10 Mbit/second ethernet -- is a 1.5 Mbit/second T1 satellite channel In this case, HostA can send at roughly 6 times the bottleneck speed. In doing so, RouterA is going to have to buffer a lot of data from a single HostA connection. If there are other hosts and more flows, the router is going to have to buffer even more data. This sounds like it adds a fair amount of complexity in the router, as well. Finally, this only works well if you are sending "one-way data". If this is some sort of request/response operation, like a web page access, this solution buys nothing. The response is still going to take a long time. (Of course, an asymetric setup will cut the delay down). Just my $0.02. allman ------------------------------ Date: Mon, 14 Apr 1997 00:26:13 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency Fred, Since you were brave enough to bring this up, I'll own-up to being remiss in reposting a private discussion on similar proposal made by Matthew Halsey (halsem@smtpgate.adm.intelsat.edu) back in February despite Aaron's request(s). I won't repost Matt's message (nor my response) here, as I haven't asked permission to do so. The concept is bound to be *slightly* controversial :o), thus my initial balking - my project has already done enough unconventional things to what was once TCP and to bring more of them to general attention... We have, however, been weighing the merits of splitting the transport connection for some time. There is also the work done by Bakre and Badrinath (Rutgers) on indirect-tcp that falls into this area; It wasn't clear whether or not you are advocating terminating transport connections at the "routers" or merely spoofing. Although your referring to the intermediate entities as "routers" doesn't justify me in doing so, I'll assume that it is the former because I don't like the latter... Because of this difference in functionality, I'm going to use the term "transport tunnel entity" instead of router in my babbling below; >Dan Shell and I had a thought, which I shared with Aaron and Lisa during >the recent meeting, and I wonder what anyone thought about it. For the >record, I should tell you that I have discussed it with some of the TCP >gurus at Cisco, and they told me >>I think the principal problem with this idea is that it breaks the >>end-to-end model for everyone who would pass through that part of >>this network. >> >>Some intermediate node is keeping state about a connection, and if >>that information is lost, the connection is likely to be silently >>damaged. Consider doing a long FTP from Europe and having several >>segments worth of data missing from the middle because the spoofer >>ACKed them to the server and then died without being able to reliably >>send them to the client. This is a valid source of heartburn (and concern). When we (the folks working on the NASA/DOD SCPS effort) first starting kicking this type of approach around a couple of years ago, we kept gravitating back to this particular issue. Though we never really achieved a comfortable closure on this, *my* feelings (which may or may not be shared by the others involved) were that: o If you don't attempt to make this entirely transparent to the end-systems involved (or at least one of them) then breaking the end-to-end model isn't too vile. Don't web-caches and proxies break the end-to-end model to some extent? o Application level acknowledgements become more important, but they are not a bad idea anyway for many applications. The types of apps that application-level ACKs don;t make sense (such telnet) really don't derive any benefit from this type of "transport tunneling" anyway. I can recommend this for *my* environment because we're not really dealing with shrink-wrapped clients/applications. This is not the case for most service providers. o In my world, the intermediate systems involved will be periodically snapping intermediate state to non-volatile storage - even transport state. This allows us to recover in case of failure. There is still the possibility of data loss, but I think it is semi-acceptable. That's easy for me to say, the owner's of the data might disagree. Check-pointing at the intermediate systems *probably* won't scale well in most environments, so you've still got a greater chance of data loss, but then again, if the intermediate system crashes before reliably relaying data, the transaction *at the application level* will probably fail too, resulting in another attempt at the data push/pull. I'm willing to believe that I'm all wrong about this though... o The creation of "transport tunnels" is a valid approach for transferring data over media that have radically differing properties. For instance, if I am transmitting data over a satellite link, I've gone to considerable effort and expense to get the data to the ground correctly. *I* DO NOT want terrestrial based losses (due to congestion or other causes) to be propagated back across the space-link. Since I can not always engineer the end-to-end network to be congestion free, I'd like the ability to split the transport connection isolating the two (or more) subnetworks. This isn't TCP, but that's OK *for my primary environment*. That disclaimer is important here. As I've yet to build one of these beasts as a (dis)proof-of-concept, this is all conjecture on my part. It is on the list of "things to do", hopefully sooner than later. >There are also some interesting problems in routing; if the route is load >shared with another route, the mechanism I'm about to describe wouldn't >work properly. It would be better if the application could set up the >same structure and manage it - but it would now have to be built into the >applications. So what I am proposing is a hack with a limited >applicability domain. That said... Well, perhaps it is not that restrictive... You describe below a transparent scheme (transparent to not only the applications, but the hosts they run on); Consider the possibility of this being a slightly less transparent proxying scheme. Your rate mechanisms are then regulated by the proxy. In terms of fairness (or possibly prioritization), this should work out. >Ze network picture: > satellite > / \ > / \ > / \ > host --/-- ??? --/-- router router --/-- ??? --/-- host > >Presume a TCP implementation in the router that implements SACK and RFC >1323, and which has an augmented API that would allow the application to >inspect and set some of TCP's inner variables. Suppose that (somehow) >when host left wants to TCP to host right, the routers in the middle >intercept the TCP SYN and turn it into three TCP connections: > > host left <-> router left > router left <-> router right > router right <-> host right > >The two router-host TCP connections can now transit traffic in a manner >consistent with the limits of the potentially non-wonderful TCPs in the >hosts, including slow start, fast recovery, and all that. This gives them >an opportunity for something that amounts to a rate-shaped rather than a >slow-start approach on the long delay link. The rate is the standard >one-window-per-rtt, but since the long hop's rtt is presumably some >multiple of the terrestrial rtts, it becomes a series of traffic bursts >and ultimately a stream of traffic feeding the link. Right, the key to this is that the left or right subnets can *NOT* overload the "trans-avian-carrier" subnet. This is because we're enforcing window-based flow control at both hosts. Send/receive buffer sizes should be set to some reasonable value to impose proper back-pressure on the hosts. Advertised windows at the transport-tunnel entities are paced by the acknowledgements across the "trans-avian-carrier" subnet. All the rate-control does is impose a *maximum* transmission rate across the satellite link/subnet. If there isn't enough available window, data will be sent more slowly and back-pressure will be applied to the sourcing hosts. >Suppose that the trans-avian-carrier TCP sets its initial permitted >window not to 1 MSS, but to the initial credit from the far side, which >somehow set the offered window equal to twice the delay-bandwidth product >(twice because we need to fill the pipe for the round-trip interval, not >just the one-way-transmission interval)? As data arrives, it immediately >heads for the bird, and if that's the only data in question, will use the >entire facility. When badness sets in, fast-retransmit/fast-recovery will >reduce the permitted window and start slowly increasing it again, >adapting the rate to ambient capacity. In addition, SACK will make sure >lost data gets resent reasonably quickly. This all looks good except that my gut-feeling is this is not the best way to handle the badness. I'd redraw the picture *slightly* differently: | long-delay | | / cloud \ | | / \ | | / \ | host --/-- ??? --/-- transport transport --/-- ??? --/-- host tunnel tunnel entity entity & & cache cache | | |<- TCP to best of ->|<- TCP-like ->|<- TCP to best of ->| host's ability transport host's ability tunnel The transport connections are *terminated* at the boundaries of the "long-delay" cloud. We are not spoofing, but we do change/violate the end-to-end semantics. In our scheme, the TCP segments are preserved, but the headers altered (loss tolerant compression when desired/applicable options added to SYN/SYN-ACK, selective retransmission requests added when necessary, etc.). The intent is to avoid reassembly/re-segmentation of the data at the tunnel terminus points. This will lead the tunnel transport to use the same MSS as over the "conventional" subnet. This may, or may not be a problem. Caches are added here at the terminus points to speed-up web-based operations (and avoid unnecessary duplicate "pulls" over the satellite link(s)). If you don't have a cache hit, then your still going to have a long-delay in your path, but slow-starts will be done on the subnets with lower latencies, so things shouldn't be too terrible. Similarly, in the event of congestion in the non-satellite subnets, recovery should be much quicker as we are not propagating these events over the long-delay path. If we assume that we've successfully isolated the "trans-avian-carrier" subnet (it might be more than just a single long-delay hop) from the vagaries of the rest of the overall network, we have also given ourselves some *real* power to make intelligent responses to losses: o As always, our losses will be from one of three sources: - Congestion (TCP's default loss assumption) - Channel Errors (still a possibility of some links) - Link Outages (causes vary depending on a case by case basis) o Because we have isolated traffic on this subnet (we terminate the transport connection at the cloud boundaries) we are free to handle losses in a manner more appropriate to this environment. In other words, we have the freedom to choose our default source of loss. This will depend upon the system involved. - If your satellite channel is generally clean, then losses can be assumed to be due to congestion at routers within our own cloud, so our response should be to throttle back our transmission rate. Since we are dealing with rate-based flow control, we need to decide how much to cut-back our aggregate traffic from the specified max. We might want to have incremental shrinkage/growth or we might find something else is appropriate. In some cases, it is still possible to differentiate errors from congestion if your ground-stations can signal "corruption events" to near-side hosts after they occur. I won't elaborate on this here as it gets involved and doesn't really add much to the discussion. - If your satellite channel can experience error rates in excess of 10E-8, then you probably want to use channel errors as your default source of losses. In this case, we do *not* want to reduce our transmission rate, as doing so will not help. Employing SACK or a SNACK will get you a speedy retransmission of the missing data. When congestion events *do* occur, these can be signaled through use of source-quenches. If the congestion event is on the near-side of the source, this will reduce the response time by almost a RTT. - If you experience link-outages, these too can be signaled and timers can be frozen until the link-outage event passes. In some cases, this can prevent retransmission timers from backing off "to the moon" and help maximize link utilization when the link is restored. >One could also play with window sizes. As noted, the window over the bird >wants to initially approximate double the delay-bandwidth product - >perhaps with some excess in order to assure continuous transmission. But >one might make it > > window originally = 2*delay*bandwidth > max(0, > min(window as adjusted by rfc-2001 window modification > algorithms, 64K - size of untransmitted data queue in next TCP)) > >Should the right-hand transmission get interrupted, or the rate turn out >to be less than the capacity of the bird, this should apply back-pressure >across the bird to minimize the buffer utilization in the router. >Similarly, the session from the sender to the first hop router could be >limited in this way: > > window originally = probably one MSS > max(0, > min(window as adjusted by rfc-2001 window modification > algorithms, 64K - size of untransmitted data queue in next TCP)) If your channel turns out to be "noisy", you are going to want to allow the maximum window size within the isolated cloud to grow to larger than 2*delay*bandwidth. This allows the connection to recover from errors without draining the pipe. I don't have hard numbers to use as a rule of thumb here (yet) as experimentation has shown the intuitive answers to be incorrect. Even though you are using rate-control, the window will provide the required back-pressure. Although the rate-control might allow an aggregate flow of say, 1.5Mbps, but you can only send as much data as you have window. The rate control ends-up being an upper limit, not an absolute rate. >There are a couple of possibilities for the router-router TCP - one could >imagine there being one TCP chared by all the flows doing this, with >packet tags as in RFC 1006 plus a sub-session identifier. There are >fairness questions, but this would eliminate the SYN/SYN-ACK interval on >the first transmission. One could also imagine a TCP session per flow >going through, which would suffer one RTT on setup. Well, another possibility (albeit on the sleazy side of things) is to keep a cache of "canned" transport connections around for immediate use. When a new transport stream arrives (a SYN) we grab an established (but idle)connection for use. We refresh our cache as required. This eliminates the one RTT setup requirement for the non-aggregated traffic. >There are also a couple of possibilities on how to garner the sessions - >one could imagine snooping the stream for all TCP SYN or SYN-ACKS, or >using a protocol such as SOCKS to officially inform the router. I would >not want to use SOCKS across the bird, as in set up a control and a data >connection before transferring data. Snooping has security implications (or rather problems when security is otherwise applied); If the SYNs are encrypted, it won't be possible to intercept them - furthermore, even if this were possible, the newly minted traffic on the far side of the "transport tunnel" would be rejected because they'd fail authentication. Since there needs to be a web of trust for this to work (hosts and near side entity need to trust one another, intermediate entities need to trust each other) this type of scheme really can't be entirely transparent. Go back to the proxy analogy mentioned above; End-systems need to specify their transport proxy, so why not also do a key exchange? This need not be done every time, only when an association doesn't already exist. I'm not sure if there also need to be associations based upon the bigger tuple {host1, intermediate1, intermediate2, host2}, probably. >I dunno - enough rambling. What-cha think? I (obviously) think the concept has some merit; I'll rummage through the notebooks and try to collect a coherent picture of our previous (and current) musing on the matter. If you (or anyone else) wants to pursue this further, please consider me interested and willing (I'll have to see how able). Eric Travis ------------------------------ Date: Mon, 14 Apr 1997 01:11:46 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency Mark, > I see a number of problems with this approach... The main reason I > don't like it is the reason outlined by your TCP gurus. It changes > TCP into a "semi-reliable" protocol (that is, even if the sender > receives all the ACKs for a given transfer, the receiver can still > end up missing pieces of the data). Agreed, but if you change the spoofing to a split transport approach the concern (still valid) becomes limited to the final window's work of data (all intermediate data *must* be delivered to get thus far). Even this problem can be mitigated by preventing a proper close for the sender until the proxying connection(s) successfully close. A FIN should be ACKed, but the connection should not be closed by the proxy (no FIN) until it has received a corresponding FIN from it's peer. This way, it the destination host *does not* ultimately receive all the data (and the FIN), the originator will be stuck in FINWT2 - just as if the corresponding peer had crashed, not an intermediate proxy. Ugly? Yes, but workable. > It seems like this solution does not scale well. Lets use the > following example network... > > HostA ==== RouterA ---- RouterB ==== HostB > > == is a 10 Mbit/second ethernet > -- is a 1.5 Mbit/second T1 satellite channel > > In this case, HostA can send at roughly 6 times the bottleneck > speed. In doing so, RouterA is going to have to buffer a lot of > data from a single HostA connection. If there are other hosts and > more flows, the router is going to have to buffer even more data. > This sounds like it adds a fair amount of complexity in the router, > as well. Why wouldn't host A become window limited? I interpret the scheme to be that Router A need only buffer as much window as it feels comfortable buffering for a particular connection. As it advertizes a window to host A, it is master of it's own fate. Simlarly, since the bandwidth*delay of the host A <--> router A path is bound to be reasonable, the window should be sufficiently reasonable as well. Eric Travis ------------------------------ Date: Mon, 14 Apr 1997 08:46:41 -0400 From: Michael Richardson Subject: Re: Stray thought on mitigation of latency - -----BEGIN PGP SIGNED MESSAGE----- >>>>> "Eric" == Eric Travis writes: Eric> It wasn't clear whether or not you are advocating Eric> terminating transport connections at the "routers" or merely Eric> spoofing. Although your referring to the intermediate I wanted to point out that current "transparent" application layer firewalls practice IP "spoofing". They actually just terminate all TCP connections at the firewall, and go up and down the TCP stack. They don't spoof so much as they merely skip the "packet is local?" test. So, this notion is not so strange. And yes, they *do* suffer from storing state in the "routers", and few of them actually attempt to save the state. Eric> Right, the key to this is that the left or right subnets can Eric> *NOT* overload the "trans-avian-carrier" subnet. This is Eric> because we're enforcing window-based flow control at both Eric> hosts. Send/receive buffer sizes should be set to some Eric> reasonable value to impose proper back-pressure on the Eric> hosts. Advertised windows at the transport-tunnel entities There are, btw, some deadlocks that are present in naive application layer firewall implementations. :!mcr!: | Network security consulting and Michael Richardson | contract programming WWW: mcr@sandelman.ottawa.on.ca. PGP key available. - -----BEGIN PGP SIGNATURE----- Version: 2.6.3ia Charset: latin1 Comment: Processed by Mailcrypt 3.4, an Emacs/PGP interface iQB1AwUBM1InKKZpLyXYhL+BAQEwYQMAs4eyTcQUTkbIymnBfZN0JeXMylRp0nNR X1zaO5fERwdhfoAaUAiYnyLKaxfDju5usBIOd+1IWvr0yjbaBzn7vFuhp8SvenEG Led0c6CRdhWbWWhf32QWWig/pehz72UQ =3IZL - -----END PGP SIGNATURE----- ------------------------------ Date: Mon, 14 Apr 1997 09:55:03 -0400 From: Mark Allman Subject: Re: Stray thought on mitigation of latency > Agreed, but if you change the spoofing to a split transport > approach the concern (still valid) becomes limited to the final > window's work of data (all intermediate data *must* be delivered > to get thus far). Even this problem can be mitigated by > preventing a proper close for the sender until the proxying > connection(s) successfully close. > > A FIN should be ACKed, but the connection should not be closed by > the proxy (no FIN) until it has received a corresponding FIN from > it's peer. This way, it the destination host *does not* > ultimately receive all the data (and the FIN), the originator will > be stuck in FINWT2 - just as if the corresponding peer had > crashed, not an intermediate proxy. After reading your other message, I understand what you are talking about. But, that is not exactly how I read Fred's note. :-) I understand the points you make... I do not have as many doubts (or fears) about your interpretation of the idea. However, I still wonder how much this will buy you. Obviously you control the protocol on the satellite channel, which guarantees that you are using a protocol with appropriate mechanisms (big windows, SACKs, etc.). You are not guaranteed this if you rely on the end point's TCP. But, if the end points are running decent TCP I am not sure that the gain is all that great. So, I would agree that it is an interesting idea that warrents further study... allman ------------------------------ Date: Mon, 14 Apr 1997 14:57:48 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message , Fred Baker writes: > > I dunno - enough rambling. What-cha think? Absolutely terrible idea! This has been proposed before. If you have two routes, the spoofer and real host can end up RSTing each others connections. It is also very hard to scale. IMO - SFQ along the entire path plus adequate buffering and RED (and SFQ) at the border routers is what is needed. Link layer FEC is needed if the satelite link is lossy. Curtis ------------------------------ Date: Mon, 14 Apr 1997 14:59:18 -0400 From: Daniel R Glover Subject: draft text Eric, The NASA Research Announcement on "Satellite/Terrestrial Communications Networks and Architectures" contains some top-level problem description text. Please feel free to use any of it for the Internet-draft. The URL is: http://procurement.nasa.gov/EPS/LeRC/Synopses/NRA-97-LeRC-3/sol.html I am looking forward to getting the draft outline so that I can make some more substantial contributions. R/ Dan /************************** Dan Glover dglover@lerc.nasa.gov NASA Lewis Research Center MS 54-2 Cleveland, OH, USA 44135 (216)433-2847 **************************/ ------------------------------ Date: 14 Apr 1997 13:51:23 -0700 From: "Aaron Falk" Subject: TCPSAT BOF Minutes Mail*Link=AE SMTP TCPSAT BOF Minutes TCP Over Satellite BOF =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D April 10 and 11, 1997 Chair: Aaron Falk, TRW (Aaron.Falk@trw.com) Minutes reported by Mark Allman (mallman@cs.ohiou.edu). April 10, 1997 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Aaron Falk started the meeting with a brief overview of the agenda and meeting purpose. The BOF was called to determine if interest existed for the formation of a working group that would be charged with documenting the issues involved in using TCP on satellite channels. Tim Shepard, BBN (shep@bbn.com) presented a view of the problem that satellite channels present: -the big problem is the delay -generally receive windows are low in the net; an 8K receive window can provide maximum throughput of 128 kilobits/second -this throughput is not bad in some situations (compared to a modem, for example) -but, to use all available bandwidth, we need bigger windows -but, when using big windows, slow start can cause a large amount of packet loss -selective acknowledgments (SACKs) help this problem -the problem is well understood on the receiver side -we need to allow bigger windows with window scaling and PAWS (RFC 1323) -we need to generate selective acknowledgments (RFC 2018) -the problem is not well understood on the sender side; more research is needed -a question was raised about whether RED queuing would help; answer: it might, but we don't know yet Yongguang Zhang, Hughes (ygz@isl.hrl.hac.com) presented some recent research results using the NASA ACTS system: -experimental environment: -ACTS satellite -Reno/SACK FreeBSD machines -workload: tcplib traffic -24 KB windows -avg. file size: 27 KB -compared to a terrestrial network with 50 ms RTT -measured useful traffic/hour -terrestrial channel was able to better utilize the available network capacity when compared to the satellite channel -ongoing experiments: -ATM vs. non-ATM on either side of the satellite channel -other TCP versions (Tahoe, Reno, Vegas, 4K slow start proposal) -testing under delay emulation Shawn Ostermann, Ohio Univ. (ostermann@cs.ohiou.edu) presented some of the current solutions being proposed and studied: -short transfers do not work very well (small number of segments vs. the long RTT and slow start are against us) -long transfers have problems too (the windows are big, which make the congestion control algorithms take a long time to ramp up (either at the beginning of the transfer, or after a loss) -OU built a multiple connection FTP client and server (XFTP) -was able to utilize 96% of available bandwidth -lessons from XFTP: -we need bigger windows -need more buffering in the routers (and maybe alternate queuing, like RED) -detecting and correcting loss -cant let pipe empty because: -takes 4 seconds to slow start to 100 KB -takes 50 seconds to recover from one packet loss (using congestion avoidance) -we need to use fast retransmit and recovery -we need to use SACKs -we need to use MTU discovery so that we are sending as much data as possible in each segment -what we understand: -long connections with no loss do very well -short and medium connections do not do well -short transfers never leave slow start -delayed ACKs make slow start even worse -what might help? (open issues) -bigger initial windows (Floyd) -more aggressive window increase algorithm (Allman, et. al) -the slow start changes above have increased throughput and did not significantly decrease goodput or increase loss in tests at OU -do not perform congestion control if a drop is due to corruption, rather than congestion -use an ssthresh estimator -use RED -goal: -extend TCP while keeping backward compatibility -do not introduce mechanisms that will hurt other types of networks -a question was asked about using indirect-TCP (i.e., breaking the TCP connection into 2 connection at the basestation); Shawn answered that it was an interesting idea that he did not know a lot about Discussion: Aaron started a discussion about whether or not a WG should be created to document the problems of using TCP over satellite links and some possible solutions. -the proposed charter calls for cataloging problems and mitigating mechanisms -out-of-scope: -non-interoperable TCP implementations -substantial changes to the TCP protocol -link-layer changes -research Some comments: -What is the purpose of a working group? Use a mailing list, since the scope is small, the group will be short-lived and the deliverable is only one I-D/RFC -Allyn Romanow (co-area director for transport area) commented that there is already quite a bit of initiative involved in this group and that a staged approach is worthwhile to try out -Matt Mathis: unfocused WGs are unhealthy and this will be a very focused WG, so it is a good experiment -Fred Baker: this type of WG will make the net more useful in the third world and therefore, we should make the effort. Also, these are not just satellite questions but apply to high delay*bandwidth links everywhere. -Allyn conducted a straw poll which was in favor of creating the WG Goal: to deliver a document at the IETF meeting in Washington, DC (12/97). WWW Site: http://www.isr.umd.edu/CCDS/IPoS/ TCP Over Satellite mailing list: To access it send email to: majordomo@listserv.trw.com with the following in the body of the message: subscribe tcp-over-satellite Interim meeting might be necessary. April 11, 1997 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Eric Travis outlined the contents of the draft that will be produced. Purpose: -identify and explain performance problems, including pointers to simulation results, emulation results and experiments conducted over real satellites -identify and explain mechanisms that help overcome the outlined performance problems -categorize engineering problems and research problems -discussion: -we need a document for people running real networks that outlines the problems and solutions for problems encountered in the satellite environment -want to outline various network topologies that contain satellites: -examples given: node-sat-node node-sat-net-node node-net-sat-net-node node-???-sat-???-node -specific issues: -identify and describe: -large delay*bandwidth -long feedback path -asymmetric channels -corruption -??? -provide a problem statement and explanation of possible solutions for each of the above -discussion (what do they provide to each environment outlined above) -references to research results -tag each area as an engineering or research problem -validation of problem/solution -outline interactions between mitigating techniques -outline research areas -provide recommendations -what is available? -what is useful? -what should be applied? Discussion: -outline the constraints, i.e. that the solutions should not have negative impacts on the Internet -we need to provide validation where possible -maybe include something about application or network layer protocols, if they have an impact on TCP Work Schedule: -1st draft target date: 6/30/97 -2nd draft target date: last day to submit drafts before Munich meeting -3rd draft target date: post Munich -4th draft target date: October Aaron asked people doing TCP over satellite research to drop a short note to the mailing list outlining their work. - ------------------ RFC822 Header Follows ------------------ Received: by qmgate.trw.com with ADMIN;13 Apr 1997 19:39:26 -0700 Received: from thoth.cs.ohiou.edu by mailhub1.trw.com; Sun, 13 Apr 97 = 19:38:16 -0700 Received: from thoth.cs.ohiou.edu by thoth.cs.ohiou.edu (8.6.11/1.930630) id WAA03562; Sun, 13 Apr 1997 22:39:20 -0400 Message-Id: <199704140239.WAA03562@thoth.cs.ohiou.edu> To: Aaron.Falk@trw.com From: Mark Allman Reply-To: mallman@oucsace.cs.ohiou.edu Subject: Meeting notes... Organization: Late Night Hackers, Ohio University, Athens, Ohio Song-of-the-Day: Into the Great Wide Open Date: Sun, 13 Apr 1997 22:39:19 -0400 Sender: mallman@thoth.cs.ohiou.edu ------------------------------ End of tcp-over-satellite-digest V1 #12 *************************************** tcp-over-satellite-digest Thursday, April 17 1997 Volume 01 : Number 013 In this issues: Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Outline of work: Keith Scott Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency The next two messages... Memphis Action Items and Schedule of Events Outline Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: draft text Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency my work TCP over Satellite at Ohio State Univ Re: Stray thought on mitigation of latency See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 14 Apr 1997 17:24:01 -0400 From: Tim Shepard Subject: Re: Stray thought on mitigation of latency I just have two comments to throw into this discussion: 1) Should this sort of spoofing or transport termination and regeneration also be done over terrestrial links with high delay-bandwidth products? (Like say on an undersea OC-3c between North America and Australia. Or say on an OC-12c that crosses a few time zone boundries?) 2) What about IPSEC ESP? (If that ever becomes popular, then TCP headers will be hidden behind encryption that the routers will not have keys for.) -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Mon, 14 Apr 1997 14:37:01 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 2:57 PM -0400 4/14/97, Curtis Villamizar wrote: >If you have two routes, the spoofer and real host can end up RSTing >each others connections. > >It is also very hard to scale. absolutely true. >IMO - SFQ along the entire path plus adequate buffering and RED (and >SFQ) at the border routers is what is needed. How does this address latency? =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Mon, 14 Apr 1997 14:33:51 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 1:11 AM -0400 4/14/97, Eric Travis wrote: >Why wouldn't host A become window limited? I interpret the scheme to >be that Router A need only buffer as much window as it feels comfortable >buffering for a particular connection. As it advertizes a window to >host A, it is master of it's own fate. Simlarly, since the bandwidth*delay >of the host A <--> router A path is bound to be reasonable, the window >should be sufficiently reasonable as well. I should expect that it could advertise a relatively small window - 16 or 32K - until it has succeeeded in filling the effective window over the bird, and then reduce (well, not advance) the window. There remain scaling issues (this will fly for tens, perhaps hundreds, of connections, but not thousands or higher). =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Mon, 14 Apr 1997 14:35:07 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 12:26 AM -0400 4/14/97, Eric Travis wrote: >It wasn't clear whether or not you are advocating terminating transport >connections at the "routers" or merely spoofing. I was thinking about spoofing. Actual connections would solve a lot of the concerns, but I am not sure that wouldn't require us to do something explicit, meaning a change to the applications. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Mon, 14 Apr 1997 15:11:02 -0800 From: Keith Scott Subject: Outline of work: Keith Scott Hi. It was suggested at the IETF BOF that people interested in contributing to the tcp-over-satellite Informational Draft give a short description of their work. Here's me: Keith Scott Keith.Scott@jpl.nasa.gov (818) 354-9250 At JPL we're looking at modifications to TCP, and soon, ATM, to assess how well they improve/hurt mobile users accessing the internet over satellites. In our current view, a mobile user can either be a single user (i.e. PDA or laptop with satellite modem), or a "mobile switch" (i.e. aircraft, train, ship). Since we are concerned only with mobile users, we _can_ make some assumptions about where the (first) satellite hop is in the network. We are currently building a testbed to evaluate recent suggestions (RFC 1323, vegas, etc.) as they apply to the mobile channel. At the moment, the testbed will be 2 Linux boxes and an Adtech channel simulator. We will use propagation data that we have taken (TDRSS and ACTS) as input to the simulator to accurately reflect a mobile user communicating with a GEO satellite. If anyone has pointers to other propagation data, especially if it is MEO or LEO, I'd be most interested. Conversely, once things are up and running, we may be able to provide simulated satellite time to interested parties. --keith - ----------------------------------------------------------------------------- Keith Scott kscott@zorba.jpl.nasa.gov Jet Propulsion Laboratory 4800 Oak Grove MS 161-260 (Voice) +1.818.354.9250 Pasadena, CA 91109-8099 (FAX) +1.818.393.4643 - ----------------------------------------------------------------------------- ------------------------------ Date: Mon, 14 Apr 1997 15:21:37 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 5:24 PM -0400 4/14/97, Tim Shepard wrote: >1) Should this sort of spoofing or transport termination and >regeneration also be done over terrestrial links with high >delay-bandwidth products? (Like say on an undersea OC-3c between >North America and Australia. Or say on an OC-12c that crosses a few >time zone boundries?) You heard me say the other day that we might have some spin-off from this work that would have other uses. This is a good example. I think Curtis might argue that there's gotta be a better way, and I might agree with him. But there might be something here to learn. >2) What about IPSEC ESP? (If that ever becomes popular, then TCP >headers will be hidden behind encryption that the routers will not >have keys for.) yup. ESP will make this hard. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Mon, 14 Apr 1997 21:24:45 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message , Fred Baker writes: > At 2:57 PM -0400 4/14/97, Curtis Villamizar wrote: > >If you have two routes, the spoofer and real host can end up RSTing > >each others connections. > > > >It is also very hard to scale. > > absolutely true. > > >IMO - SFQ along the entire path plus adequate buffering and RED (and > >SFQ) at the border routers is what is needed. > > How does this address latency? This doesn't help the initial slow start problem. RED does help keep the pipe close to 100% full. SFQ insures that you don't introduce a drop to a flow that has only begun to open cwnd, dooming it to spend the rest of its lifetime ramping cwnd up one segment per RTT at a time from a low ssthresh value. Even the spoofing gateway suffers that problem. If SFQ is implemented elsewhere along the path such that drops are experience only by flows with a standing queue, the long delay path fairness penalty and multiple bottleneck penalty are erased. The routers along the path have to be very immune to dropping traffic to stable destinations for any of this to work, even when bad things like major route flap happen (route flap involving other routes, not the routes in use). That seems to be a problem today (arbitrary drop routers). Curtis ------------------------------ Date: Mon, 14 Apr 1997 22:32:36 -0400 (EDT) From: Eric Travis Subject: The next two messages... As promised in Memphis, under seperate cover I will be sending two messages to the list: 1. Action Items and Tentative Schedule from Friday's BOF Session 2. Draft document outline as discussed in Memphis; What this is falls somewhere between draft outline and an attempt to collect the related discussion. Upon receiving these messages, please review them and begin/continue your contributions. Eric ------------------------------ Date: Mon, 14 Apr 1997 22:34:20 -0400 (EDT) From: Eric Travis Subject: Memphis Action Items and Schedule of Events As with any good meeting, we managed to generate a tentative schedule and even a pair of action item in Memphis. Action Items: ============= 1. Within the next two weeks (ending April 30), it is requested that people post to the mailing list a brief summary of their related work/research. Pointers to papers and web-sites are encouraged. 2. Within the next two weeks (ending April 30), we are soliciting initial inputs to the Informational RFC. The draft outline discussed in Memphis will be posted in a subsequent message. Initially important elements are content and concepts, not polished prose. These initial contributions will help us shape the scope and depth of the document, so please try and provide what you can. Contributing does *not* sign anyone up for further commitment. During the first week or so of May, individuals will be solicited to take responsibility for major chunks of content. Obviously those who express an interest will be excellent candidates, but don't be surprised if you get asked anyway :o) Saying no is OK, I understand having other commitments. There is a difference between contributors and volunteers; You can contribute without committing. Tentative Schedule ================== April 11, 1997 Begin! April 30, 1997 Initial inputs due to the mailing list (don't stop after April 30!) May 9, 1997 Solicitation of volunteers June 30, 1997 Draft 1 to be sent to mailing list July 31, 1997 Draft 2 to be sent to mailing list August 11-15, 1997 IETF Munich September 1, 1997 Draft 3 to be sent to mailing list/posted to web [Fine scheduling for the period between September and December TBD] October 6, 1997 Draft 4 to be sent to mailing list/posted to web November 3, 1997 Draft 5 to be sent to mailing list/posted to web Late November, 1997 Final Draft December 1997, IETF DC (?) Finish! ------------------------------ Date: Mon, 14 Apr 1997 22:36:31 -0400 (EDT) From: Eric Travis Subject: Outline As promised in my previous message: Below is a semi-conventional outline as discussed at the Friday BOF session in Memphis. Where possible, I've tried to provided editorial comments to reflect the BOF discussion. It's clear that what I've got below is *very* ambitious; I don't really expect us to hit all of this, but I'm willing to try. As inputs come in, we will reshape the scope and outline to correspond with what is reasonable. Please contribute what you can and whenever possible, post these contributions to the mailing list. They *will* be collected. Don't stress (at this point) about polished prose; Flames related to such will not be tolerated - at least not be me. The point is to get the ideas out, the polishing is relatively easy. Contributing doesn't commit you to further contributions. Explicitly volunteering of effort does, however :o) ====================================================== Draft Outline as Presented/Discussed in Memphis: I. Purpose a. Ensure that any and all recommendations resulting from this effort will not adversely impact other traffic on the shared (non-satellite) media. b. Provide a comprehensive document describing the problems (and potential solutions) associated with TCP performance in the satellite environment. c. Identify, explain *and* wherever possible quantify impacts of satellite environment on TCP performance Quantification: Simulation OK, but attempts should be made to validate simulations Emulations better, same desires regarding validation apply Live measurements most desirable, but probably most elusive Bottom line, whatever is available is desirable; Please contribute any relevant results and efforts wherever possible. Quantifications must *not* be conducted in a vendor specific manner. Impacts of base algorithms and common (or proposed) implementation techniques are acceptable (as are pointers to published work covering this area). I'm of the opinion that discussions/categorizations along the lines of 4.3BSD Tahoe, 4.3BSD Reno, 4.4BSD, Net/1, Net/2, Net/3, etc. are OK. I'm willing to revise my opinion here if necessary. In no uncertain terms are proprietary algorithms, implementations, etc. going to be acceptable. d. Categorize the problems according to those that can be addressed through engineering (applying what we already know) and those that require further research. For the latter, this is simply a tagging/identification effort. Anything else is out of scope of this group. e. Identify and explain what mitigation techniques exist and (in general) how to use them. Efforts should concentrate on the more mature, somewhat well understood techniques/options such as those in RFC 1323, RFC 2018, as well as others that have already been discussed within this group (J.C. Hoe's work, the proposed "4K initial cwnd", etc.). I'm not going to attempt to provide a comprehensive listing here. Explanations should cover not only the specific performance problems that a mitigation technique (attempts) to overcome, but also the why, how and where applicable the problems involved. As above, *nothing* proprietary is going to be acceptable. Along those lines, is the "how to use" coverage. This is likely to be difficult and if it proves too difficult then it should fall off the end of the document. Pointers/discussion to things like using socket-options is probably OK, but not things like how to use ndd on Solaris to tweak things. II. Overview of the problem domain A broad overview of the differences in the satellite environment that cause difficulties for TCP. There should be not effort to go into great detail here, that will be done in the next section. I envision a main element of this section to be a categorization of different architectures/topologies (common) to satellite applications. This should prove to be useful for the discussion of specific issues and mitigation techniques in section 3. Furthermore, such a categorization will allow for comparisons/contrasts to be made by those in non-satellite environments. [Please see the note in the next section regarding scope.] III. Specific Issues =================================================== Note regarding scope: ----------------------- There was some good discussion during Friday's session in Memphis as to whether or not we should address the performance impacts that applications and "link" technologies have on TCP in the satellite environment; As an example at the application level, the (unnecessary) interrogative nature of some applications, such as FTP does impact perceived user-level performance in long delay environments. I don't need to provide examples related to link(ish) technologies. Although this is a gray area (for us), I'd like to restate that *I* will be sympathetic to related contributions or pointers on these topics. I won't promise that these items will make it into the final document - we'll have to see how things fall together. Even if the contributions don't make it into this Informational RFC, I'm sure we can prevent them from going to waste. Also, let me be explicit that network layer topic *are* within the scope of this effort. I expect to see lots of content to be from this area. =================================================== a. Identify and describe the problems related to TCP performance in satellite environments. In addition, we wish to provide some validation as to the level to which the problem is understood. This will help sort the areas where engineering is appropriate/possible and those areas that require more research. A first cut at an ordered categorization of problem *areas* is as follows: 1. Large Bandwidth * Delay Product 2. Long Feedback Delay Path 3. Asymmetric Channels 4. Errors Before anyone starts complaining about this, there *are* environments where you get error-rates worse than 10E-8 *after* application of strong FEC; 5. ??? b. Identify, describe and *assess* the applicability of currently available (or proposed) mitigation techniques keyed to their associated problem areas. c. Tag those areas where engineering will not suffice as areas promising for research. d. For each specific issue identified: 1. Problem Statement: 2. Semi-detailed explanation of the problem 3. Mitigation techniques identified 4. Discussion of mitigation techniques a. 1st Order discussion on technique (isolated, no interactions) Assess the benefits and potential pitfalls of this technique keyed to the different topologies provided in section 2. This is will prove to be important. b. If applicable, tag this area as one needing further research. IV. Interacting Mitigations This section should *attempt* to address the potential impacts of mixing the different mitigation techniques identified/discussed above. V. Areas Requiring Further Engineering Pull together those areas identified in Section 3 that can be handled through engineering, but which fall out of scope of this group. V. Identified Research Areas Collect all the tagged research areas identified in as requiring further research. VI. Recommendations ------------------------------ Date: Mon, 14 Apr 1997 22:59:30 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency > I wanted to point out that current "transparent" application layer > firewalls practice IP "spoofing". They actually just terminate all TCP > connections at the firewall, and go up and down the TCP stack. They > don't spoof so much as they merely skip the "packet is local?" test. > So, this notion is not so strange. And yes, they *do* suffer from > storing state in the "routers", and few of them actually attempt to > save the state. The current "transparent" application layer firewalls were probably the source of the scheme we've been considering. Very few ideas end up being orignal (even fewer good) - no judgement made here; My view of where "transport tunnel termini" are applicable probably maps well to where there are/will be firewalls too. In many cases the satellite link involved is owned/leased by an organization to interconnect their users to an (the?) larger internet. I'd imagine the scalability to be identical to that of a similar firewall. I'm not necessarily advocating that such a scheme could be used for arbitary trunking pipes on a much larger backbone network. Eric ------------------------------ Date: Mon, 14 Apr 1997 23:13:36 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency >However, I still wonder how much this will buy you. Obviously you >control the protocol on the satellite channel, which guarantees that >you are using a protocol with appropriate mechanisms (big windows, >SACKs, etc.). You are not guaranteed this if you rely on the end >point's TCP. But, if the end points are running decent TCP I am >not sure that the gain is all that great. So, I would agree that >it is an interesting idea that warrents further study... Even if your hosts are running a decent TCP (big windows, SACK, etc.) I think one thing (not the only one) that such a scheme *might* buy you (and everyone you share media with) is shortening feedback loops for dealing with congestion. Imagine if my feedback loop has a satellite hop and I've got a decent size cwnd built up. An intermediate router becomes congested and tosses one of your packets. By the time you recognize your loss, you've pumped a significant amount more traffic through the congested router, much to the chagrin of everyone else involved. By eliminating the long delay from the feedback path, the loop is not much shorter, and congestion events will be reduced in duration. Will RED help here, of course! I fully hope/expect the world to go RED (what a way to lose a clearance!), but even then, I think there are benefits to be derived. Lots of long-delay, large bandwidth*delay pipes might have unpleasant effects on a larger network... Of course, this is all conjecture on my part right now... Eric ------------------------------ Date: Mon, 14 Apr 1997 23:25:32 -0400 (EDT) From: Eric Travis Subject: Re: draft text > > The NASA Research Announcement on "Satellite/Terrestrial > Communications Networks and Architectures" contains some top-level problem > description text. Please feel free to use any of it for the Internet-draft. > The URL is: > > http://procurement.nasa.gov/EPS/LeRC/Synopses/NRA-97-LeRC-3/sol.html Thanks! It should help alot. > I am looking forward to getting the draft outline so that I can make some > more substantial contributions. I just pushed it out the door. Sorry for the delay. Eric ------------------------------ Date: Mon, 14 Apr 1997 23:41:46 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency > > I was thinking about spoofing. Actual connections would solve a lot of the > concerns, but I am not sure that wouldn't require us to do something > explicit, meaning a change to the applications. > My big concerns with the spoofing approach are: o Once someone inserts some security, you are toast o Sooner than later, someone clever will come along and you'll wish someone had already inserted some security :o) I agree that connections might not be as transparent as spoofing, but as more and more applications are becoming firewall/proxy aware, I'm less concerned - but it would be nice not to have to know you are dealing with an intermediate entity. Eric ------------------------------ Date: Mon, 14 Apr 1997 23:49:39 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency > > I should expect that it could advertise a relatively small window - 16 or > 32K - until it has succeeeded in filling the effective window over the > bird, and then reduce (well, not advance) the window. There remain scaling > issues (this will fly for tens, perhaps hundreds, of connections, but not > thousands or higher). > I imagine the scalability (and applicability) would be about the same as for a firewalling scheme. Neither spoofing or terminated connections will be viable for use on huge trunks, but when applied at corporate network boundaries and/or ISP POPs I think scalability might be workable. ------------------------------ Date: Mon, 14 Apr 1997 23:58:33 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency > > 1) Should this sort of spoofing or transport termination and > regeneration also be done over terrestrial links with high > delay-bandwidth products? (Like say on an undersea OC-3c between > North America and Australia. Or say on an OC-12c that crosses a few > time zone boundries?) Unless the traffic is consists of less than several hundred simultaneous flows (or so) probably not. Even if we were talking about a manageable number of flows, I haven't convinced myself that such rates are sustainable. :o( > > 2) What about IPSEC ESP? (If that ever becomes popular, then TCP > headers will be hidden behind encryption that the routers will not > have keys for.) Ah, don't spoof, terminate those connections. If you still don't trust the intermediate systems, do your own encryption above TCP (in addition to everything else). This is unsatisfying, but... ------------------------------ Date: Tue, 15 Apr 1997 00:16:29 -0400 From: Mark Allman Subject: Re: Stray thought on mitigation of latency > Even if your hosts are running a decent TCP (big windows, SACK, > etc.) I think one thing (not the only one) that such a scheme > *might* buy you (and everyone you share media with) is shortening > feedback loops for dealing with congestion. Note that RED will have to generate some sort of source quench rather than simply dropping a packet to signal congestion or the feedback loop is not shortened. (Do not interpret that as meaning that RED is not desireable if it drops packets rather than sending ECN, as it clearly is very useful). allman ------------------------------ Date: Tue, 15 Apr 1997 00:38:26 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency > > > Even if your hosts are running a decent TCP (big windows, SACK, > > etc.) I think one thing (not the only one) that such a scheme > > *might* buy you (and everyone you share media with) is shortening > > feedback loops for dealing with congestion. > > Note that RED will have to generate some sort of source quench > rather than simply dropping a packet to signal congestion or the > feedback loop is not shortened. (Do not interpret that as meaning > that RED is not desireable if it drops packets rather than sending > ECN, as it clearly is very useful). I think you've got a win even without source-quench. Maybe we are talking about different endpoints defining the feedback loop :o( While I want to see RED with a source quench, you still don't *need* it to shorten the feeback loop if you are terminating your connections and the congestion isn't in the long-delay path. Since all that drivel can be interpreted a number of ways, let me try again: o Source Quench will always shorten the feedback loop. o Even if I don't have source quench available (bummer), by segregating the long delay cloud, you *still* shorten the feedback loop. Congestion notifications (dup-acks) do NOT travel across the long delay cloud unless there are congestion based losses within the long delay clound. Consider: host A <-???->Terminus B <- long-delay ->Terminus B <- ??? -> Host B We've got 3 different transport connections: a) host A <-> Terminus A b) Terminus A <-> Terminus B c) Terminus B <-> host B Now, if there is congestion in the paths of (a or c), then the length of the feedback loop is the length of the path between the host and the appropriate terminus. Perhaps it is not terribly short, but it is still shorter than the path between hosts A and B. In every case, you've reduced the feedback loop and, therefore, the length of the congestion event. Nevertheless, I'd want RED and the *option* of having the routers generate source quenches in all three clouds. Eric ------------------------------ Date: Tue, 15 Apr 1997 00:51:54 -0400 From: Mark Allman Subject: Re: Stray thought on mitigation of latency > I think you've got a win even without source-quench. Maybe we are > talking about different endpoints defining the feedback loop :o( We are... Your feedback loop is different from mine... RED (with ECN) does not buy as much as it does if we did not split the connection. As you point out, if we split the connection we get a shorter feedback loop anyway. It seems to me that in this case RED (with ECN) will not shorten the feedback loop by much. If the TCP connection is not split the feedback loop is much longer and therefore, RED in the gateways before the satellite link (from the sender's point of view) will help alot more. RED + good versions of TCP may make splitting the connection much less attractive. And, even if the connections are split, I agree that RED should be impemented everywhere. But, I have no proof either. :-) allman ------------------------------ Date: Tue, 15 Apr 1997 13:17:05 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 9:24 PM -0400 4/14/97, Curtis Villamizar wrote: >This doesn't help the initial slow start problem. RED does help keep >the pipe close to 100% full. SFQ insures that you don't introduce a >drop to a flow that has only begun to open cwnd, dooming it to spend >the rest of its lifetime ramping cwnd up one segment per RTT at a time >from a low ssthresh value. Even the spoofing gateway suffers that >problem. RED and WFQ/SFQ are ways to address queuing, and if I have anything to do with the router, they will be active. The question here is latency. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Wed, 16 Apr 1997 14:31:21 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message , Eric Travis wr ites: > > > > 1) Should this sort of spoofing or transport termination and > > regeneration also be done over terrestrial links with high > > delay-bandwidth products? (Like say on an undersea OC-3c between > > North America and Australia. Or say on an OC-12c that crosses a few > > time zone boundries?) > > Unless the traffic is consists of less than several hundred simultaneous > flows (or so) probably not. Even if we were talking about a manageable > number of flows, I haven't convinced myself that such rates are > sustainable. :o( At OC3 and OC12 coders need to count machine instructions in the route lookup and elsewhere in the forwarding path. Adding TCP connection state and lookup and header manipulation would make routers at this speed infeasible without adding a whole lot more processing power. Most application gateway style firewalls start to have trouble somewhat below 10 Mb/s. Packet filtering or simple header translation is easier than maintaining sender and receiver full state. There is also a problem if the sender is much faster than the reciever (generally the case for HTTP, which dominated most traffic). If the satellite link is a large window TCP flow, both sides will fill their available TCP buffers (or transfer the entire file to the receive side) even if the receiver is behind a modem and can't do anything near the T1 (or higher) speed of the satellite link. Curtis ------------------------------ Date: Wed, 16 Apr 1997 16:36:19 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message , Fred Baker writes: > At 9:24 PM -0400 4/14/97, Curtis Villamizar wrote: > >This doesn't help the initial slow start problem. RED does help keep > >the pipe close to 100% full. SFQ insures that you don't introduce a > >drop to a flow that has only begun to open cwnd, dooming it to spend > >the rest of its lifetime ramping cwnd up one segment per RTT at a time > >from a low ssthresh value. Even the spoofing gateway suffers that > >problem. > > RED and WFQ/SFQ are ways to address queuing, and if I have anything to do > with the router, they will be active. The question here is latency. My question is "who cares?". You don't improve the time it takes to interrupt in flow if you switch HTTP pages (for example) by making the TCP trasfer go in hops. The application control information still needs to go end2end and the data in flight in the other direction is still in flight. If you are doing a bulk transfer, what you want to accomplish is the highest goodput and the lowest retransmit rate and secondarily not introduce excessive data in flight. If you buy into the claim that HTTP is a semi-interactive bulk transfer then you also want to be able to avoid excessive data in flight so that when application level control indicates that a different file should be given priority, the switch to the new file is not further delayed by the excessive data in flight. Adding proxies to the pipeline can substantially increase the amount of excess data in flight and slow that changeover. You certainly won't be doing much to help purely interactive stuff like telnet, since that too is requires end to end interaction at the application layer. "Latency" is a nebulous goal. Exactly what problem are you trying to solve? Curtis ------------------------------ Date: Wed, 16 Apr 1997 14:18:02 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 2:31 PM -0400 4/16/97, Curtis Villamizar wrote: >At OC3 and OC12 coders need to count machine instructions in the route >lookup and elsewhere in the forwarding path. Adding TCP connection >state and lookup and header manipulation would make routers at this >speed infeasible without adding a whole lot more processing power. at OC3 and OC12, it's beyond counting instructions. At T-3 rates, we are counting memory references, and above that, we're counting clock ticks. That said, the bottom line is the improvement in performance of the application, not the network, and if we need more-better-faster to make the application work well, guess what we're going to be doing. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Wed, 16 Apr 1997 14:24:39 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency >You certainly won't be doing much to help purely interactive stuff >like telnet, since that too is requires end to end interaction at the >application layer. agreed. I trust you also agree that true interactive (telnet) and transaction (X-Windows etc) traffic is rapidly disappearing from the backbone, that the bulk of the unicast traffic is in file transfers of some kind - smtp, nntp, ftp, http, etc. >"Latency" is a nebulous goal. Exactly what problem are you trying to >solve? The biggest single problem that satellites (and other long delay links) introduce for TCP is that TCP moves one window per RTT. A long RTT makes startup - and since the median transfer never leaves the startup phase, tpyical TCP transactions - painful and slow. IT also makes recovery from a drop slow, especially in recovering the high rate of goodput after a non-congestion-related drop. I'm trying to figure out how to introduce a rate based control without changing TCP. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Wed, 16 Apr 1997 16:31:50 -0700 From: Joe Touch Subject: Re: Stray thought on mitigation of latency >The biggest single problem that satellites (and other long delay links) >introduce for TCP is that TCP moves one window per RTT. A long RTT makes >startup - and since the median transfer never leaves the startup phase, >tpyical TCP transactions - painful and slow. IT also makes recovery from a >drop slow, especially in recovering the high rate of goodput after a >non-congestion-related drop. > >I'm trying to figure out how to introduce a rate based control without >changing TCP. We've got an implementation under way for different reasons here at ISI. The original idea was to avoid 'slamming the window shut and restarting' after a data source gap. It would also apply to satellite connections, when combined with TCB reuse (see our internet draft on that issue). Joe ------------------------------ Date: Thu, 17 Apr 1997 10:07:00 -0400 From: matthew halsey Subject: my work Form: Memo Text: (11 lines follow) Amongst other things that I do at INTELSAT, I have been trying to understand the issues associated with TCP via satellite and what 'fixes' are available. I have also recently started using OPNET to model some satellite TCP links although I am far from expert at it yet. I have illustrated some concepts using MS Excel which may be made available for this paper if required. Most of my work is performed for Intelsat Restricted documents so I would need some clearing before offering them. Consider me willing to assist with the doc wherever and however possible. Thanks Matt matthew.halsey@intelsat.int Use Proportional Font: true ------------------------------ Date: Thu, 17 Apr 1997 11:14:24 -0400 (EDT) From: rohit goyal Subject: TCP over Satellite at Ohio State Univ Hi all, We have done some work with TCP over satellite with ATM networks. The most recent ATM Forum contributions can be found at: http://www.cis.ohio-state.edu/~jain/atmf/atm97-0423.ps http://www.cis.ohio-state.edu/~jain/atmf/atm97-0424.ps Or, from http://www.cis.ohio-state.edu/~jain, go to ATM Forum Contritions and look for the following: "ATM Forum/97-0423: Selective Acknowledgements and UBR+ Drop Policies to Improve TCP/UBR Performance over Terrestrial and Satellite Networks, (April 1997)" and "ATM Forum/97-0424: Guaranteed Rate for Improving TCP Performance on UBR+ over Terrestrial and Satellite Networks, (April 1997)" I would appreciate any comments, critiques and suggestions. Thanks, - -Rohit - -- Rohit Goyal, CIS PhD Student (W) (614)-688-4482 395 Dreese Lab, 2015 Neil Ave (H) (614)-488-5785 The Ohio State University Fax: (614)-292-2911 Columbus OH 43210 goyal@cis.ohio-state.edu http://www.cis.ohio-state.edu/~goyal ------------------------------ Date: Thu, 17 Apr 1997 09:45:42 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 4:31 PM -0700 4/16/97, Joe Touch wrote: >It would also apply to >satellite connections, when combined with TCB reuse (see our >internet draft on that issue). TCB re-use (by which I expect you mean "remembering the window for a period of time after a TCP closes") doesn't help on the first shot, and the information degrades in utility quickly. How does this play with email lists and web accesses, where any given access to a network is likely to be the first access in quite a while? =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ End of tcp-over-satellite-digest V1 #13 *************************************** tcp-over-satellite-digest Saturday, April 19 1997 Volume 01 : Number 014 In this issues: Globecom workshop See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Thu, 17 Apr 1997 22:15:00 -0400 From: Arun Welch Subject: Globecom workshop As I mentioned at the meeting, another workshop is being held at GlobeCom. The details can be found at http://mrpink.lerc.nasa.gov/gcomws97.html. ...arun ------------------------------ End of tcp-over-satellite-digest V1 #14 *************************************** tcp-over-satellite-digest Wednesday, April 23 1997 Volume 01 : Number 015 In this issues: RTTM Measurement and effects on BW Re: RTTM Measurement and effects on BW Re: RTTM Measurement and effects on BW Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency t-o-s digest for 4/19th Position Open Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency FW: Notification: Inbound Mail Failure - Address not found FW: Notification: Inbound Mail Failure - Address not found FW: Notification: Inbound Mail Failure - Address not found Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency RE: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency IPoS URL has changed Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 21 Apr 1997 11:06 -0500 From: "Martone Mike" Subject: RTTM Measurement and effects on BW Hi all- How does the manner in which vanilla TCP use RTT measurements cause performance problems for large delay networks? I read RFC-1323 but it was not clear on what the underlying problem was...although I think I find where the solution starts. Which brings me to the next question... Has 1323 implementations for the RTTM problem been implemented ubiquitously? ------------------------------ Date: Mon, 21 Apr 1997 11:50:36 -0400 (EDT) From: Eric Travis Subject: Re: RTTM Measurement and effects on BW Sorry if I end up answering the wrong question(s) here, but: If I'm understanding correctly, the "RTTM problem" with vanilla TCP is that they generally the frequency of measuring the RTT is once per window (and retransmissions are *not* timed); The larger your window, the greater the degree of your undersampling - and the less accurate/valid the perceived RTT. You fix this undersampling by using the timestamp option as defined in RFC-1323. Timestamps do work, at least for us :o) Now, as for "ubiquitous" implementations/deployment, I'm going to take a firm stand and say (with qualification) no; Although I believe that most (if not all) of the latest releases of the Unix(ce) of you choice has 1323 (or soon will), I don't believe that the other "shrink-wrapped" OSes more prevalent as *clients* have timestamps. Then again, these puppies aren't capable of windows > 64K, so that's irrelevant. Furthermore, even though the 1323 is available, it doesn't mean that most servers/clients have had their OSes upgraded to reflect this. This will probably change over time, but it doesn't help you right now. Eric Travis > Hi all- > How does the manner in which vanilla TCP use RTT measurements cause > performance problems for large delay networks? I read RFC-1323 but it was > not clear on what the underlying problem was...although I think I find where > the solution starts. Which brings me to the next question... > Has 1323 implementations for the RTTM problem been implemented > ubiquitously? > ------------------------------ Date: Mon, 21 Apr 1997 10:22:39 -0700 From: Fred Baker Subject: Re: RTTM Measurement and effects on BW At 11:06 AM -0500 4/21/97, Martone Mike wrote: >Hi all- > How does the manner in which vanilla TCP use RTT measurements cause >performance problems for large delay networks? I read RFC-1323 but it was >not clear on what the underlying problem was...although I think I find where >the solution starts. Which brings me to the next question... > Has 1323 implementations for the RTTM problem been implemented >ubiquitously? no. Hence the tcp-implementation-get-your-act-together working group in the IETF... =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "I don't want parole. I'm too busy working on my web site." Charles Manson, 3-27-97 at his parole hearing. ------------------------------ Date: Mon, 21 Apr 1997 16:23:18 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message , Fred Baker writes: > > The biggest single problem that satellites (and other long delay links) > introduce for TCP is that TCP moves one window per RTT. A long RTT makes > startup - and since the median transfer never leaves the startup phase, > tpyical TCP transactions - painful and slow. IT also makes recovery from a > drop slow, especially in recovering the high rate of goodput after a > non-congestion-related drop. > > I'm trying to figure out how to introduce a rate based control without > changing TCP. I think there are simpler alternatives to the problems above. I'm not sure what you mean by painfully slow. Today I can click on a web page and lose the SYN packet and wait 6 seconds for the SYN to be retransmitted without involving a satellite link. The fix to that problem is to not drop packets for new flows (due to first packet cache update drops or overload due to other state setup such as flow switching or tag switching). Full window at T1 and 500 msec is under 100KB. At 1024 MSS, thats 100 segments which takes about 7 RTTs to reach. At 500 msec RTT, it takes 3.5 seconds to reach full speed, during which close to 100KB is sent. For large bulk transfer, this is just fine. For small transfer, such as 16KB (16 1KB segments), 4 RTTs or 2 seconds is required. The startup problem isn't bad at all unless your application is poorly suited for long RTT. An real example, is where a software vendor interested in Internet games and with potential backing, told us that for smooth interaction they needed 15 msec RTT anywhere in the US and if our network couldn't do it they'd find someone who could meet the requirement. Some applications just aren't well suited and we're not trying to solve that. The most serious problem IMO is premature loss in slow start. If an early packet is lost, ssthresh gets set quite low and the linear ramp up to 100 segments will take way too long. The solution here is to make sure this doesn't happen by implementing some form of fair queueing. Until a flow reaches a large window and is providing a significant load it will not see loss. The next most serious problem is multiple losses. This is possible with fair queueing and tail drop. RED helps here. Another big help, with or without SACK, is the fix to fast recovery to only half the window once until the ACK for the last outstanding segment at the time of first dup ACK was detected. Curtis ------------------------------ Date: Mon, 21 Apr 1997 14:16:00 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 4:23 PM -0400 4/21/97, Curtis Villamizar wrote: >I'm not sure what you mean by painfully slow. Today I can click on a >web page and lose the SYN packet and wait 6 seconds for the SYN to be >retransmitted without involving a satellite link. yes, but does it happen on every session? I have a user who contacted me not too long ago complaining that his T-1 satellite link from Alaska to Colorado (which he says has a two second RTT ) is delivering no more than about 2 KBPS actual throughput. IMHO, the most logical reason is that he sends one packet, waits two seconds, sends two packets, waits two seconds, and 50% of the time is now done with his transfer. Yes, dropping the odd SYN packet is painful, and yes, accessing any Netscape site is very painful. But my user is telling me that *everything* is painful. >The fix to that problem is to not drop packets for new flows (due to >first packet cache update drops or overload due to other state setup >such as flow switching or tag switching). You're suggesting that in my user's scenario a packet was dropped? I doubt it. I could imagine one being dropped once in a while during ARP processing, but once that's done, the ARP entry is there for hours, usually weeks. I don't think this is an every session issue. >The startup problem isn't bad at all unless your application is poorly >suited for long RTT. The web? email? The median unidirectional flow in the internet, KC tells us, is on the order of 10 packets. How about people that just want to do what you do, as routinely as you do it? Everything you do isn't large files, I guarantee you. Most of what people want to do on the net is semi-transaction-oriented - click on this link and get the picture of the bright shiny new whats-it and feel this urge to go buy one. Treat it as a public library and do research on the web. Gee whiz, I have 128K to my home because I won't put up with 33.6 eight hours a day and five days a week. With a quarter-second-each-way-delay added in, I'd go completely nuts. I think your perspective would change quite a bit if you put something between your workstation and the net that introduced a quarter second delay each way, an ADTRAN box or a delaying system. >The most serious problem IMO is premature loss in slow start. If an >early packet is lost, ssthresh gets set quite low and the linear ramp >up to 100 segments will take way too long. The solution here is to >make sure this doesn't happen by implementing some form of fair >queueing. And if you'll recall, I have both WFQ and WRED in the router. WFQ is, in fact, being used by my Alaskan customer. His problem is not premature loss, it is latency during slow start. That's what I want to fix. Come on, Curtis, if you don't have something to add that will help solve the problem, can you let the rest of us work on it? =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The surest sign that intelligent life exists elsewhere in the universe is that it has never tried to contact us. Calvin and Hobbes (Bill Watterson) ------------------------------ Date: Mon, 21 Apr 1997 16:31:52 -0700 From: Dan Molinelli Subject: t-o-s digest for 4/19th i'm sending this to the list to verify the t-o-s list addresses. later dan === - ------- Forwarded Message Date: Sat, 19 Apr 1997 01:05:04 -0700 From: owner-tcp-over-satellite-digest@@achtung.sp.TRW.COM (tcp-over-satellit e-digest) To: tcp-over-satellite-digest@achtung.sp.TRW.COM Subject: tcp-over-satellite-digest V1 #14 tcp-over-satellite-digest Saturday, April 19 1997 Volume 01 : Number 014 In this issues: Globecom workshop See the end of the digest for information about tcp-over-satellite-digest. - - ---------------------------------------------------------------------- Date: Thu, 17 Apr 1997 22:15:00 -0400 From: Arun Welch Subject: Globecom workshop As I mentioned at the meeting, another workshop is being held at GlobeCom. The details can be found at http://mrpink.lerc.nasa.gov/gcomws97.html. ...arun - - ------------------------------ End of tcp-over-satellite-digest V1 #14 *************************************** - ------- End of Forwarded Message ------------------------------ Date: Mon, 21 Apr 1997 18:04:00 +0800 From: bill@nsn.net (Bill Sepmeier) Subject: Position Open Please see: http://nsn.net/jobs.html Opening for: Chief Engineer, Satellite Networks Thanks, all .... *********************************************************************** Bill Sepmeier VP Engineering NSN NETWORK SERVICES bill@sepmeier.com bill@nsn.net http://nsn.net "Honey, I Shrunk The Liver" Proposed Title for Don Imus' Movie ------------------------------ Date: Tue, 22 Apr 1997 13:46:30 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message , Fred Baker writes: > At 4:23 PM -0400 4/21/97, Curtis Villamizar wrote: > >I'm not sure what you mean by painfully slow. Today I can click on a > >web page and lose the SYN packet and wait 6 seconds for the SYN to be > >retransmitted without involving a satellite link. > > yes, but does it happen on every session? I have a user who contacted me > not too long ago complaining that his T-1 satellite link from Alaska to > Colorado (which he says has a two second RTT fills the void>) is delivering no more than about 2 KBPS actual throughput. > IMHO, the most logical reason is that he sends one packet, waits two > seconds, sends two packets, waits two seconds, and 50% of the time is now > done with his transfer. > > Yes, dropping the odd SYN packet is painful, and yes, accessing any > Netscape site is very painful. But my user is telling me that *everything* > is painful. If most of the web contact of interest is not in Alaske, put a web proxy that supports HTTP 1.1 on either side of the satellite link. Point one proxy server at the other. You now control the TCP window on either side and if multiple users keep HTTP traffic flowing you can avoid the ramp up. > >The fix to that problem is to not drop packets for new flows (due to > >first packet cache update drops or overload due to other state setup > >such as flow switching or tag switching). > > You're suggesting that in my user's scenario a packet was dropped? I doubt > it. I could imagine one being dropped once in a while during ARP > processing, but once that's done, the ARP entry is there for hours, usually > weeks. I don't think this is an every session issue. > > >The startup problem isn't bad at all unless your application is poorly > >suited for long RTT. > > The web? email? The median unidirectional flow in the internet, KC tells > us, is on the order of 10 packets. How about people that just want to do > what you do, as routinely as you do it? Everything you do isn't large > files, I guarantee you. Most of what people want to do on the net is > semi-transaction-oriented - click on this link and get the picture of the > bright shiny new whats-it and feel this urge to go buy one. Treat it as a > public library and do research on the web. Gee whiz, I have 128K to my home > because I won't put up with 33.6 eight hours a day and five days a week. > With a quarter-second-each-way-delay added in, I'd go completely nuts. Email doesn't matter much. Its not transferred while the user is waiting unless your POP server is on the other side of the satellite which is just plain dumb. I doubt that is the case. For web, the 2 seconds for an HTML file with no images isn't painful. If bandwidth is not an issue on the satellite link but delay is, parrallel image fetch makes a lot of sense. Then a 2KB HTML and N 16KB images would take roughly 5-6 RTTs or 3 seconds. If you do the web proxy thing and the satellite link has available bandwidth, then the continental US side with congested ISP links and servers may become the limit, not the satellite link. Data contributed by Darren Kerr (Cisco) which I think he got from either John Hawkinson (BBN) or Daniel McRob (ANS) shows a distribution of flows: 1 2 4 8 16 32 64 128 256 512 1024 2048 4096+ --- ---- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- 25M 19M 109M 71M 32M 11M 4653K 2118K 882K 423K 276K 145K 51481 25M 38M 436M 568M 512M 352M 298M 271M 226M 217M 283M 297M 211M 0.6% 1.0% 11.7% 15.2% 13.7% 9.4% 8.0% 7.3% 6.1% 5.8% 7.6% 8.0% 5.7% The first colum is the number of flows recorded. The second is the number of packets contributed by flows of that size. The third is the percentage of traffic contributed by flows of that size. There are a lot of small flows but a high percentage of the packets flowing through this router where associated with longer duration flows. > I think your perspective would change quite a bit if you put something > between your workstation and the net that introduced a quarter second delay > each way, an ADTRAN box or a delaying system. I'm fairly sure apache allows you to configure a proxy server that points to another proxy. Why don't you put an apache proxy on either side of the satellite link to Alaska and see if that makes a big difference. I trust you can find a PC running BSD on either end or other suitable host. > >The most serious problem IMO is premature loss in slow start. If an > >early packet is lost, ssthresh gets set quite low and the linear ramp > >up to 100 segments will take way too long. The solution here is to > >make sure this doesn't happen by implementing some form of fair > >queueing. > > And if you'll recall, I have both WFQ and WRED in the router. WFQ is, in > fact, being used by my Alaskan customer. His problem is not premature loss, > it is latency during slow start. That's what I want to fix. It sounds like there is no loss, just delay, so WFQ and RED have no effect. If there is congestion, then short circuiting slow start may not be a real good idea. > Come on, Curtis, if you don't have something to add that will help solve > the problem, can you let the rest of us work on it? Try the HTTP proxy. I'm not trying to prevent solutions, just trying to steer toward solutions that are simple and effective if possible rather than mucking with TCP and making a special TCP for satellite if one is not really needed. Curtis btw- Is this dial access over POTS in Alaska across a satellite link, then into the lossy Internet? If so, then a proxy on either side would be a big win and a proxy on both sides, one pointing to the other, even more so. ------------------------------ Date: Tue, 22 Apr 1997 15:38 -0500 From: "Martone Mike" Subject: Re: Stray thought on mitigation of latency ---------- From: Curtis Villamizar To: Fred Baker Cc: curtis; tcp-over-satellite Subject: Re: Stray thought on mitigation of latency Date: Tuesday, April 22, 1997 3:11PM In message , Fred Baker writes: > At 4:23 PM -0400 4/21/97, Curtis Villamizar wrote: > >I'm not sure what you mean by painfully slow. Today I can click on a > >web page and lose the SYN packet and wait 6 seconds for the SYN to be > >retransmitted without involving a satellite link. > > yes, but does it happen on every session? I have a user who contacted me > not too long ago complaining that his T-1 satellite link from Alaska to > Colorado (which he says has a two second RTT fills the void>) is delivering no more than about 2 KBPS actual throughput. > IMHO, the most logical reason is that he sends one packet, waits two > seconds, sends two packets, waits two seconds, and 50% of the time is now > done with his transfer. > > Yes, dropping the odd SYN packet is painful, and yes, accessing any > Netscape site is very painful. But my user is telling me that *everything* > is painful. If most of the web contact of interest is not in Alaske, put a web proxy that supports HTTP 1.1 on either side of the satellite link. Point one proxy server at the other. You now control the TCP window on either side and if multiple users keep HTTP traffic flowing you can avoid the ramp up. --------------------------------------- Pardon the ignorance, but....what does HTTP 1.1 offer that improves performance in this case. I thought HTTP 1.0 does parallel TCP sessions to retrieve things, and HTTP 1.1 uses a single connection. Wouldn't mutliple connections actually be more beneficial in the case of BW-Delay Product issues, as well as the slow start? Please elaborate on HTTP 1.1. ------------------------------ Date: Tue, 22 Apr 1997 16:17:56 -0400 From: Postmaster Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Tuesday, April 22, 1997 4:07 PM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: ; > From: ; > Subject: Re: Stray thought on mitigation of la > > > ------------------------------ Date: Tue, 22 Apr 1997 16:23:30 -0400 From: Postmaster Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Monday, April 21, 1997 8:12 PM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: > From: > Subject: Position Open > > > ------------------------------ Date: Tue, 22 Apr 1997 16:23:41 -0400 From: Postmaster Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Monday, April 21, 1997 7:46 PM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: > From: > Subject: t-o-s digest for 4/19th > > > ------------------------------ Date: Tue, 22 Apr 1997 13:34:36 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 3:38 PM -0500 4/22/97, Martone Mike wrote: >Pardon the ignorance, but....what does HTTP 1.1 offer that improves >performance in this case. I thought HTTP 1.0 does parallel TCP sessions to >retrieve things, and HTTP 1.1 uses a single connection. Wouldn't mutliple >connections actually be more beneficial in the case of BW-Delay Product >issues, as well as the slow start? Netscape uses multiple TCPs, but that's not fundamental to HTTP 1.0; HTTP 1.0 simply moves a file from here to there. HTTP 1.1, in this context, sends multiple files from here to there serially. The thing that this fixes is that we now have memory of the results of slow start across several files, which means that there's a lot smoother behavior. Mark Allman will argue that multiple parallel TCPs is a bit better across a satellite, because you have a faster startup. I'm not sure that solves the problem for small files, though. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The surest sign that intelligent life exists elsewhere in the universe is that it has never tried to contact us. Calvin and Hobbes (Bill Watterson) ------------------------------ Date: Tue, 22 Apr 1997 14:02:20 -0700 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Re: Stray thought on mitigation of latency > Pardon the ignorance, but....what does HTTP 1.1 offer that improves > performance in this case. I thought HTTP 1.0 does parallel TCP sessions to > retrieve things, and HTTP 1.1 uses a single connection. Wouldn't mutliple > connections actually be more beneficial in the case of BW-Delay Product > issues, as well as the slow start? > > Please elaborate on HTTP 1.1. HTTP 1.1 does persistent connection, and also pipelining. Our experience of using HTTP 1.1 over NASA ACTS satellite shows that it is significant faster then HTTP 1.0. (About 3-4 times faster on the pages that we tried.) Maybe TCPSAT WG should recommend HTTP 1.1 to be the protocol used for WWW over satellite. --ygz ============================================================================== Yongguang Zhang, Ph.D., Research Staff Member (Computer Science) Hughes Research Laboratories, RL-96, 3011 Malibu Canyon Road, Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com phone: 310-317-5147 URL: http://www.wins.hrl.com/people/ygz fax: 310-317-5695 ------------------------------ Date: Tue, 22 Apr 1997 16:57:00 -0400 From: matthew halsey Subject: Re: Stray thought on mitigation of latency This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-00337F35 Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Reply Text: (5 lines follow) Pardon my ignorance also but... Can anyone recommend a good web page that basically spells out the differences of HTTP 1.1 and 1.0? Thanks Matt Original text: (37 lines follow) From YGZ@SMTPGATE (Yongguang Zhang) {ygz%isl.hrl.hac.com.@intelsat1.intelsat.int}, on 22/4/97 5:02 PM: To: MARTONEM@SMTPGATE (Martone Mike) {martonem@bah.com} Cc: CURTIS@SMTPGATE {curtis@ans.net}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} > Pardon the ignorance, but....what does HTTP 1.1 offer that improves > performance in this case. I thought HTTP 1.0 does parallel TCP sessions to > retrieve things, and HTTP 1.1 uses a single connection. Wouldn't mutliple > connections actually be more beneficial in the case of BW-Delay Product > issues, as well as the slow start? > > Please elaborate on HTTP 1.1. HTTP 1.1 does persistent connection, and also pipelining. Our experience of using HTTP 1.1 over NASA ACTS satellite shows that it is significant faster then HTTP 1.0. (About 3-4 times faster on the pages that we tried.) Maybe TCPSAT WG should recommend HTTP 1.1 to be the protocol used for WWW over satellite. --ygz ============================================================================= = Yongguang Zhang, Ph.D., Research Staff Member (Computer Science) Hughes Research Laboratories, RL-96, 3011 Malibu Canyon Road, Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com phone: 310-317-5147 URL: http://www.wins.hrl.com/people/ygz fax: 310-317-5695 Use Proportional Font: true Previous From: YGZ@SMTPGATE (Yongguang Zhang) {ygz%isl.hrl.hac.com.@intelsat1.intelsat.int} Previous To: MARTONEM@SMTPGATE (Martone Mike) {martonem@bah.com} Previous Cc: CURTIS@SMTPGATE {curtis@ans.net}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-00337F35 Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjb9kCgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQADQDEAQAAAAAAAAAAAAAAAAAA AAAAAAAAT3JpZ2luYWwgdGV4dDcFRgoAAAAAAAAAAAAAsgEDADcFrAECAAIA AAAdAAIAAQABAAABAAAAAAAAAgABATcEAAAAAAAAOP8AAAAAAACQAQAAAAAA AAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8AAAAA AACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAGgAAQBoAKAAAQCgAAABAQAAAQEBAgABAfd/AgACAfd/AgBJAfd/ AgCYAfd/AgDmAfd/AgAxAvd/AgBWAvd/AgBYAvd/AgB4Avd/AgB5Avd/AgCz Avd/AgD+Avd/AgBLA/d/AgBWA/d/AgBXA/d/AgChA/d/AgCxA/d/AgCyA/d/ AgC6A/d/AgC7A/d/AgAKBPd/AgBLBPd/AgCaBPd/AgDpBPd/AgA4Bfd/AAAA AAAAAAAAAAAAZAABgAEBAAMBgAQBAAYBgAcBAAkBgAoBAAwBgA0BAA8BgBAB ABIBgBMBABUAAAAAAAAAAAAAAABkAAGwAQFgAwEQBQHABgFwCAEgCgHQCwGA DQEwDwHgEAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-00337F35-- ------------------------------ Date: Tue, 22 Apr 1997 18:10:08 -0400 From: Tim Shepard Subject: Re: Stray thought on mitigation of latency > Can anyone recommend a good web page that basically spells out the > differences of HTTP 1.1 and 1.0? I'm not completely up to speed on everything you'll find there, but I think you might find what you want down this trail... http://www.w3.org/ http://www.w3.org/pub/WWW/Protocols/NL-PerfNote.html http://www.w3.org/pub/WWW/Protocols/HTTP/Performance/Pipeline.html There's also an (some) internet draft(s) done by the HTTP-1.1 WG which you should be able to find in the usual internet-draft places (ftp.ietf.org). -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Tue, 22 Apr 1997 19:24:37 -0400 From: Mark Allman Subject: Re: Stray thought on mitigation of latency > Mark Allman will argue that multiple parallel TCPs is a bit better > across a satellite, because you have a faster startup. I'm not > sure that solves the problem for small files, though. Quite to the contrary Fred. I have never advocated the use of multiple parallel TCP connections in anything other than experimental environments. I believe that 1.1 has great potential to be a big win. allman ------------------------------ Date: Tue, 22 Apr 1997 16:51:36 -0700 From: Fred Baker Subject: Re: Stray thought on mitigation of latency At 7:24 PM -0400 4/22/97, Mark Allman wrote: >> Mark Allman will argue that multiple parallel TCPs is a bit better >> across a satellite, because you have a faster startup. I'm not >> sure that solves the problem for small files, though. > >Quite to the contrary Fred. I have never advocated the use of >multiple parallel TCP connections in anything other than >experimental environments. I believe that 1.1 has great potential >to be a big win. sorry, I thought last time we discussed this you pointed me to an article that seemed to be pushing hard on the idea of using multiple TCP sessions. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The surest sign that intelligent life exists elsewhere in the universe is that it has never tried to contact us. Calvin and Hobbes (Bill Watterson) ------------------------------ Date: Tue, 22 Apr 1997 22:38:44 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency Since Tim has already provided the URL for the "pipeline" paper at W3, I don't have anything relevant to contribute, but... > HTTP 1.1 does persistent connection, and also pipelining. > Our experience of using HTTP 1.1 over NASA ACTS satellite shows that it is > significant faster then HTTP 1.0. (About 3-4 times faster on the pages that > we tried.) > > Maybe TCPSAT WG should recommend HTTP 1.1 to be the protocol used for WWW > over satellite. Funny you should mention this! We need someone who would be comfortable taking the lead on writing up HTTP 1.1 for the draft document. Volunteers please let me know! Thanks! Eric ------------------------------ Date: Wed, 23 Apr 1997 10:40:48 +0800 From: Barry Raveendran Greene Subject: RE: Stray thought on mitigation of latency Hi, Agreed, dual caches will help. In fact a signal cache helps. User who get frustrated with a download of the Web page and stop or reload will not waist the bandwidth. The Cache can be configured to finish the download and cache the objects. > I'm fairly sure apache allows you to configure a proxy server that > points to another proxy. Why don't you put an apache proxy on either > side of the satellite link to Alaska and see if that makes a big > difference. I trust you can find a PC running BSD on either end or > other suitable host. You may have better luck with Squid. There is a excellent support community for Squid. Key URL is http://www.nlanr.net/Cache. Look for the software section for the current Squid pointer. > btw- Is this dial access over POTS in Alaska across a satellite link, > then into the lossy Internet? If so, then a proxy on either side > would be a big win and a proxy on both sides, one pointing to the > other, even more so. If you cannot get a proxy on both sides of the link, then just get one on your side and join the global cache hierarchy. Details are on the NLANR Cache page. Barry - -- - -- - -- Barry Raveendran Greene | || || | Senior Consultant | || || | Consulting Engineering | |||| |||| | tel: +65 738-5535 ext 235 | ..:||||||:..:||||||:.. | e-mail: bgreene@cisco.com | c i s c o S y s t e m s | ------------------------------ Date: Wed, 23 Apr 1997 04:48:43 -0700 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Re: Stray thought on mitigation of latency > Funny you should mention this! We need someone who would be comfortable > taking the lead on writing up HTTP 1.1 for the draft document. Volunteers > please let me know! I am sorry I missed the Friday session at Memphis. What's the conclusion or action list? --ygz ------------------------------ Date: Wed, 23 Apr 1997 11:05:34 -0400 (EDT) From: "M. Scott Corson" Subject: IPoS URL has changed The IPoS howepage is now located at... http://www.isr.umd.edu/CSHCN/Links/IPoS/ - -Scott ******************************************************************************* M. Scott Corson Institute for Systems Research corson@isr.umd.edu A.V. Williams Bldg. (115) Phone: 301-405-6630 (Rm. 2205 AVW) University of Maryland Fax: 301-314-8586 College Park, MD 20742 ******************************************************************************* ------------------------------ Date: Wed, 23 Apr 1997 13:36:27 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency < Snip > >If most of the web contact of interest is not in Alaske, put a web >proxy that supports HTTP 1.1 on either side of the satellite link. >Point one proxy server at the other. You now control the TCP window >on either side and if multiple users keep HTTP traffic flowing you can >avoid the ramp up. Exactly! You don't *necessarily* get this benefit from spoofing on either side of the satellite link. Furthermore, in the process of isolating the long(er)-delay paths from the terrestrial network, you get a couple of more benefits: o It is more likely that the servers, or as I've heard them referred to, "content-providers" are going to get upgrades to support 1323 and 2018 extensions long before all the individual PCs and whatever will. Because it is an application-level thing, we are likely to see HTTP-1.1 widely distributed in browsers before we can also get wide-scale usage of something like SACK. This is OK, but we can do better; By making sure that your proxy/cache/whatever has the appropriate options (1322 and 2018 defined options), you gain the benefits on the connection between the proxy and the server when the server is equally capable. Not quite the same as end-to-end, but much better than nothing at all. You begin to see performance increases sooner than waiting a few more years for RFC-1323 and RFC-2018 defined options to show up on everyone's (i.e., Joe Windowsuser's) desktop. o Connections respond quicker to congestion notification (loss), reducing the length of the congestion event (better for all traffic on the shared network segments) *and* making recovery for our connections fairer than if we were spanning the long-delay path. o If it turns out that it is necessary to do something "different" for TCP over satellite links (say, slow-start), you can do so without spilling the contamination over the wider, shared network segments. This allows us to experiment with options on the private network segments and inter-operate with the Internet at large without risking detrimental behavior to shared traffic over the larger network. >Email doesn't matter much. Its not transferred while the user is >waiting unless your POP server is on the other side of the satellite >which is just plain dumb. I doubt that is the case. This brings up an interesting point, we need to make sure we explain in the document *why* configurations like that are just plain dumb; Although it seems obvious to me, education is one of our goals. >For web, the 2 seconds for an HTML file with no images isn't painful. >If bandwidth is not an issue on the satellite link but delay is, >parallel image fetch makes a lot of sense. Then a 2KB HTML and N >16KB images would take roughly 5-6 RTTs or 3 seconds. >If you do the web proxy thing and the satellite link has available >bandwidth, then the continental US side with congested ISP links and >servers may become the limit, not the satellite link. Right, but you don't necessarily want the effects of the retransmissions on the terrestrial network to spill over to effect the satellite link. Once the data has traversed the satellite link successfully, retransmissions are unnecessary and wasteful. If the traffic source is on the far side of the satellite link from the congestion, you *do* want it to respond to congestion by slowing down, but you don't necessarily want him to retransmit any data that already made it over the link once. This would be an inefficient use of satellite capacity. By splitting the transport connection, you can achieve the slow-down through back-pressure over the satellite link but not require needless retransmissions over the more costly media. The proxy entity on the near side of the congestion can handle the required retransmissions. You'll notice that in most/all traffic flows, I want to treat the proxy as a pipeline with slightly elastic buffers (not big enough to allow a huge rate disparity across the different connections, but big enough to smooth the effects of congestion on the terrestrial network). I don't want to wait for a connection to terminate before sending the data on the next connection in the chain. The semantics may be slightly unconventional, but it lets you apply back-pressure across the split connections and makes the required buffering manageable. < Snip > >There are a lot of small flows but a high percentage of the packets >flowing through this router where associated with longer duration >flows. [For discussion] And in this environment, it is probably appropriate to treat different types of flows differently. o Bulk Traffic (example: FTP, SMTP) o Semi-Interactive Traffic (example: HTTP) O Interactive Traffic (example: telnet) The first two classes *may* go through a proxy entity, but the latter *needs* to deal with the inherent latency end-to-end, so a proxy will provide more complexity, but perhaps only limited benefits. < Snip > >Try the HTTP proxy. I'm not trying to prevent solutions, just trying >to steer toward solutions that are simple and effective if possible >rather than mucking with TCP and making a special TCP for satellite >if one is not really needed. I think this posture is an appropriate one to take when approaching this (or any other) problem. Eric ------------------------------ Date: Wed, 23 Apr 1997 14:33:10 -0400 (EDT) From: Eric Travis Subject: Re: Stray thought on mitigation of latency >I am sorry I missed the Friday session at Memphis. >What's the conclusion or action list? In a nutshell, I *believe* we decided to push forward assuming (hoping is a better word) that we will get the charter approved and be granted WG status. In the event that we don't, we will still go forward and submit the results as an Internet Draft. For a more precise blow-by-blow, Mark Allman posted an amazingly complete set of minutes to this list a little over a week ago. Since I don't think the archive for this mailing list is available, we need to get the meeting minutes reposted. There were also individual messages regarding: - Action items from Memphis - Draft document outline - Tentative schedule I'll repost these shortly (my e-mail access is spotty), probably with a refinement to the draft outline. If the meeting minutes are reposted by then, I'll repost them too. Eric ------------------------------ End of tcp-over-satellite-digest V1 #15 *************************************** tcp-over-satellite-digest Thursday, April 24 1997 Volume 01 : Number 016 In this issues: Re: Stray thought on mitigation of latency Re: Stray thought on mitigation of latency TCP/SATCOM draft suggestion FW: Notification: Inbound Mail Failure - Address not found Re: Stray thought on mitigation of latency Re: TCP/SATCOM draft suggestion Re: TCP/SATCOM draft suggestion Re: TCP/SATCOM draft suggestion Work outline Work outline Definition of High Bandwidth Delay Product Re: Definition of High Bandwidth Delay Product Re: Definition of High Bandwidth Delay Product FW: Notification: Inbound Mail Failure - Address not found FW: Notification: Inbound Mail Failure - Address not found Re: Re: TCP/SATCOM draft suggestion REMINDER: Research/Work Outlines Due Tomorrow!! FW: Notification: Inbound Mail Failure - Address not found Cuevas address Repost:Outline See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Wed, 23 Apr 1997 17:05:03 -0400 From: Mark Allman Subject: Re: Stray thought on mitigation of latency > Since I don't think the archive for this mailing list is > available, we need to get the meeting minutes reposted. Here they are... allman - --- TCP Over Satellite BOF ====================================================================== April 10 and 11, 1997 Chair: Aaron Falk, TRW (Aaron.Falk@trw.com) Minutes reported by Mark Allman (mallman@cs.ohiou.edu). April 10, 1997 ====================================================================== Aaron Falk started the meeting with a brief overview of the agenda and meeting purpose. The BOF was called to determine if interest existed for the formation of a working group that would be charged with documenting the issues involved in using TCP on satellite channels. Tim Shepard, BBN (shep@bbn.com) presented a view of the problem that satellite channels present: -the big problem is the delay -generally receive windows are low in the net; an 8K receive window can provide maximum throughput of 128 kilobits/second -this throughput is not bad in some situations (compared to a modem, for example) -but, to use all available bandwidth, we need bigger windows -but, when using big windows, slow start can cause a large amount of packet loss -selective acknowledgments (SACKs) help this problem -the problem is well understood on the receiver side -we need to allow bigger windows with window scaling and PAWS (RFC 1323) -we need to generate selective acknowledgments (RFC 2018) -the problem is not well understood on the sender side; more research is needed -a question was raised about whether RED queuing would help; answer: it might, but we don't know yet Yongguang Zhang, Hughes (ygz@isl.hrl.hac.com) presented some recent research results using the NASA ACTS system: -experimental environment: -ACTS satellite -Reno/SACK FreeBSD machines -workload: tcplib traffic -24 KB windows -avg. file size: 27 KB -compared to a terrestrial network with 50 ms RTT -measured useful traffic/hour -terrestrial channel was able to better utilize the available network capacity when compared to the satellite channel -ongoing experiments: -ATM vs. non-ATM on either side of the satellite channel -other TCP versions (Tahoe, Reno, Vegas, 4K slow start proposal) -testing under delay emulation Shawn Ostermann, Ohio Univ. (ostermann@cs.ohiou.edu) presented some of the current solutions being proposed and studied: -short transfers do not work very well (small number of segments vs. the long RTT and slow start are against us) -long transfers have problems too (the windows are big, which make the congestion control algorithms take a long time to ramp up (either at the beginning of the transfer, or after a loss) -OU built a multiple connection FTP client and server (XFTP) -was able to utilize 96% of available bandwidth -lessons from XFTP: -we need bigger windows -need more buffering in the routers (and maybe alternate queuing, like RED) -detecting and correcting loss -cant let pipe empty because: -takes 4 seconds to slow start to 100 KB -takes 50 seconds to recover from one packet loss (using congestion avoidance) -we need to use fast retransmit and recovery -we need to use SACKs -we need to use MTU discovery so that we are sending as much data as possible in each segment -what we understand: -long connections with no loss do very well -short and medium connections do not do well -short transfers never leave slow start -delayed ACKs make slow start even worse -what might help? (open issues) -bigger initial windows (Floyd) -more aggressive window increase algorithm (Allman, et. al) -the slow start changes above have increased throughput and did not significantly decrease goodput or increase loss in tests at OU -do not perform congestion control if a drop is due to corruption, rather than congestion -use an ssthresh estimator -use RED -goal: -extend TCP while keeping backward compatibility -do not introduce mechanisms that will hurt other types of networks -a question was asked about using indirect-TCP (i.e., breaking the TCP connection into 2 connection at the basestation); Shawn answered that it was an interesting idea that he did not know a lot about Discussion: Aaron started a discussion about whether or not a WG should be created to document the problems of using TCP over satellite links and some possible solutions. -the proposed charter calls for cataloging problems and mitigating mechanisms -out-of-scope: -non-interoperable TCP implementations -substantial changes to the TCP protocol -link-layer changes -research Some comments: -What is the purpose of a working group? Use a mailing list, since the scope is small, the group will be short-lived and the deliverable is only one I-D/RFC -Allyn Romanow (co-area director for transport area) commented that there is already quite a bit of initiative involved in this group and that a staged approach is worthwhile to try out -Matt Mathis: unfocused WGs are unhealthy and this will be a very focused WG, so it is a good experiment -Fred Baker: this type of WG will make the net more useful in the third world and therefore, we should make the effort. Also, these are not just satellite questions but apply to high delay*bandwidth links everywhere. -Allyn conducted a straw poll which was in favor of creating the WG Goal: to deliver a document at the IETF meeting in Washington, DC (12/97). WWW Site: http://www.isr.umd.edu/CCDS/IPoS/ TCP Over Satellite mailing list: To access it send email to: majordomo@listserv.trw.com with the following in the body of the message: subscribe tcp-over-satellite Interim meeting might be necessary. April 11, 1997 ====================================================================== Eric Travis outlined the contents of the draft that will be produced. Purpose: -identify and explain performance problems, including pointers to simulation results, emulation results and experiments conducted over real satellites -identify and explain mechanisms that help overcome the outlined performance problems -categorize engineering problems and research problems -discussion: -we need a document for people running real networks that outlines the problems and solutions for problems encountered in the satellite environment -want to outline various network topologies that contain satellites: -examples given: node-sat-node node-sat-net-node node-net-sat-net-node node-???-sat-???-node -specific issues: -identify and describe: -large delay*bandwidth -long feedback path -asymmetric channels -corruption -??? -provide a problem statement and explanation of possible solutions for each of the above -discussion (what do they provide to each environment outlined above) -references to research results -tag each area as an engineering or research problem -validation of problem/solution -outline interactions between mitigating techniques -outline research areas -provide recommendations -what is available? -what is useful? -what should be applied? Discussion: -outline the constraints, i.e. that the solutions should not have negative impacts on the Internet -we need to provide validation where possible -maybe include something about application or network layer protocols, if they have an impact on TCP Work Schedule: -1st draft target date: 6/30/97 -2nd draft target date: last day to submit drafts before Munich meeting -3rd draft target date: post Munich -4th draft target date: October Aaron asked people doing TCP over satellite research to drop a short note to the mailing list outlining their work. ------------------------------ Date: Wed, 23 Apr 1997 17:26:46 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message <9239026A@etownpo1.bah.com>, "Martone Mike" writes: > > ---------- > From: Curtis Villamizar > If most of the web contact of interest is not in Alaske, put a web > proxy that supports HTTP 1.1 on either side of the satellite link. > Point one proxy server at the other. You now control the TCP window > on either side and if multiple users keep HTTP traffic flowing you can > avoid the ramp up. > > --------------------------------------- > Pardon the ignorance, but....what does HTTP 1.1 offer that improves > performance in this case. I thought HTTP 1.0 does parallel TCP sessions to > retrieve things, and HTTP 1.1 uses a single connection. Wouldn't mutliple > connections actually be more beneficial in the case of BW-Delay Product > issues, as well as the slow start? The benefit here is putting a proxy on either side of the satelite. Using HTTP 1.1 would provide a further benefit if the two proxies had a somewhat continuous workload. This might be the case if a large subset of the people on the Alaska side of the satellite pointed their browser at the proxy on the Alaska side. If HTTP 1.0 was used, every page and image would slow start. Since all access across the link is to the proxy on the other side, with HTTP 1.1 there is a good chance of being able to pick up the TCP flow before it idles and repeats slow start. Most proxy servers allow a configurable number of TCP flows in parallel. This could also be used to avoid the case where ssthresh gets set low on a few flows due to a loss when cwnd is small and to speed the idle ramp up. I'd still set this low. I'm not an expert webmaster so I may be confusing apache and squid capabilities here. I think you need the module mod_proxy and need to use ProxyRemote on the Alaska side and ProxyRequests on both sides. ProxyRequests on ProxyRemote * http://proxy.lower48.net Looking at the apache docs, it appears that the only control on the number of simultaneous connections is MaxClients. I think setting MaxClients low will result in connection refused rather than queueing additional responses, so this may not be a viable means to limit the number of connections. Curtis ------------------------------ Date: Wed, 23 Apr 1997 17:39:50 -0400 (EDT) From: Karen Lisa Hansen Subject: TCP/SATCOM draft suggestion - ---------- Forwarded message ---- From: Eric Travis To: Karen Lisa Hansen Cc: travis@clark.net Subject: Re: TCP/SATCOM draft >I have been getting email from the Coast Guard in Alaska (different than >Fred's AK contact), and they are interested beyond HTTP: > >"Our requirements go way beyond http. In addition to Internet/Intranet >services, our Alaska circuits are intended to provide remote host access >(moving toward WEB frontend, but not there yet), e-mail, remote system >management, and possibly IPTV for remote training, etc. The end state >we will be looking for is the most efficient TCP transfer possible, >given the delay." > >My concern is, when our document is written, will it address the >information needs of the "person in the field"? do we need some benchmark >of what the "Coast Guard in Alaska" expects to achieve over GEOs? I now share your concern. We *do* need to make sure that we provide guidance for the "person in the field". A benchmark of what these folks need would be very useful - Do you want to take this topic to the larger mailing list? I think it is would be most useful! I was kind of expecting (OK, hoping) this type of thing would show-up, and that's why I said that I would be sympathetic to *any* types of inputs back in Memphis. Based on some good discussions with Aaron and Mark Allman, I'm planning on going in and refining the structure of the draft outline. This provides me with another significan issue to take into consideration. Based on your message, I'm now toying with the thought of suggesting that we add a section or appendix to the document that would be a "field-user's guide" to deploying network services over satellite links. This would provide guidance/collected wisdom to "best-practices" in setting up various services (e-mail, http, management, etc.). I have been hoping that we (the consensus) wouldn't settle for just an HTTP-centric view of applications. This (and some of the traffic on the mailing list) tells me that general guidance on structuring various services over long-delay paths also needs to be addressed. I'm comfortable with that - and it is appropriate. I'm willing to expand our target scope and prune back based upon the inputs that we get. Then again, I always set my standards too high and lose sleep as a result, but here I think it is the right way to go. Eric ------------------------------ Date: Wed, 23 Apr 1997 17:45:37 -0400 From: Postmaster Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Wednesday, April 23, 1997 5:45 PM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: > From: > Subject: TCP/SATCOM draft suggestion > > > ------------------------------ Date: Wed, 23 Apr 1997 21:18:56 -0400 From: Curtis Villamizar Subject: Re: Stray thought on mitigation of latency In message <01BC5008.267CBA00@bgreene-pc.cisco.com>, Barry Raveendran Greene wr ites: > > > I'm fairly sure apache allows you to configure a proxy server that > > points to another proxy. Why don't you put an apache proxy on either > > side of the satellite link to Alaska and see if that makes a big > > difference. I trust you can find a PC running BSD on either end or > > other suitable host. > > You may have better luck with Squid. There is a excellent support community > for Squid. Key URL is http://www.nlanr.net/Cache. Look for the software > section for the current Squid pointer. A bit off topic by now, but... I'm running apache for local content with a squid cache to front end slow connections (my house, the local library :). Works great. The only reason I mentioned trying to use apache for this is that squid doesn't support HTTP 1.1 persistant connections last I checked. There is no mention in the 1.1.9 release notes. I think the persistant connections would be a very big win over satellite so the suggestion to try apache. Otherwise squid would be preferred. Curtis ------------------------------ Date: Wed, 23 Apr 1997 22:52:28 -0400 From: Arun Welch Subject: Re: TCP/SATCOM draft suggestion At 05:39 PM 4/23/97 -0400, Karen Lisa Hansen wrote: > >---------- Forwarded message ---- >From: Eric Travis >To: Karen Lisa Hansen >Cc: travis@clark.net >Subject: Re: TCP/SATCOM draft > > > >> >>My concern is, when our document is written, will it address the >>information needs of the "person in the field"? do we need some benchmark >>of what the "Coast Guard in Alaska" expects to achieve over GEOs? > >We *do* need to make sure that we provide guidance for the >"person in the field". A benchmark of what these folks need would be very >useful - Do you want to take this topic to the larger mailing list? >I think it is would be most useful! > I agree that it would be useful, but we need to contain the scope appropriately. There can be enormous variability depending on the lower-level protocols, the OS's involved, etc. We need to keep it general enough ("use HTTP 1.1, caching, etc), without going into specifics on the OS vagaries. Yes, this will make the document less useful, but we might be able to solve that by providing pointers to published papers with actual results or in an appendix. For example, I've probably still got the settings we used in Unicos for running HiPPI over ACTS, and I can dig up the Solaris 2.6 ndd settings for running IP over OC-12 ATM on ACTS. Another alternative would be to build a collection of this sort of data, but *not* publish it in the RFC, but rather keep it alive on a web site somewhere. This way it would always be current. Is the RFC intended to be an informational or BCP? ...arun ------------------------------ Date: Wed, 23 Apr 1997 23:46:23 -0400 (EDT) From: Eric Travis Subject: Re: TCP/SATCOM draft suggestion My thought was to keep things *very* general, but to provide guidance as such things at an architectural level. It would be useful to hear what kind of services the Alaskan Coast Guard (and others) need to provide over their satellite link(s). There is a perception that the sole driver of this is HTTP traffic, but as Karen illustrates and Fred has reminded us on several occasions, there are folks without rich connectivity to CONUS (and no fiber) need to do *everything* over their satellite link(s). While it is not really related to the scope of the RFC, I *do* like the idea of there being a "live" collection of relevant data somewhere on the net. I'm sure it would be of tremendous help to the community. Eric >I agree that it would be useful, but we need to contain the scope >appropriately. There can be enormous variability depending on the >lower-level protocols, the OS's involved, etc. We need to keep it general >enough ("use HTTP 1.1, caching, etc), without going into specifics on the >OS vagaries. Yes, this will make the document less useful, but we might >be able to solve that by providing pointers to published papers with >actual results or in an appendix. For example, I've probably still got >the settings we used in Unicos for running HiPPI over ACTS, and I can dig >up the Solaris 2.6 ndd settings for running IP over OC-12 ATM on ACTS. >Another alternative would be to build a collection of this sort of data, >but *not* publish it in the RFC, but rather keep it alive on a web site >somewhere. This way it would always be current. ------------------------------ Date: Thu, 24 Apr 1997 07:21:22 -0400 From: Arun Welch Subject: Re: TCP/SATCOM draft suggestion At 11:46 PM 4/23/97 -0400, Eric Travis wrote: > > >While it is not really related to the scope of the RFC, >I *do* like the idea of there being a "live" collection of >relevant data somewhere on the net. I'm sure it would be >of tremendous help to the community. > If folks will send me data, I'll put it together. ...arun ------------------------------ Date: Thu, 24 Apr 1997 07:46:27 -0400 From: Arun Welch Subject: Work outline >Aaron asked people doing TCP over satellite research to drop a short >note to the mailing list outlining their work. I've worked on various ACTS high data rate projects for the last 3 years. The first was when I was at OARnet, when I built and ran the OSU end of the Viento project (URL's at the bottom of this message). A couple of months ago I worked on the OC-12 ATM testing that Dave Beering put together, and some time next month I'll be building the OSU end of the Mission network. What I'll be doing after that is pretty much up in the air, so if anyone knows of open positions doing interesting work I'd be happy to send you a resume. On the other hand, I might end up having lots of time to work on the RFC :-). ...arun http://www.cgrg.ohio-state.edu/accad/research/viento/ http://www.cgrg.ohio-state.edu/accad/research/mission/ http://mrpink.lerc.nasa.gov/aamexp.html ------------------------------ Date: 24 Apr 1997 09:14 EDT From: "David Mercer" Subject: Work outline My research does not really concern satellites but is rather concerned with TCP over high latency links. In particular, TCP over non-tranparent (NT Mode) GSM. The link layer protocol employed by NT mode GSM (Radio Link Protocol (RLP)) is HDLCish and can cause VERY large packet latencies while still providing acceptable throughput (9.6Kbps). What makes this link unique is the fact that it is not a fixed, speed of light delay. Rather it is a function of link errors (BER of 10-3) and the FEC scheme employed by RLP. This makes the latency highly variable and a function of moment-to-moment channel characteristics. My interest in TCP over Sat is that it is a specific case (fixed round trip delay) of the general (long RT delay with varience) problem I am working on. DAve - -- Dave Mercer, P.Eng. Computing Technology Lab, Nortel Technology (613) 763-9362 ------------------------------ Date: Thu, 24 Apr 1997 09:24:11 +0000 From: "David Mercer" Subject: Definition of High Bandwidth Delay Product > > This is a MIME Encoded message. If you are reading this text, > then your mail reader does not support the MIME (Multipurpose > Internet Mail Extensions) standard. To take full advantage of > the features of this message, you need to upgrade your mailer to > a MIME V1.0 compliant package. Some parts of this message may > be in human readable form. > > MIME Decoding Utilities > ----------------------- > To make use of this encoded message, you can decode it using > a MIME Decoding utility. The following are some freely available > MIME decoding utilities: > > UNIX Users > ----------- > Metamail : ftp://ftp.bellcore.com/pub/nsb/mm2.7.tar.Z > This is the Metamail source code distribution. > Mpack : ftp://ftp.andrew.cmu.edu/pub/mpack/mpack-1.5-src.tar.Z > Source code for all platforms. > MACINTOSH Users > --------------- > Mpack : ftp://ftp.andrew.cmu.edu/pub/mpack/mpack-1.5-mac.hqx > Contains the Macintosh binaries > PC Users > --------- > Metamail : ftp://ftp.bellcore.com/pub/nsb/mm2.7.dos.zip > This is the MS-DOS binaries > > Mpack : ftp://ftp.andrew.cmu.edu/pub/mpack/mpack15d.zip > This contains the MS-DOS binaries > MIME-Aware Mail User Agents > --------------------------- > Note that the following Mail User Agents support MIME and hence if > one of these are used the message will be automatically decoded > be in a readable state: > Elm Version 2.4 (UNIX) > ftp://ftp.uu.net/networking/mail/elm > Eudora 1.4.4 (Macintosh, MS-Windows) > ftp://ftp.qualcomm.com/quest/eudora/windows/1.4/eudor144.exe > ftp://ftp.qualcomm.com/quest/eudora/mac/1.4/eudora144.hqx > Pegasus mail (MS-DOS, MS-Windows, Macintosh) > ftp://risc.ua.edu/pub/network/pegasus/* > Pine (UNIX) > ftp://ftp.cac.washington.edu/pine/pine.tar.Z > - --------------34939E6A0B640E279880B7A8 Content-type: text/plain; charset="us-ascii" Can anybody put a number on what defines a "High Bandwidth Delay Product"? If this is a FAQ, sorry, I just joined the list. DAve - --------------34939E6A0B640E279880B7A8 Content-type: text/x-vcard; charset="us-ascii"; name="vcard.vcf" Content-transfer-encoding: 7bit Content-Description: Card for David Mercer Content-Disposition: attachment; filename="vcard.vcf" begin:vcard fn:David Mercer n:Mercer;David org:Computing Technology Lab, Nortel Technology adr:;;PO Box 3511, Station C;Ottawa;Ontario;K1Y 4H7; email;internet:davidme@nortel.ca title:Internet Specialist tel;work:(613) 763-9362 tel;fax:(613) 763-4222 x-mozilla-cpt:;0 x-mozilla-html:FALSE end:vcard - --------------34939E6A0B640E279880B7A8-- ------------------------------ Date: Thu, 24 Apr 1997 10:54:24 -0400 From: Daniel R Glover Subject: Re: Definition of High Bandwidth Delay Product At 09:24 AM 4/24/97 +0000, you (David Mercer) wrote: >Can anybody put a number on what defines a "High Bandwidth Delay >Product"? > This might be a good thing to put in the draft, so I'll take a stab at it: The bandwidth*delay product is a measure of the capacity of a link and is defined as the product of the bandwidth and the round-trip time. It also corresponds to the amount of buffering required (window size) at the ends of the "pipe" for maximum utilization of a TCP connection. The maximum window size allowed by the TCP header's 16 bit Window field is 65535 bytes. Therefore, a bandwidth*delay product greater than 65535 bytes may be said to be "high." RFC1323 defines a Long, Fat Network (LFN) as a network containing a "long, fat pipe" which in turn is defined as having a high bandwidth*delay product. RFC1323 specifies a window scaling TCP option which allows a maximum window size of 1,073,725,440 bytes. For a T1 connection of 1.544 M bits/sec, a high bandwidth*delay product would occur with a round-trip delay greater than around 340 msec. For a T1 geosynchronous satellite connection with a delay of, say, 500 msec, the bandwidth*delay product is 772,000 bits or 96,500 bytes which is high by the definition above (>65535 bytes) and falls in the LFN category. For a terrestrial T1 connection with 100 msec round-trip time the bandwidth delay product is 19,300 bytes which is not "high." R/ Dan /************************** Dan Glover dglover@lerc.nasa.gov NASA Lewis Research Center MS 54-2 Cleveland, OH, USA 44135 (216)433-2847 **************************/ ------------------------------ Date: Thu, 24 Apr 1997 11:20:00 -0400 From: matthew halsey Subject: Re: Definition of High Bandwidth Delay Product This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-005DE2FE Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Reply Text: (18 lines follow) One other important thing to remember is that the RTT in the bandwidth-delay product is the end-2-end RTT. Therefore, whilst a satellite may incur close to 600ms RTT (unless you are exactly at the sub-satellite point, on the equator, talking to yourself!) you may experience up to 100 ms of terrestrial delay, due to either congestion and/or routing delays, on either side. This takes the rtt up to 800 ms. Of course this varies upon configuration and I am assuming a satellite connecting two terrestrial networks. Therefore with a 65,536 Byte buffer available and 800ms RTT, the throughput cannot exceed 65536 x 8 / 0.6 = 874 kbps. Now the problem comes when people use TCPs who don't allow this much receive window buffer. An 8192 Byte buffer means a througput of 81.92 kbps per single TCP session. I also feel that this should be spelled out in the document. Matt Original text: (48 lines follow) From DANIEL@SMTPGATE (Daniel R Glover) {Daniel.R.Glover@lerc.nasa.gov}, on 24/4/97 10:54 AM: To: DAVIDME@SMTPGATE ("David Mercer") {davidme@nortel.ca} Cc: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} At 09:24 AM 4/24/97 +0000, you (David Mercer) wrote: >Can anybody put a number on what defines a "High Bandwidth Delay >Product"? > This might be a good thing to put in the draft, so I'll take a stab at it: The bandwidth*delay product is a measure of the capacity of a link and is defined as the product of the bandwidth and the round-trip time. It also corresponds to the amount of buffering required (window size) at the ends of the "pipe" for maximum utilization of a TCP connection. The maximum window size allowed by the TCP header's 16 bit Window field is 65535 bytes. Therefore, a bandwidth*delay product greater than 65535 bytes may be said to be "high." RFC1323 defines a Long, Fat Network (LFN) as a network containing a "long, fat pipe" which in turn is defined as having a high bandwidth*delay product. RFC1323 specifies a window scaling TCP option which allows a maximum window size of 1,073,725,440 bytes. For a T1 connection of 1.544 M bits/sec, a high bandwidth*delay product would occur with a round-trip delay greater than around 340 msec. For a T1 geosynchronous satellite connection with a delay of, say, 500 msec, the bandwidth*delay product is 772,000 bits or 96,500 bytes which is high by the definition above (>65535 bytes) and falls in the LFN category. For a terrestrial T1 connection with 100 msec round-trip time the bandwidth delay product is 19,300 bytes which is not "high." R/ Dan /************************** Dan Glover dglover@lerc.nasa.gov NASA Lewis Research Center MS 54-2 Cleveland, OH, USA 44135 (216)433-2847 **************************/ Use Proportional Font: true Previous From: DANIEL@SMTPGATE (Daniel R Glover) {Daniel.R.Glover@lerc.nasa.gov} Previous To: DAVIDME@SMTPGATE ("David Mercer") {davidme@nortel.ca} Original to: DAVIDME@SMTPGATE ("David Mercer") {davidme@nortel.ca} Previous Cc: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Original cc: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-005DE2FE Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjXcrCgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQADQAwAgAAAAAAAAAAAAAAAAAA AAAAAAAAT3JpZ2luYWwgdGV4dAsHRgoAAAAAAAAAAAAAHgIDAAsHGAICAAIA AAAvAAIAAQABANYAAAAAAAAAAgDXADUGAAAAAAAAOP8AAAAAAACQAQAAAAAA AAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8AAAAA AACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAF4AAQBeAJgAAQCYANYAAQDWANcAAgDXAPd/AgAMAfd/AgBOAfd/ AgBZAfd/AgBbAfd/AgBcAfd/AgCnAfd/AgCoAfd/AgCpAfd/AgDzAfd/AgA9 Avd/AgCKAvd/AgDWAvd/AgAbA/d/AgBoA/d/AgBzA/d/AgB0A/d/AgC/A/d/ AgAMBPd/AgBYBPd/AgB1BPd/AgB2BPd/AgC+BPd/AgAKBfd/AgBSBfd/AgCf Bfd/AgDlBfd/AgAxBvd/AgBeBvd/AgBfBvd/AgBiBvd/AgBmBvd/AgBnBvd/ AgBoBvd/AgCEBvd/AgCPBvd/AgClBvd/AgDABvd/AgDIBvd/AgDhBvd/AgDv Bvd/AgALB/d/AgAMB/d/AAAAAAAAAAAAAAAAZAABgAEBAAMBgAQBAAYBgAcB AAkBgAoBAAwBgA0BAA8BgBABABIBgBMBABUAAAAAAAAAAAAAAABkAAGwAQFg AwEQBQHABgFwCAEgCgHQCwGADQEwDwHgEAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-005DE2FE-- ------------------------------ Date: Thu, 24 Apr 1997 11:30:08 -0400 From: Postmaster Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Thursday, April 24, 1997 7:59 AM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: > From: > Subject: Work outline > > > ------------------------------ Date: Thu, 24 Apr 1997 11:27:46 -0400 From: Postmaster Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Thursday, April 24, 1997 9:34 AM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: BNR400 > From: BNR400 > Subject: Definition of High Bandwidth Delay Pr > > > ------------------------------ Date: Thu, 24 Apr 1997 10:34:41 -0400 (EDT) From: Arun Welch Subject: Re: Re: TCP/SATCOM draft suggestion Arun Welch: >At 11:46 PM 4/23/97 -0400, Eric Travis wrote: >> >> >>While it is not really related to the scope of the RFC, >>I *do* like the idea of there being a "live" collection of >>relevant data somewhere on the net. I'm sure it would be >>of tremendous help to the community. >> > >If folks will send me data, I'll put it together. I knew I'd seen something like this around already. http://www.psc.edu/networking/perf_tune.html is on the right track, but mostly covers the high-bandwidth arena. Maybe extending it to handle the low-bandwidth end would be enough. ...arun ------------------------------ Date: Thu, 24 Apr 1997 08:42:54 -0700 From: Aaron Falk Subject: REMINDER: Research/Work Outlines Due Tomorrow!! Folks- As you may recall, I asked those of you who are doing TCP-over-satellite related work to post a _brief_ outline of your activities so that the larger community could learn about what what you are doing. Remember that our groups lifetime is very short. I hope that you can support the Friday deadline for submission of your research summaries. Thanks, Aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technologies Division afalk@mailsrv1.trw.com ------------------------------ Date: Thu, 24 Apr 1997 11:28:10 -0400 From: "Maughan, Steven, NCSIM" Subject: FW: Notification: Inbound Mail Failure - Address not found >---------- >From: System Administrator[SMTP:postmaster@att.com] >Sent: Thursday, April 24, 1997 9:22 AM >To: Postmaster >Subject: Notification: Inbound Mail Failure - Address not found > >A mail message was not sent because the following address(es) could not be >found: > > ecuevas@ems.att.com > >The message that caused this notification was: > > To: > From: > Subject: Work outline > > > ------------------------------ Date: Thu, 24 Apr 1997 13:30:43 -0400 From: "Eric A. Bobinsky" Subject: Cuevas address Aaron- the address I have on file for Enrique Cuevase is "cuevas@ems.att.com", not "ecuevas@ems.att.com" Maybe that's why your postmaster is hiccuping. Eric ------------------------------ Date: Thu, 24 Apr 1997 14:53:59 -0400 (EDT) From: Eric Travis Subject: Repost:Outline This *will* be modified this weekend based on discussion (public and private) since it's original posting. I'll send the update Sunday or Monday. Eric - ---------- Forwarded message ---------- Date: Mon, 14 Apr 1997 22:36:31 -0400 (EDT) From: Eric Travis To: tcp-over-satellite@achtung.sp.trw.com Subject: Outline As promised in my previous message: Below is a semi-conventional outline as discussed at the Friday BOF session in Memphis. Where possible, I've tried to provided editorial comments to reflect the BOF discussion. It's clear that what I've got below is *very* ambitious; I don't really expect us to hit all of this, but I'm willing to try. As inputs come in, we will reshape the scope and outline to correspond with what is reasonable. Please contribute what you can and whenever possible, post these contributions to the mailing list. They *will* be collected. Don't stress (at this point) about polished prose; Flames related to such will not be tolerated - at least not be me. The point is to get the ideas out, the polishing is relatively easy. Contributing doesn't commit you to further contributions. Explicitly volunteering of effort does, however :o) ====================================================== Draft Outline as Presented/Discussed in Memphis: I. Purpose a. Ensure that any and all recommendations resulting from this effort will not adversely impact other traffic on the shared (non-satellite) media. b. Provide a comprehensive document describing the problems (and potential solutions) associated with TCP performance in the satellite environment. c. Identify, explain *and* wherever possible quantify impacts of satellite environment on TCP performance Quantification: Simulation OK, but attempts should be made to validate simulations Emulations better, same desires regarding validation apply Live measurements most desirable, but probably most elusive Bottom line, whatever is available is desirable; Please contribute any relevant results and efforts wherever possible. Quantifications must *not* be conducted in a vendor specific manner. Impacts of base algorithms and common (or proposed) implementation techniques are acceptable (as are pointers to published work covering this area). I'm of the opinion that discussions/categorizations along the lines of 4.3BSD Tahoe, 4.3BSD Reno, 4.4BSD, Net/1, Net/2, Net/3, etc. are OK. I'm willing to revise my opinion here if necessary. In no uncertain terms are proprietary algorithms, implementations, etc. going to be acceptable. d. Categorize the problems according to those that can be addressed through engineering (applying what we already know) and those that require further research. For the latter, this is simply a tagging/identification effort. Anything else is out of scope of this group. e. Identify and explain what mitigation techniques exist and (in general) how to use them. Efforts should concentrate on the more mature, somewhat well understood techniques/options such as those in RFC 1323, RFC 2018, as well as others that have already been discussed within this group (J.C. Hoe's work, the proposed "4K initial cwnd", etc.). I'm not going to attempt to provide a comprehensive listing here. Explanations should cover not only the specific performance problems that a mitigation technique (attempts) to overcome, but also the why, how and where applicable the problems involved. As above, *nothing* proprietary is going to be acceptable. Along those lines, is the "how to use" coverage. This is likely to be difficult and if it proves too difficult then it should fall off the end of the document. Pointers/discussion to things like using socket-options is probably OK, but not things like how to use ndd on Solaris to tweak things. II. Overview of the problem domain A broad overview of the differences in the satellite environment that cause difficulties for TCP. There should be not effort to go into great detail here, that will be done in the next section. I envision a main element of this section to be a categorization of different architectures/topologies (common) to satellite applications. This should prove to be useful for the discussion of specific issues and mitigation techniques in section 3. Furthermore, such a categorization will allow for comparisons/contrasts to be made by those in non-satellite environments. [Please see the note in the next section regarding scope.] III. Specific Issues =================================================== Note regarding scope: ----------------------- There was some good discussion during Friday's session in Memphis as to whether or not we should address the performance impacts that applications and "link" technologies have on TCP in the satellite environment; As an example at the application level, the (unnecessary) interrogative nature of some applications, such as FTP does impact perceived user-level performance in long delay environments. I don't need to provide examples related to link(ish) technologies. Although this is a gray area (for us), I'd like to restate that *I* will be sympathetic to related contributions or pointers on these topics. I won't promise that these items will make it into the final document - we'll have to see how things fall together. Even if the contributions don't make it into this Informational RFC, I'm sure we can prevent them from going to waste. Also, let me be explicit that network layer topic *are* within the scope of this effort. I expect to see lots of content to be from this area. =================================================== a. Identify and describe the problems related to TCP performance in satellite environments. In addition, we wish to provide some validation as to the level to which the problem is understood. This will help sort the areas where engineering is appropriate/possible and those areas that require more research. A first cut at an ordered categorization of problem *areas* is as follows: 1. Large Bandwidth * Delay Product 2. Long Feedback Delay Path 3. Asymmetric Channels 4. Errors Before anyone starts complaining about this, there *are* environments where you get error-rates worse than 10E-8 *after* application of strong FEC; 5. ??? b. Identify, describe and *assess* the applicability of currently available (or proposed) mitigation techniques keyed to their associated problem areas. c. Tag those areas where engineering will not suffice as areas promising for research. d. For each specific issue identified: 1. Problem Statement: 2. Semi-detailed explanation of the problem 3. Mitigation techniques identified 4. Discussion of mitigation techniques a. 1st Order discussion on technique (isolated, no interactions) Assess the benefits and potential pitfalls of this technique keyed to the different topologies provided in section 2. This is will prove to be important. b. If applicable, tag this area as one needing further research. IV. Interacting Mitigations This section should *attempt* to address the potential impacts of mixing the different mitigation techniques identified/discussed above. V. Areas Requiring Further Engineering Pull together those areas identified in Section 3 that can be handled through engineering, but which fall out of scope of this group. V. Identified Research Areas Collect all the tagged research areas identified in as requiring further research. VI. Recommendations ------------------------------ End of tcp-over-satellite-digest V1 #16 *************************************** tcp-over-satellite-digest Friday, April 25 1997 Volume 01 : Number 017 In this issues: Repost: Memphis Action Items and Schedule of Events Re: REMINDER: Research/Work Outlines Due Tomorrow!! What I did during my summer vaction Re: Definition of High Bandwidth Delay Product Re: Definition of High Bandwidth Delay Product Outline of work - Teledesic Re: REMINDER: Research/Work Outlines Due Tomorrow!! Re: REMINDER: Research/Work Outlines Due Tomorrow!! Re: REMINDER: Research/Work Outlines Due Tomorrow!! the paragraph(s) requested by Aaron REPORT on activities: EUMETSAT Research at Ohio University Research at Texas A&M University Re: REMINDER: Research/Work Outlines Due Tomorrow!! Re: Definition of High Bandwidth Delay Product Re: TCP/SATCOM draft suggestion TCP Research at NLM Re: Definition of High Bandwidth Delay Product See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Thu, 24 Apr 1997 14:51:41 -0400 (EDT) From: Eric Travis Subject: Repost: Memphis Action Items and Schedule of Events - ---------- Forwarded message ---------- Date: Mon, 14 Apr 1997 22:34:20 -0400 (EDT) From: Eric Travis To: tcp-over-satellite@achtung.sp.trw.com Subject: Memphis Action Items and Schedule of Events As with any good meeting, we managed to generate a tentative schedule and even a pair of action item in Memphis. Action Items: ============= 1. Within the next two weeks (ending April 30), it is requested that people post to the mailing list a brief summary of their related work/research. Pointers to papers and web-sites are encouraged. 2. Within the next two weeks (ending April 30), we are soliciting initial inputs to the Informational RFC. The draft outline discussed in Memphis will be posted in a subsequent message. Initially important elements are content and concepts, not polished prose. These initial contributions will help us shape the scope and depth of the document, so please try and provide what you can. Contributing does *not* sign anyone up for further commitment. During the first week or so of May, individuals will be solicited to take responsibility for major chunks of content. Obviously those who express an interest will be excellent candidates, but don't be surprised if you get asked anyway :o) Saying no is OK, I understand having other commitments. There is a difference between contributors and volunteers; You can contribute without committing. Tentative Schedule ================== April 11, 1997 Begin! April 30, 1997 Initial inputs due to the mailing list (don't stop after April 30!) May 9, 1997 Solicitation of volunteers June 30, 1997 Draft 1 to be sent to mailing list July 31, 1997 Draft 2 to be sent to mailing list August 11-15, 1997 IETF Munich September 1, 1997 Draft 3 to be sent to mailing list/posted to web [Fine scheduling for the period between September and December TBD] October 6, 1997 Draft 4 to be sent to mailing list/posted to web November 3, 1997 Draft 5 to be sent to mailing list/posted to web Late November, 1997 Final Draft December 1997, IETF DC (?) Finish! ------------------------------ Date: Thu, 24 Apr 1997 12:21:25 -0700 From: adrian.hooke@jpl.nasa.gov (adrian hooke) Subject: Re: REMINDER: Research/Work Outlines Due Tomorrow!! Aaron: In response to your request "to post a brief outline of your activities so that the larger community could learn about what you are doing", here's what we are doing in NASA with our Space Communications Protocol Standards (SCPS) project. We are attacking the problem of communicating between an end system on the ground and another onboard a remote spacecraft. The two end systems are interconnected by a variety of subnets, at least one of which contains a long-delay/weak-signal space link. Since the space end system is in a hostile environment, it also tends to be relatively computationally challenged compared with its ground peer. Recognizing this weird environment, our goal is to provide a suite of standard data handling protocols which nevertheless (from a user viewpoint) make the remote space vehicle appear to be "just another node on the Internet". The SCPS protocols include: 1. A file handling protocol (the SCPS File Protocol, or SCPS-FP), optimized towards the up-loading of spacecraft commands and software, and the downloading of collections of observational data. The SCPS-FP is based on FTP. 2. An underlying retransmission control protocol (the SCPS Transport Protocol, or SCPS-TP), optimized to provide reliable end-to-end delivery of spacecraft command and telemetry messages between computers that are communicating over a network containing one or more space data transmission paths. The SCPS-TP is based on TCP/UDP. 3. A data protection mechanism (the SCPS Security Protocol, or SCPS-SP) that provides the end-to-end security and integrity of such message exchange. The SCPS-SP is derived from the Secure Data Network (SDNS) SP3 protocol, the ISO Network Layer Security Protocol (NLSP), the Integrated Network Layer Security Protocol (I-NLSP), and the IETF Internet Protocol Security (IPSEC) Encapsulating Security Payload (ESP) and Authentication Header (AH) protocols. 4. A scaleable networking protocol (the SCPS Network Protocol, or SCPS-NP) that supports both connectionless and connection-oriented routing of these messages through networks containing space data links. The SCPS-NP is based on IP, with modifications to support increased communications efficiency and new routing needs through constellations of orbiting spacecraft. The SCPS are being developed as a joint project between NASA and the Department of Defense and are being progressed towards international standards via the program of work of the international Consultative Committee for Space Data Systems (CCSDS). The latest draft specifications are currently in final editing and should be available on 97-04-28 via a new SCPS Web page; I'll send out the locator early next week when they are ready. Best regards, Adrian J. Hooke Manager, NASA Space Mission Operations Standardization Program Jet Propulsion Laboratory, California Institute of Technology M/S 301-235, 4800 Oak Grove Drive Pasadena, California 91109-8099, USA. Tel +1.818.354.3063 Fax +1.818.354.9068 Pager 800.759.8888 PIN 1109388 Email: adrian.hooke@jpl.nasa.gov ------------------------------ Date: Thu, 24 Apr 1997 15:38:17 -0400 (EDT) From: Eric Travis Subject: What I did during my summer vaction [ I posted the action items, so...] I've spent the past several years working on the joint NASA/DOD (SCPS) Space Communications Protocol Standards effort. Loosely, this program is chartered with specifying a standard suite of communications protocols (network, security, transport and file transfer) for operation in resource constrained, "stressed" environments (including terrestrial deployment). If you map SCPS == Stressed Communication Protocol Standards, that's legal. My involvement in this has given me an unconventional perspective on things. The amazing thing is, despite that the number of organizations involved actually exceeds the number of people, we still get cool stuff done. It's certainly a pleasant first for me! My primary involvement revolves around the development of the SCPS Transport Protocol (SCPS-TP) and applying chewing gum & borrowed parts to the SCPS testbed; SCPS-TP is based upon, and is interoperable with, TCP. For the most part, We are TCP a few environment specific enhancements/tweaks and a several new options. Because we allow configurations that do "non-standard" things with TCP's congestion avoidance/control mechanisms, we can't rightfully call the result TCP anymore. Some of the optional enhancements are: o rate-based flow control o modify (or eliminate) ack-clocking for highly asymmetric channels (2000+:1) o allow the default response to loss be something other than congestion o separate responses to congestion, corruption and link-outages o provide a Selective Negative Acknowledgment (SNACK) o provide a partial-reliability transport service in addition to TCP's fully reliable transport service. This is a negotiated service that provides data in sequence, but not necessarily completely. This allows the application to specify how "hard" to attempt to deliver the data. Gaps in the data are reported to the receiving application. o other stuff We've also had to do other "interesting" things in terms of implementation for hosting in constrained (processing capacity, memory, etc.) systems. Testing has been conducted via emulation in the lab, over live bent-pipe tests and we have even had the SCPS suite hosted on the DRA STRV-1b spacecraft. It's been a very cool project in terms of the people involved and the technical challenges. We've learned a lot about operating transport protocols (as well as network, security and application protocols) in the space environment, much of it relevant to this effort. I hope we can use our experience to help out here. [If you want more information on SCPS, please contact me *off-line* and I'll point you in the right direction for ^^^^^^^^ the aspect you are interested] Eric SCPS people might be verbose, but our protocols aren't :o) ------------------------------ Date: Thu, 24 Apr 1997 16:48:34 -0400 From: Daniel R Glover Subject: Re: Definition of High Bandwidth Delay Product At 11:20 AM 4/24/97 -0400, you (matthew halsey) wrote: >One other important thing to remember is that the RTT in the bandwidth-delay >product is the end-2-end RTT. Therefore, whilst a satellite may incur close >to 600ms RTT (unless you are exactly at the sub-satellite point, on the >equator, talking to yourself!) Yes, the best case propagation time for one hop (talking to yourself at the equator) is 240 msec and the RTT would be 480 msec. In addition, there may be a few msec of processing delay (coding, etc.). The worst case propagation delay for one hop is 279 msec with RTT of 558 msec. A good number to use for a typical GEO satellite propagation time is probably 270 msec/hop for 540 msec RTT. >you may experience up to 100 ms of >terrestrial delay, due to either congestion and/or routing delays, on either >side. This takes the rtt up to 800 ms. Of course this varies upon >configuration and I am assuming a satellite connecting two terrestrial >networks. I like the idea of something called "routing delay" to add to the propagation delay for comparison purposes. For example, on a coast-to-coast fiber, the RTT propagation delay would be something like 30 msec. A comparison value for terrestrial RTT would then be 30 + 100 = 130 msec vs. a satellite RTT of 540 + 100 = 640 msec. I was using 100 msec and 500 msec respectively as nice round numbers. Using 130 msec and 640 msec, the example paragraph could read: For a T1 connection of 1.544 M bits/sec, a high bandwidth*delay product would occur with a round-trip delay greater than around 340 msec. For a T1 geosynchronous satellite connection with a total delay (propagation + routing) of, say, 640 msec, the bandwidth*delay product is 988,160 bits or 123,520 bytes which is high by the definition above (>65535 bytes) and falls in the LFN category. For a terrestrial T1 connection with 130 msec round-trip time the bandwidth delay product is 25,090 bytes which is not "high." However, this description should have made clear what the assumed network topology is. This will be possible when the architecture diagrams are available. We should come up with typical delay ranges for the different topologies. The satellite with terrestrial networks on both ends is not the typical case today. The satellite is probably jumping over a bunch of routers in a coast-to-coast link and in that case should use a lower value than 100 msec for the routing delay. In some of our ACTS experiments with Hughes (Cleveland to Malibu), RTTs were in the 550-580 msec range which included a couple of switches and routers on each end. We need to add another example to demonstrate that as bandwidth increases, even terrestrial systems will have high B*D products. For example, a 155 Mbps connection reaches a high B*D product at a little over 3 msec RTT. It's important for us to get the definitions and examples straight, but I don't think window size is the main problem we face, so we don't want to go overboard on this in the draft. >Therefore with a 65,536 Byte buffer available and 800ms RTT, the throughput >cannot exceed >65536 x 8 / 0.6 = 874 kbps. >Now the problem comes when people use TCPs who don't allow this much receive >window buffer. >An 8192 Byte buffer means a througput of 81.92 kbps per single TCP session. > >I also feel that this should be spelled out in the document. > I agree in general. There is going to be an architectures section which describes various network topologies. Something we may want to add there is a multiple-hop scenario (that will get the RTT up). That may be covered by a "combination" category which is any combination of the basic architectures that Eric showed in Memphis. R/ Dan /************************** Dan Glover dglover@lerc.nasa.gov NASA Lewis Research Center MS 54-2 Cleveland, OH, USA 44135 (216)433-2847 **************************/ ------------------------------ Date: Thu, 24 Apr 1997 14:18:29 -0700 From: Fred Baker Subject: Re: Definition of High Bandwidth Delay Product At 4:48 PM -0400 4/24/97, Daniel R Glover wrote: >I like the idea of something called "routing delay" to add to the >propagation delay for comparison purposes. I'd like you to not use the word "routing", for political reasons: folks have a strange idea that routers don't switch traffic, and ask "would a switch be different?" grump... generally, we talk about "store-and-forward delay", "queuing delay", and "propagation delay." Measured propagation delay coast to coast is generally reported as about 70-90 ms on commercial grade fiber, but is generally neglected from more local calculations, especially at speeds on the order of T-1 or lower. store-and-forward delay in a router depends a lot on its implementation, but is by definition the amount of time that elapses from the receipt of the first bit of a message (ie, when we start to store it) to its presentation to another interface for queuing (when the software thinks it was transmitted). This is simplistically message size/incoming interface rate + (constant * stochastic function) first term is clearly how long it takes to store the message, and the second term is how long it takes to look up the route cache entry, fiddle with the bits and call a subroutine. In a cut-through switch, such as ATM, the message size is the size of a cell, while in a packet switch, it is the size of a packet. queuing delay is a stochastic function of an arrival distribution, arrival rate, and departure rate. A standard Poisson M/M/1 or M/D/1 model generally describes this pretty well, although it will be somewhat pessimistic. The sum of these distributions on a path from here to there is obviously not Poisson; there's a body of research that suggests that network traffic obeys Chaos Theory - is self-similar - in at least some instances, and that there are so many flows in the backbone that these flows converge to a surging steady-ish state in the backbone. RED is supposed to help make that steady-ish state more steady on average, and WFQ/SFQ is supposed to deterministically make it very steady. propagation delay includes all of the time it takes a bit to go from one serial interface to the next one, which may include, as you say, encode and decode delays as well as signal propagation. It also includes hidden overheads, such as the extra bit times implied by HDLC and FEC bit stuffing, internal framing bits, and the like. I think the most general way to describe all this is to say that, on a typical internet path (which may well transit twenty routers) the sum of the delays approximates 100 ms, and when you introduce a satellite, you are raising the propagation delay of one of the links en route to something on the order of 270 ms. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The surest sign that intelligent life exists elsewhere in the universe is that it has never tried to contact us. Calvin and Hobbes (Bill Watterson) ------------------------------ Date: Thu, 24 Apr 1997 14:36:13 -0700 From: Marie-Jose Montpetit Subject: Outline of work - Teledesic At Teledesic, the Network team is investigating TCP-over-satellite. Our > work makes an attempt to create a satellite system that seemlessly works with TCP implementations that were built for common environments, >including current terrestrial networks. As such, for example, we attempt to understand all the parameter space including bandwith and delay requirements and their future development. We are however focusing on a global, high performance, low latency network and work at maintaining low latency across our network. Dr. Marie-Jose Montpetit Senior Member of the Technical Staff Teledesic Corporation 2300 Carillon Point Kirkland, WA 98033 marie@teledesic.com >www.teledesic.com ------------------------------ Date: Thu, 24 Apr 1997 14:50:20 -0700 (PDT) From: Cheryl DeMatteis Subject: Re: REMINDER: Research/Work Outlines Due Tomorrow!! Quoting Aaron Falk: > > Folks- > > As you may recall, I asked those of you who are doing TCP-over-satellite > related work to post a _brief_ outline of your activities so that the > larger community could learn about what what you are doing. > > Remember that our groups lifetime is very short. I hope that you can > support the Friday deadline for submission of your research summaries. > > Thanks, > > Aaron > > -- > Aaron Falk (310) 814-4932 > TRW, Inc > Electronics Systems & Technologies Division > afalk@mailsrv1.trw.com > > - -- Cheryl K. DeMatteis The Aerospace Corporation, M1/102 cdematt@aero.org Computer Science and Technology (310)336-1189 2350 E. El Segundo Blvd. (310)336-4402 (fax) El Segundo, CA 90245-4691 ------------------------------ Date: Thu, 24 Apr 1997 15:02:11 -0700 (PDT) From: Cheryl DeMatteis Subject: Re: REMINDER: Research/Work Outlines Due Tomorrow!! (Sorry for the previous blank message) We are working on a Darpa funded project -- Battlefield Awareness Data Dissemination (BADD). The architecture consists of one uplink/sender (so far) and multiple downlinks/receivers using a GBS Satellite. The satellite is a unidirectional broadcast medium. Many of the "routing" issues are being dealt with in the Unidirectional Routing WG (udlr) but I wanted to mention that there are various types of applications that will be utilizing this network. The range of applications consist of database transactions, web browsing, email and transferring large graphic images. - -- Cheryl K. DeMatteis The Aerospace Corporation, M1/102 cdematt@aero.org Computer Science and Technology (310)336-1189 2350 E. El Segundo Blvd. (310)336-4402 (fax) El Segundo, CA 90245-4691 ------------------------------ Date: Thu, 24 Apr 1997 16:01:47 -0700 (PDT) From: Renaud Bruyeron Subject: Re: REMINDER: Research/Work Outlines Due Tomorrow!! We are currently experimenting with TCP Sack over long delay links. We have built a testbed that uses PC's running freeBSD as end hosts and a Sun workstation with large amount of memory as a router which can imitate long delays and random loss patterns. From the hosts point of view, the communication channel looks like a LFN. The issues to be investigated include the effectiveness of TCP SACK under various delay and loss conditions, the impact of TCP SACK connections on other TCP connections when both share the same bottleneck point (e.g. would TCP connections with SACK option take much more bandwidth from the non-SACK connections?), and the limitation of the TCP option field length (40 bytes max). At most each TCP acknowledgment can carry 4 SACK blocks; if the timestamp option is used, it is reduced to 3 blocks; if another option is added, the number of SACK blocks drops down to 2. We will investigate the consequences of the limited Sack blocks with large windows, and explore possible improvement to the current SACK specification. - -------------------------------------------------------------------------- TCP Sack Project Internet Research Lab Computer Science Department at UCLA Team : Renaud Bruyeron bruyeron@cs.ucla.edu Bruno Hemon bruno@cs.ucla.edu Advisor : Lixia Zhang lixia@cs.ucla.edu - -------------------------------------------------------------------------- ------------------------------ Date: Thu, 24 Apr 1997 21:01:28 -0400 From: Tim Shepard Subject: the paragraph(s) requested by Aaron Aaron wants a paragraph or two describing what each of us on the tcp-over-satellite mailing list are working on. Here's ours: With funding from NASA (with program managed by Dan Glover at LERC), Craig Partridge and I are working on an investigation into what changes might need to be made to TCP and its congestion-control-related algorithms to improve its performance over paths which may have high-delay satellite links. Our goal is not to produce any satellite-specific solutions. Our goal is "a better TCP" which operates well over wider ranges of delay-bandwidth product and delay (to include one or two GEOS hops) than does the usual Tahoe or Reno TCPs. This would allow seamless integration of satellite links into the net in cases where perhaps neither of the systems at the TCP endpoints has any opportunity to be aware of the existence of a satellite hop on the path (other than the larger round-trip delays which it experiences). I feel that eighty percent of the pieces necessary to accomplish this are already out there. (E.g. RFC1323, RFC2018, the FACK work by the folks @psc.edu, etc.) We just need to figure out exactly what is the required set of pieces, and which pieces (if any) are missing, we need to make sure that no harm will be done in the area of congestion, and then we need to convince everybody who implements TCP that they should do the right thing (!) and incorporate all our recommended pieces. This is definitely research (at this point in time), and we aren't the only ones pursuing this. This work needs to be done for the Internet in general if it is to be able to effectively support single-user high-rate continent-crossing or ocean-crossing terrestrial links (where, for example, someone who has paid for a dedicated continent-crossing OC-12c expects to be able to move their gigabytes quickly using applications which are built on top of TCP). A T1 or DS3 over a GEOS is just the same problem, modulo a scaling in time (in my opinion). -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Fri, 25 Apr 1997 14:37:04 +0100 From: Frank Zeppenfeldt Subject: REPORT on activities: EUMETSAT Here our contribution to the tcp-over-satellite inventarisation of current research and field tests: CONTEXT We are in the beginning of building up a groundsegment for MSG (Meteosat Second Generation) in Darmstadt, Germany. One of the issues we have looked in, is the delivery of real-time data from a remote groundstation to the processing center using satellite links. This would mean a 24 hrs/365 days TCP session of 4 Mbps over a satellite hop. We decided to prototype this. PROTOTYPE ACTIVITIES We have tested using the configuration: workstation10BaseT-----cisco4700--serialRS449---satellite simulator----serialRS449---cisco4700------workstation10BaseT The satellite simulator was the AdTech SX12 that simulates PDH/RS-449 up to 8 Mbps, and allows an flexible way to program errors/bursts etc. It was for us also an experiment to see if the CISCO synchronous ports can be clocked up to 4 Mbps. PPP as link layer, no compression. OS TESTED The following operating systems supporting RFC1323 have been tested: Digital UNIX 3.2C Digital UNIX 4.0 IRIX 5.3 Solaris 2.5 with tcp-lfn Solaris 2.4 with tcp-lfn (both the free one and the SUN Consulting one) FreeBSD 2.1.5 HP UX 10.01/10.10 All tests have been done symmetrically, e.g no SUN to Alpha tests. For us only Digital UNIX was the only exceptable candidate, with respect to fast retransmit, slow start and behaviour to two bit errors in one window. We have been looking not only to pure performance behaviour but also: - - if we need to go to our backup link that could be terrestrial, can we still use the same socket SEND/RECV buffers ? Normally not, as the CISCO queues cannot hold this:the RTT changes, terrestrial link is not always garantueed higer that 10E-7,... - - we cannot assume a error free link: the contractual baseline for buying a link is 10E-8, we need something that uses, say 90% of the channel bandwidth when we get 10E-8. - - (... lots of other small things that happen in the operational life of a link to a groundstation) FUTURE ACTIVITIES Will depend on the chosen implementation of the link to the groundstation. The full report will be available in some time. I will post to this list. Best regards, Frank =================================================== ================================= Frank Zeppenfeldt Ground Station Network Engineer zeppenfeldt@eumetsat.de tel: +49 6151 807351 fax: +49 6151 807426 EUMETSAT Am Kavalleriesand 31 64295 Darmstadt Germany =================================================== ================================= ------------------------------ Date: Fri, 25 Apr 1997 11:49:51 -0400 From: Mark Allman Subject: Research at Ohio University Here is a quick outline of the work being done at Ohio University regarding TCP over Satellite by Mark Allman, Chris Hayes, Hans Kruse and Shawn Ostermann. Our research has been focused on TCP, as well as, application-level protocols over satellite links. Our application-level results and preliminary TCP results are available at the URL given below. We are currently investigating recent proposals for new TCP mechanisms including TCP with larger windows, PAWS, etc. (RFC 1323), TCP with SACK (RFC 2018), more aggressive slow start, forward acknowledgements (FACKs), Janey Hoe's SIGCOMM proposals, and HTTP 1.1. We are testing these mechanisms in hardware and software emulators, as well as, over NASA ACTS. http://jarok.cs.ohiou.edu/projects/satellite ------------------------------ Date: Fri, 25 Apr 1997 12:23:26 -0500 (CDT) From: Biaz Saad Subject: Research at Texas A&M University Here is a brief summary of our research on TCP at Texas A&M Univ. We do not work specifically on TCP over satellite. We consider TCP over wireless links. Our main concern is the discrepancy of the bit error rate between wired links and wireless links. On a wired network, the assumption that most of the packet losses are due to congestion is fully justified. But, on a path with wireless links and/or satellite links, this assumption is no more valid. TCP (1988) Tahoe reacts to all losses as congestion, triggering slow-start. TCP Reno with Fast retransmit and Fast recovery, can modulate this response and avoid slow-start. But, TCP is still vulnerable to corruption bursts which can occur on wireless links. Our research has two goals: (i) to design techniques (additions to TCP) to distinguish the congestion losses from corruption losses. (ii) to design techniques that improve TCP performance in presence of errors, without explicitly knowing whether the loss is due to errors or not. For more information on our work, see http://www.cs.tamu.edu/faculty/vaidya/mobile.html ------------------------------ Date: Fri, 25 Apr 1997 13:48:32 -0400 From: Curtis Villamizar Subject: Re: REMINDER: Research/Work Outlines Due Tomorrow!! In message <335F7F7E.8645B120@trw.com>, Aaron Falk writes: > Folks- > > As you may recall, I asked those of you who are doing TCP-over-satellite > related work to post a _brief_ outline of your activities so that the > larger community could learn about what what you are doing. > > Remember that our groups lifetime is very short. I hope that you can > support the Friday deadline for submission of your research summaries. > > Thanks, > > Aaron > > -- > Aaron Falk (310) 814-4932 > TRW, Inc > Electronics Systems & Technologies Division > afalk@mailsrv1.trw.com Aaron, I'm not sure if you want to include this since there is no full time staff dedicated to it. This is a secondary priority for me, and could end up getting delayed (approaching indefinitely). So far I have finished most of the tail drop simulations and produced plots of goodput and loss (but not fairness) vs queue capacity for tail drop. Curtis ANS is a major provider of Internet service in the US with a growing presence in Europe, the Pacific Rim, and elsewhere. As the provider of the T3 NSFNET backbone network service from 1991-1995, ANS has for a very long time been interested in improving performance over higher delay bandwidth product paths primarily by altering the behavior of network elements such as routers or switches. As a major Internet service provider now, this interest continues. Performance simulations have recently been pursued as a means to better understand TCP performance in large scale networks such as ANSNET. This work is ongoing. The simulations will concentrate on: high delay bandwidth products very large number of source and destination pairs multiple bottlenecks traffic distribution with a mix of short and long duration flows examine the effects of buffer size on goodput, fairness, loss examine effects of tail drop vs SFQ, RED, EPD, SFQ and RED goals: improving goodput and fairness, reducing loss TCP is regarded to be the protocol of greatest interest due to its overwhelming dominance of Internet traffic. Relying on widespread deployment of improvements to host implentations of TCP is not considered viable. This is does not mean that these improvement are not of interest. They should clearly be encouraged, particularly as a means to improve the performance of applications which are more sensitive to performance, such as high bandwidth applications. It is important that whatever recommendations are made (by this work or by any similar work) to improve either the behavior of network elements of the behavior of host implementations interacts well with the existing deployed base of host TCP, and works well in all conditions under which TCP is expected to operate, including very low bandwidth links, very high bandwidth, and very long delays, and including long duration high bandwidth flow, short duration flows and large aggregations of flows. Sacrificing performance in one set of conditions for the benefit of another is not acceptable. For this reason, simulation conditions will include very long delays such as those experienced with satellite links. ------------------------------ Date: Fri, 25 Apr 97 11:23:13 PST8 From: vbarajas@CCGATE.HAC.COM Subject: Re: Definition of High Bandwidth Delay Product Fred and Dan (et. al..) I commiserate with your concern about the label "routing delay". But my problem is the "popular agreement" that Internet delay is between 70 -90 msec. My concern is what our experiments (HRL-NASA ACTS) showed when we simulcasted an MBONE multicast over a terrestrial link and satellite. The two paths [origination point Cleveland (NASA Lewis Res. Ctr.) reception point Hughes Res. Labs Malibu] when viewed at HRL exhibited similar amounts of delay. This sort of flies in the face of the recent "GEO BAD, LEO GOOD" press releases that we've been subjected to lately. Clearly we need to do more work to understand the mechanisms that are contributing to the terrestrial propagation delay. .......victor barajas ------------------------------ Date: Fri, 25 Apr 1997 14:20:09 -0400 From: Curtis Villamizar Subject: Re: TCP/SATCOM draft suggestion In message , Eric Travis wr ites: > > There is a perception that the sole driver of this is HTTP > traffic, but as Karen illustrates and Fred has reminded us > on several occasions, there are folks without rich > connectivity to CONUS (and no fiber) need to do > *everything* over their satellite link(s). FTP is usually a longer bulk transfer and there not an expectation for a click and a response in a few seconds as there is with HTTP. SMTP and NNTP doesn't matter unless it gets so slow that it can't get through at all since its not interactive. It helps if you don't put a POP client and server on opposite ends of a satellite link or put an NNTP client and server and try to do NNTP inveractively over satellite. Telnet and rlogin over a long delay is always going to be painfull and isn't anything that can be done except use faster light. :-) HTTP was mentioned because it is a particularly tough case and may be more in need of improvement than others, not that it is thought to be the only service of importance. Curtis btw- NFS over satellite is probably a real bad idea. :-) ------------------------------ Date: Fri, 25 Apr 1997 15:08:05 -0400 From: long@nlm.nih.gov (Rodney Long) Subject: TCP Research at NLM Below is a description of TCP research in the Communications Engineering Branch at the National Library of Medicine. Our work is applications-oriented, in the sense that the goal in all of our work is to learn the technology for the purpose of applying it to the delivery of biomedical information over wide-area networks. We have focused particularly on the delivery of database information consisting of both text and biomedical images and have developed prototype client/server applications which we use in our performance testing. As part of this goal, we have done work in developing experimental approaches to the use of TCP, including the use of multiple parallel connections (which we call "multisocket" TCP). We have collected experimental data on the use of this approach over a T-1 NASA ACTS satellite link, and over a few Internet test sites. We also installed an RFC 1323 TCP over this link and collected data. At the present we are experimenters on the NASA High Data Rate terminals and have begun collecting performance data for running TCP over ATM over the ACTS satellite with this interface. We are running with an RFC 1323 TCP from Sun Microsystems. We have published several conference papers discussing our work. One which discusses our satellite work in particular is Long, LR, et. al. High-speed satellite access to bio-medical text/image databases. Advanced Digital Libraries '96 Forum on Research and Technology Advances in Digital Libraries, Library of Congress, Washington, D.C., May 13-15, 1996. Rodney Long Electronics Engineer Communications Engineering Branch National Library of Medicine voice 301 435-3208 fax 301 402-0341 ------------------------------ Date: Fri, 25 Apr 1997 12:29:38 -0700 From: Fred Baker Subject: Re: Definition of High Bandwidth Delay Product At 11:23 AM +0000 4/25/97, vbarajas@CCGATE.HAC.COM wrote: >I commiserate with your concern about the label "routing delay". But my >problem is the "popular agreement" that Internet delay is between 70-90 msec. well, remember that what I said was not that the *internet delay* is 70-90 ms, it was that > Measured propagation delay coast to coast is generally > reported as about 70-90 ms on commercial grade fiber I'll tell you where people get measurements of *internet* delay: coast to coast traceroutes. For example, a traceroute from my Sun in San Jose to MIT in Boston yields: red-ss20(5): traceroute mit.edu traceroute to mit.edu (18.72.0.100), 30 hops max, 40 byte packets 1 eng-ios-f-5.cisco.com (171.69.58.65) 1 ms 1 ms 1 ms 2 eng-ios-2.cisco.com (171.69.62.130) 2 ms 1 ms 1 ms 3 sj-eng-corp1.cisco.com (171.69.5.10) 2 ms 2 ms 2 ms 4 sj-wall-2.cisco.com (198.92.1.138) 2 ms 2 ms 2 ms 5 barrnet-gw.cisco.com (192.31.7.37) 3 ms 3 ms 3 ms 6 su-pr2.bbnplanet.net (131.119.26.9) 4 ms 4 ms 5 ms 7 paloalto-br1.bbnplanet.net (131.119.0.193) 63 ms 116 ms 7 ms 8 chicago1-br1.bbnplanet.net (4.0.1.2) 106 ms 362 ms 58 ms 9 chicago1-br2.bbnplanet.net (4.0.1.194) 243 ms 245 ms 61 ms 10 boston1-br1.bbnplanet.net (4.0.1.126) 363 ms 99 ms 128 ms 11 boston1-br2.bbnplanet.net (4.0.1.182) 424 ms 82 ms 75 ms 12 cambridge2-br1.bbnplanet.net (4.0.1.186) 78 ms 78 ms 79 ms 13 EXTERNAL-RTR-IHTFP-FDDI.MIT.EDU (192.233.33.3) 77 ms 75 ms 77 ms 14 E40-RTR-FDDI.MIT.EDU (18.168.0.11) 78 ms 76 ms 76 ms 15 MIT.MIT.EDU (18.72.0.100) 77 ms 75 ms 77 ms There are clearly a couple of numbers in there above 100 ms, but the average of the times on the last six lines is 116 ms. My guess is that none of the links in the list is below DS3 (the ones inside Cisco will be FDDI, and the ones in the backbone are either FDDI, DS3, or OC-3), so this won't take into account low speed (T-1) lines at the periphery. You can expect them to add some tens to hundreds of milliseconds to the mean delay, as in this traceroute taken from my home (I have a 128 KBPS Flame Delay line to Cisco): traceroute to mit.edu (18.72.0.100), 30 hops max, 40 byte packets 1 fred-cisco-fr.cisco.com (171.69.128.113) 2 ms 1 ms 1 ms 2 eng-tc-2.cisco.com (171.69.121.32) 57 ms 58 ms 59 ms 3 sj-eng-cc4.cisco.com (171.69.121.2) 62 ms 65 ms 63 ms 4 sj-eng-corp1.cisco.com (171.69.5.10) 60 ms 71 ms 58 ms 5 sj-wall-2.cisco.com (198.92.1.138) 56 ms 73 ms 62 ms 6 barrnet-gw.cisco.com (192.31.7.37) 72 ms 70 ms 59 ms 7 su-pr2.bbnplanet.net (131.119.26.9) 66 ms 65 ms 67 ms 8 paloalto-br1.bbnplanet.net (131.119.0.193) 650 ms 398 ms 297 ms 9 chicago1-br1.bbnplanet.net (4.0.1.2) 116 ms 113 ms 113 ms 10 chicago1-br2.bbnplanet.net (4.0.1.194) 120 ms 128 ms 120 ms 11 boston1-br1.bbnplanet.net (4.0.1.126) 136 ms 140 ms 132 ms 12 boston1-br2.bbnplanet.net (4.0.1.182) 134 ms 137 ms 136 ms 13 cambridge2-br1.bbnplanet.net (4.0.1.186) 132 ms 150 ms 136 ms 14 EXTERNAL-RTR-IHTFP-FDDI.MIT.EDU (192.233.33.3) 135 ms 141 ms 133 ms 15 E40-RTR-FDDI.MIT.EDU (18.168.0.11) 136 ms 138 ms 138 ms 16 MIT.MIT.EDU (18.72.0.100) 138 ms 139 ms 135 ms the mean of the last six lines is 137 ms. You ask me what my delay across the internet is going to be, and I will report to you that I did such a traceroute from my home to about 400 different addresses (extracted from the internet drafts directory), and the delays I experienced averaged 125 milliseconds with a 125 millisecond standard deviation. >Labs Malibu] when viewed at HRL exhibited similar amounts of delay. This sort >of flies in the face of the recent "GEO BAD, LEO GOOD" press releases "LEO good, GEO bad" is drivel, pure as the driven snow, as we all know. The entire question leaves me muttering under my breath about newspaper reporters, marketing folks, and other people for whom first grade would be a step up. But it's not worth getting too upset, because getting upset won't change anything. Documenting reality might, though. Terrestrial of various types, GEO and LEO are different technologies with different trade-offs. In each, there are good reasons to use it, and good reasons to pick anything but. From my humble perspective, if we can somehow mitigate latency in satellite links, we will have done a good thing. If we can then transfer what we have learned to other kinds of networks, we will have done better. But right now, I'm only worried about the former problem. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The surest sign that intelligent life exists elsewhere in the universe is that it has never tried to contact us. Calvin and Hobbes (Bill Watterson) ------------------------------ End of tcp-over-satellite-digest V1 #17 *************************************** tcp-over-satellite-digest Saturday, April 26 1997 Volume 01 : Number 018 In this issues: Re: REMINDER: Research/Work Outlines Due Tomorrow!! Re: Definition of High Bandwidth Delay Product Re: TCP/SATCOM draft suggestion NASA/GSFC TCP to Satellite research work outline Re: Definition of High Bandwidth Delay Product Re: Definition of High Bandwidth Delay Product Satellite research at Hughes Research Labs (HRL) Re: TCP/SATCOM draft suggestion See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Fri, 25 Apr 97 15:56:54 EST From: "Vincent Chin" Subject: Re: REMINDER: Research/Work Outlines Due Tomorrow!! Arron, I am working on a messaging system for the DoD. One of my involvement is on satellite communication to users in tactical areas. Basically the environment is X.400 messages over TCP using satellite links. I am very new in this area and have been learning a lot just reading the t-o-s emails. Vince. Vincent Chin Defense Information Systems Agency DISA/JIEO Ft. Monmouth, NJ 07703 email: chinv@ftm.disa.mil Voice: 908-427-6849 FAX: 908-427-6798 DSN: 987 ______________________________ Reply Separator _________________________________ Subject: REMINDER: Research/Work Outlines Due Tomorrow!! Author: afalk@emu.sp.trw.com at smtp-ftm Date: 4/24/97 11:47 AM Folks- As you may recall, I asked those of you who are doing TCP-over-satellite related work to post a _brief_ outline of your activities so that the larger community could learn about what what you are doing. Remember that our groups lifetime is very short. I hope that you can support the Friday deadline for submission of your research summaries. Thanks, Aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technologies Division afalk@mailsrv1.trw.com ------------------------------ Date: Fri, 25 Apr 1997 13:26:49 -0700 From: touch@ISI.EDU Subject: Re: Definition of High Bandwidth Delay Product > From: Daniel R Glover > At 09:24 AM 4/24/97 +0000, you (David Mercer) wrote: > >Can anybody put a number on what defines a "High Bandwidth Delay > >Product"? > > > The bandwidth*delay product is a measure of the capacity of a link and is > defined as the product of the bandwidth and the round-trip time. It also > corresponds to the amount of buffering required (window size) at the ends of > the "pipe" for maximum utilization of a TCP connection. The maximum window I would use "bandwidth-delay product" or "bandwidth*delay"; because the use of '*' and 'product' is redundent. Also, this is more than a TCP issue. It is the communication latency between endpoints, and thus buffering equal to the comm latency is required for any direct feedback mechanism, of which TCP's ACKs are only one scheme. I would NOT define 'high' in absolute terms of TCP's current implementation, for this reason. I would define 'high' in terms that have impact. Buffering can always be increased by modifying the protocol. However, this presumes infinite offered load (a very large file to send). I would define 'high' as 'near the size of the information of an association. I.e., high = as large as the file you're transmitting. BWD's in the terabyte range are no problem, unless you have only 1 Mbyte of data to send. Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: Fri, 25 Apr 1997 15:54:01 -0400 (EDT) From: Eric Travis Subject: Re: TCP/SATCOM draft suggestion > > > > There is a perception that the sole driver of this is HTTP > > traffic, but as Karen illustrates and Fred has reminded us > > on several occasions, there are folks without rich > > connectivity to CONUS (and no fiber) need to do > > *everything* over their satellite link(s). > I hope you didn't misinterpret what caused me to write the above; It wasn't a claim that *you* were underscoping the problem, but from my interactions with people, the "focusing on HTTP" perception *is* fairly common and we need to overcome that problem. This just happened to be an appropriate place to mention it. [ Good stuff deleted ] > > HTTP was mentioned because it is a particularly tough case and may be > more in need of improvement than others, not that it is thought to be > the only service of importance. > I agree that HTTP is probably the hardest case (until something else comes along) because of the expected instant gratification from clicking the mouse. But, how many times have any of us answered queries from folks who lease X Mbps capacity over satellite link and can't figure out why they can't get more than more than a fraction of that on their FTP sessions? I'm willing to bet that there *are* some POP servers being placed on the wrong side of the satellite link and there are NNTP clients on the far side of a satellite link from the server. Most folks are just not aware of lots the problems and some of the more pedestrian fixes. These are the kinds of things that cause people to believe that TCP doesn't work over satellite links. My point was that we need to make sure we cover those areas in the collected wisdom. > Curtis > > btw- NFS over satellite is probably a real bad idea. :-) Great, there goes my grand scheme to host a slew of remote boot-servers in Uzbekistan for the diskless clients in Chile; Bummer! :o( Eric ------------------------------ Date: Fri, 25 Apr 1997 18:13:04 -0400 From: Keith Hogie Subject: NASA/GSFC TCP to Satellite research Here is a summary of TCP to Satellite work at NASA/GSFC We are a little different from TCP Over Satellite in that we are concered with TCP to science satellites. These satellites are anywhere from low-earth orbit to geosync or L1 with RTTs from 5 milliseconds up to 5 seconds. This is part of activities to find ways to meet NASA's goals of "faster, better, cheaper". We are looking into options for treating our science satellites more like a node on a network. The real goal is to be able to implement our ground and space systems using more commercial-off-the-shelf (COTS) components. This can provide great cost savings over the current process of building custom ground and space systems for each new mission. One of our areas of investigation has been TCP performance over simulated space links with various propagation delay, error and asymetric bandwidth characteristics. These tests showed both good and bad results for various spacecraft data transfer and control applications. We are looking to the TCP over Satellite working group to help resolve some of these problems through protocol enhancements while we work to resolve others through changes in the ground and satellite components. However, TCP to a satellite is not our only focus. We are currently looking into the end-to-end issues of how satellite architectures and operations need to be modified to make use of standard network capabilities such as TCP. Present satellite architectures and operations concepts are based on highly scheduled and managed data transfers which lead to complex systems and high operating costs. Using TCP to a spacecraft requires changes in current architectures and operating concepts. We are assembling a mock satellite using modern COTS components such as RISC processors and real-time operating systems to investigate both performance and architectural issues of new concepts. - ----------------------------------------------------------------------- Keith Hogie e-mail: hogie@joy.gsfc.nasa.gov Computer Sciences Corp./SSD office: 301-794-1961 fax: 301-794-2280 10110 Aerospace Road Lanham-Seabrook, MD 20706 USA 301-286-3203 @ NASA/Goddard - ------------------------------------------------------------------------ ------------------------------ Date: Fri, 25 Apr 1997 15:45:07 -0700 From: Grace Yee Subject: work outline At Lockheed Martin, we are trying to tackle the large bandwidth delay product of TCP over a GEO satellite ATM network. The effect of disabling the slow start algorithm and several other TCP enhancements are studied. We have implemented NAK over high BER and long delay links. Simulations are run using MIL3's OPNET network modeling software. TDMA protocol is used and all the control cells are sent out-of-band. Preliminary results have shown that disabling the slow-start, adding NAK and big window scaling option can improve the TCP performance. Grace Yee Lockheed Martin Missiles & Space ------------------------------ Date: Fri, 25 Apr 1997 22:32:18 -0400 From: Curtis Villamizar Subject: Re: Definition of High Bandwidth Delay Product In message , Fred Baker writes: > At 4:48 PM -0400 4/24/97, Daniel R Glover wrote: > >I like the idea of something called "routing delay" to add to the > >propagation delay for comparison purposes. > > I'd like you to not use the word "routing", for political reasons: folks > have a strange idea that routers don't switch traffic, and ask "would a > switch be different?" In pursuit of semantic purity... :-) There is no such thing as routing delay, only forwarding delay due to queueing in the forwarding path. If you have a connection oriented protocol there can be call setup delay. [ Well... If you wanted to be really picky you could claim the the few microseconds it might take for a radix tree lookup is routing delay. Of course if you had a really screwey architecture that did something wierd like send something off to another very slow process on some sort of a cache miss, then maybe you'd get a milliseconds (or find yourself on a different queue, in which case you have two forwarding paths with different forwarding delays). ] Sorry for going off on a tangent. Curtis ------------------------------ Date: Fri, 25 Apr 1997 20:43:49 -0700 From: touch@ISI.EDU Subject: Re: Definition of High Bandwidth Delay Product > From: Curtis Villamizar > > In message , Fred Baker writes: > > At 4:48 PM -0400 4/24/97, Daniel R Glover wrote: > > >I like the idea of something called "routing delay" to add to the > > >propagation delay for comparison purposes. > > In pursuit of semantic purity... :-) > > There is no such thing as routing delay, only forwarding delay due to > queueing in the forwarding path. If you have a connection oriented > protocol there can be call setup delay. > > [ Well... If you wanted to be really picky you could claim the the few > microseconds it might take for a radix tree lookup is routing delay. > Of course if you had a really screwey architecture that did something > wierd like send something off to another very slow process on some > sort of a cache miss, then maybe you'd get a milliseconds (or find > yourself on a different queue, in which case you have two forwarding > paths with different forwarding delays). ] "Routing" delay is negiglible only for 'fast-path' packets. For IP, this excludes: any packet with an option any packet encapsulated in a protocol terminated at that router (IP-in-IP encapsulation, IP Authentication headers terminated at a gateway, etc.) multicast (usually not fast-path) In addition, there are complications beyond the radix tree, e.g., policy-based routing, that can add to the per-packet routing latency, although the overall packet rate can be compensated by employing parallel routing engines. Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: Fri, 25 Apr 1997 21:55:14 -0700 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Satellite research at Hughes Research Labs (HRL) Our group in Hughes Research Labs (HRL) pursues basic research in the area of telecommunications and computer networks. One focus is networking involving communication satellites. Most of our work is experimental and/or building proof-of-concept prototypes. We own or have access to the following satellite systems for experimentation: DirecPC, DirecTV, VSAT/PES, NASA ACTS. Topics we have done recently or is pursueing: * Architecture & interconnection - interconnecting the above satellite networks together and with wireless terrestrial networks * Asymmetric routing - IETF UDLR * ATM over satellite - QoS * Multicast over satellite - MBONE over NASA ACTS, DirecPC, etc. (now tunneling, eventually native multicast) * TCP over satellite Now more on TCP over satellite. We are studying TCP over long latency network by emulation and by real experimentation (over actual satellites). The purpose of emulation (using a delay generator and regulating the bandwidth) is to provide baseline comparisons. One experiment that we are currently doing is to see what is the impact of latency on network utilization. It is well understood (by analysis and by many previous experiments) that TCP can utilize any given bandwidth and delay, if the window size is big enough and if the file is large enough. It's perhaps more interesting to see how many short TCPs (under normal window size and that transfer a normal size file) can fill up the pipe. We have some intermediate data that I presented at Memphis. Still we have many questions that we need more experiments to answer. More information: http://www.wins.hrl.com/ and my home page at http://www.wins.hrl.com/people/ygz Yongguang Zhang ============================================================================ Yongguang Zhang, Ph.D., Research Staff Member (Computer Science) Hughes Research Laboratories, RL-96, 3011 Malibu Canyon Rd, Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com phone: 310-317-5147 URL: http://www.wins.hrl.com/people/ygz fax: 310-317-5695 ------------------------------ Date: Fri, 25 Apr 1997 22:54:22 -0700 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Re: TCP/SATCOM draft suggestion > btw- NFS over satellite is probably a real bad idea. :-) Why? :-) When we got our first DirecPC in the lab two years ago. We tried almost everything. And yes, NFS. We did mount the entire file system over GEO. It was okay. It was tolerable. It took a couple seconds to do the first "ls" or open the file for the first time, but given the bandwidth available, everything else was fast. We could have batchmarked the performance if we knew how. (We can still do it, if someone tell me how to measure the NFS performance.) NFS works over satellite. AFS should work better. Swapping over satellite might be a real bad idea. :-) The worst performance that we have ever seen is X window over satellite. (Really!) When we run X clients in an Internet host and display the open over satellite to a DirecPC host, we often have to wait minutes for the first window to pop up. (Of course, after the window pops up, subsequent painting is fast.) This is because, before the X client can open a window, it has to go back and fore to the X server to resolve all those "X properties", "X resources", ... And it won't just do them in one trip; it has to do one thing at a time. (Just think about HTTP 1.0.) The most ridiculous thing is X color allocation. If you open a window with 256 colors, great: one color in a round-trip, 2 color per second. :-) But, even this is remediable. I published a paper last year suggesting several ways to run X window over satellite better. The basic idea is to batch all the X requests together, turning synchronous client/server interactions into asynchronous messages. And this can be done by placing X proxies at both ends of the satellite network, without changing X servers or applications. (http://www.wins.hrl.com/people/ygz/papers/xtech96.html) After my experiences with X window and WWW, I want to make a bold statement: What slows down TCP is not GEO satellite, What slows down TCP is application protocol. (Pardon my English ..., I wanted to imitate the famouse line "what slows down hardware is software" :-) Yongguang Zhang ============================================================================ Yongguang Zhang, Ph.D., Research Staff Member (Computer Science) Hughes Research Laboratories, RL-96, 3011 Malibu Canyon Rd, Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com phone: 310-317-5147 URL: http://www.wins.hrl.com/people/ygz fax: 310-317-5695 ------------------------------ End of tcp-over-satellite-digest V1 #18 *************************************** tcp-over-satellite-digest Saturday, May 3 1997 Volume 01 : Number 019 In this issues: Need for PAWS Re: Need for PAWS New SCPS Red Books/Home Page NASA Lewis work Outline of work in EUTELSAT See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Tue, 29 Apr 1997 11:19:00 -0400 From: matthew halsey Subject: Need for PAWS This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-002DDC91 Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Memo Text: (20 lines follow) I am trying to determine why PAWS would be necessary in satellite networks incorporating RFC 1323. I know PAWS is in 1323 and we might as well incorporate it, but why would we need it? As I understand, after reviewing 1323 and Stevens TCP/IP Illustrated, PAWS is required when the sequence numbers could be used within the MSL. The sequence numbers are re-used every 4x2**30 Bytes (4,294,967,296 Bytes) and the MSL is specified as 2 minutes but could be 30 seconds. Therefore PAWS is needed when 4,294,967,296 x 8 / speed < MSL MSL Speed Limit 30 seconds 1.145 Gbps 60 seconds 572.662 Mbps 120 seconds 286.331 Mbps If a satellite link with 1323 does not exceed 286 Mbps, would PAWS still be necessary? Matt Use Proportional Font: true Attachment Count: 0 - --mime-boundary-iaanaabhmb-002DDC91 Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjf9GCgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQABAA5AQAAAAAAAAAAAAAAAAAA AAAAAAAAVGV4dM4CSXQAAAAAAAAAAAAAJwEDAM4CIAEDAAIAAAANAAEAAQAB ALEAAAAAAAAAAgCyAAQAAAAAAAAAAQC2ABkCAAAAAAAAOP8AAAAAAACQAQAA AAAAAAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8A AAAAAAC8AgABAAABAgEiTVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAQABALoAAQC7ALsAAQC8ABQCAQAVAhUCAQAWAhYCAQAXAioCAQAr AkACAQBBAlgCAQBZAnECAQByAnICAQBzAskCAQDKAsoCAQDLAs8CAAAAAAAA AAAAAAAAZAABgAEBAAMBgAQBAAYBgAcBAAkBgAoBAAwBgA0BAA8BgBABABIB gBMBABVlAAA= - --mime-boundary-iaanaabhmb-002DDC91-- ------------------------------ Date: Tue, 29 Apr 1997 13:11:29 -0400 From: Tim Shepard Subject: Re: Need for PAWS Matt, (((There appeared to be a mime attachment on your message, with file name ATTRIBS.BND. I use mh-e to read mail, and while it does handle mime attachments, it didn't know what to do with the ATTRIBS.BND attachment. It was just an unknow binary file. I'll assume it was not important. ))) Regarding PAWS: Your understanding that it is less important at lower bitrates is basically correct. The timestamp echoing mechanism, however, may be more generally useful in disambiguating round-trip times when there have been retransmissions. In particular, it may be important for a TCP to be able to figure out what the initial round trip time for some path is. Let's say we have a path with a 1.15 second round-trip time. At time zero, a SYN is sent (to initiate a TCP connection). After one second, the SYN is retransmitted. At time 1.15 seconds, an ack is received for the SYN (with a SYN from the other side). Without the timestamp echoing option, this may look like a 0.15 second round trip. With the timestamp echoing option, it will be unambigously a 1.15 second round trip. Though I do not believe I yet understand exactly what the source TCP should do regarding the slow-start startup in this case, I believe a misunderstanding of what the real round-trip time is would sabotage almost any scheme that I can think of. Also, I should mention that PAWS may be more important than it first seems from back-of-the-envelope calculations involving MSL. MSL is now a bit of fiction. The idea originally was that routers in the Internet would subtract from the TTL field in the IP header the number of seconds the IP datagram sat in the router's queue, and also subtract at least one more from the TTL to represent the propagation delay of the link and to insure that it got decremented by at least one for each routing decision that was made. Now, no routers that I know of do anything than decrement the TTL by one per hop. I have seen a computer which, when it's ethernet was disconnected, hold on to packets for a few minutes. It dumped them all out as soon as the ethernet was reconnected. I presume that it would have sat holding the packets indefinitely had the ethernet remained disconnected indefinetly, possibly releasing the packets hours or days later. Imagine a long-lived TCP connection doing bulk transfer over the Internet. Imagine that somewhere along the path, the path is broken, and a few packets are caught and held. Meanwhile, the routing protocols eventually get the packets flowing again over some alternative route and the TCP connection continues to make progress. Then sometime later, the fault is repaired, and the held packets (which may now be many minutes or maybe a few hours old) are released into the net. Oops. So, do you really trust that the Internet will guarantee that it will never deliver a packet to you that is more than a couple of minutes old? To me it seems that PAWS is needed on any TCP connection that will wrap the TCP sequence space, even if it would take more than a few minutes to do so. A 1.5 Mbps T1-line can carry 2^32 bytes in about 6 hours. A 45 Mbps DS3 can carry 2^32 bytes in about 13 minutes. I hope this helps. -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ Date: Tue, 29 Apr 1997 10:23:09 -0700 From: adrian.hooke@jpl.nasa.gov (adrian hooke) Subject: New SCPS Red Books/Home Page As promised in my response last week to Aaron's request for work outlines, the hot-off-the-press draft specifications for the NASA/DOD Space Communications Protocol Standards (SCPS) are now available for free electronic download and review. You can access them via the brand new SCPS Home Page, which contains direct links into the Consultative Committee for Space Data Systems (CCSDS) site where the documents are stored. The SCPS Home Page is at: http://scps.jpl.nasa.gov/scps Best regards, Adrian J. Hooke Manager, NASA Space Mission Operations Standardization Program Jet Propulsion Laboratory, California Institute of Technology M/S 301-235, 4800 Oak Grove Drive Pasadena, California 91109-8099, USA. Tel +1.818.354.3063 Fax +1.818.354.9068 Pager 800.759.8888 PIN 1109388 Email: adrian.hooke@jpl.nasa.gov ------------------------------ Date: Wed, 30 Apr 1997 10:12:04 -0400 From: Daniel R Glover Subject: NASA Lewis work The Satellite Networks and Architectures Branch at NASA Lewis Research Center is chartered to enhance the role of communication satellite networks in the Global Information Infrastructure and to enable the application of such networks for NASA missions. The branch has efforts in standards and interoperability, Internet protocols, ATM, and network architectures. The Internet protocol team consists of myself, Cindy Tran, and Jim Griner with help from Paul Mallasch. We bring in summer faculty fellows and summer interns to help, also. We have testbeds that emulate networks with satellite links in them. We make use of the NASA Advanced Communications Technology Satellite (ACTS) for experiments, either loopback or with outside groups such as Hughes Research Labs. Within the branch, Opnet is used for simulation work. The branch works in cooperation with Ohio University (Kruse, Ostermann, Allman, Hayes) through a grant and with BBN Systems and Technology (Partridge, Shepard) under contract. The University of Maryland Center for Satellite and Hybrid Communication Networks (Baras, Corson, et al) also carries out a major effort in all aspects of hybrid networks under a cooperative agreement. Ohio State University (Jain) does ATM work under contract that is not directly related to TCP at this time, but the branch is using TCP over ATM in some experiments. We work closely with the Satellite Division of the TIA to achieve seamless interoperability and hosted the first meeting of its protocol working group in January. The branch Web pages are under construction and new information will be added this summer. The URL is: http://sulu.lerc.nasa.gov/5610/f5610.html Some related links: ACTS: http://kronos.lerc.nasa.gov/acts/acts.html NASA Research Announcement: http://procurement.nasa.gov/EPS/LeRC/Synopses/NRA-97-LeRC-3/sol.html OU: http://jarok.cs.ohiou.edu/projects/satellite/ UMd: http://www.isr.umd.edu/CSHCN/ Raj Jain: http://www.cis.ohio-state.edu/~jain R/ Dan > >Action Items: >============= > > 1. Within the next two weeks (ending April 30), it is > requested that people post to the mailing list a brief > summary of their related work/research. Pointers to > papers and web-sites are encouraged. ------------------------------ Date: Wed, 30 Apr 1997 18:29:22 +0200 From: AGNELLI Stefano Subject: Outline of work in EUTELSAT >Aaron asked people doing TCP over satellite research to drop a short >note to the mailing list outlining their work. The System Engineering Division of EUTELSAT has been carrying out for some years various activities intended to promote the use of satellites in broadband networks and advanced information systems (ATM, DVB, TCP-IP, etc). In the past, we run field trials for the introduction of satellite links in European ATM networks in '94 and in '96. Also, we participated in R&D projects of the European Commission for the use of ATM-based and IP-based satellite links for LAN interconnection. In all these activities, the behavior of TCP-based applications in network with large delay*bandwidth has been studied and analyzed, with extensive use of enhancements to the basic TCP protocol as recommended in RFC 1323. Since 1995, EUTELSAT, in collaboration with research institutions, has been developing digital platforms based on DVB (Digital Video Broadcasting) and MPEG-2 standards (which are already being used to broadcast digital television) to provide TCP-IP based data services via satellite. This work is particularly looked at in the IETF working group on UniDirectional Link Routing (UDLR). However it is planned to test as well the behavior of TCP over these platforms. > Pointers to papers and web-sites are encouraged. I already sent to the mailing list a paper "LAN Interconnection via ATM Satellite Links for CAD Applications: the UNOM Experiment" presented at IEEE ICC '96, Dallas. Last year, we organized an ATM Workshop. The presentations (at least those available in electronic format) are available on the Web in Acrobat Reader Format, at the address "http://www.eutelsat.org/semin". Regards, Stefano AGNELLI Broadband Networks and Advanced Information Systems EUTELSAT - Technical Dept. - System Engineering Div. 70, rue Balard - 75502 Paris Cedex 15 - France tel. (int'l) +33-1-53984640 (nat'l) 01.53.98.46.40 fax. (int'l) +33-1-53984798 (nat'l) 01.53.98.47.98 e-mail: stefano.agnelli@eutelsat.fr http://www.eutelsat.org ------------------------------ End of tcp-over-satellite-digest V1 #19 *************************************** tcp-over-satellite-digest Saturday, May 10 1997 Volume 01 : Number 020 In this issues: InterOP Keynote talk Sack over Satellite re: Sack over Satellite Re: FWD>RE>Definition of High B Re: Sack over Satellite RE: FWD>RE>Definition of High B Re: FWD>RE>Definition of High B B*D examples SACK understanding? Re: SACK understanding? Re: SACK understanding? Re: [Fwd: B*D examples] See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 05 May 1997 16:18:21 -0700 From: Dan Molinelli Subject: InterOP Keynote talk Theree will be an InterOP+Network World Keynote talk on "Advances in Wireless and Satellite Access Technologies" on thursday, 5/8th at 10am PDT. the talk will be broadcast on the MBONE (IP Mulitcast Backbone) for those of you that have access. later dan ------------------------------ Date: Mon, 5 May 1997 18:44:12 -0700 (PDT) From: Renaud Bruyeron Subject: Sack over Satellite We study the behaviour of TCP Sack over long delay links with finite bandwidth and various loss characteristics. We simulate the Satellite link using IP-in-IP tunnels. For now, we used a basic Rand() to introduce losses with a given probability, and emulated some sort of bursty loss by dropping more than one packet in a row. We believe that this not the typical loss pattern on a satellite link. We have three questions: 1) What is a typical loss rate due to hardware failures ? 2) What fraction of the losses come from drops at congested queues ? 3) What is the typical loss pattern for a satellite link ? Thanks for your feedback, - Renaud. - ------- Renaud Bruyeron. Graduate Student, Computer Science Department, UCLA Research Assistant, Internet Research Lab (TCP SACK Project) 3rd year Student at Ecole Centrale Paris (France) ------------------------------ Date: Wed, 7 May 1997 15:04:00 -0400 From: matthew halsey Subject: re: Sack over Satellite This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-00BC190F Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Reply Text: (7 lines follow) Could you please clarify an issue about SACK for me. In regular TCP, the acknowledgements are cumulative and therefore you needn't ACK every segment. However, with SACK I assume the ACKs are not cumulative so does this mean that you must ACK (or SACK) every segment? Thanks Matt Original text: (34 lines follow) From BRUYERON@SMTPGATE (Renaud Bruyeron) {bruyeron%CS.UCLA.EDU.@intelsat1.intelsat.int}, on 5/5/97 9:44 PM: To: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} We study the behaviour of TCP Sack over long delay links with finite bandwidth and various loss characteristics. We simulate the Satellite link using IP-in-IP tunnels. For now, we used a basic Rand() to introduce losses with a given probability, and emulated some sort of bursty loss by dropping more than one packet in a row. We believe that this not the typical loss pattern on a satellite link. We have three questions: 1) What is a typical loss rate due to hardware failures ? 2) What fraction of the losses come from drops at congested queues ? 3) What is the typical loss pattern for a satellite link ? Thanks for your feedback, - Renaud. - ------- Renaud Bruyeron. Graduate Student, Computer Science Department, UCLA Research Assistant, Internet Research Lab (TCP SACK Project) 3rd year Student at Ecole Centrale Paris (France) Use Proportional Font: true Previous From: BRUYERON@SMTPGATE (Renaud Bruyeron) {bruyeron%CS.UCLA.EDU.@intelsat1.intelsat.int} Previous To: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Original to: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-00BC190F Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjU83CgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQADQDcAQAAAAAAAAAAAAAAAAAA AAAAAAAAT3JpZ2luYWwgdGV4dPgDRgoAAAAAAAAAAAAAygEDAPgDxAECAAIA AAAhAAIAAQABAKsAAAAAAAAAAgCsAE0DAAAAAAAAOP8AAAAAAACQAQAAAAAA AAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8AAAAA AACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAG0AAQBtAKsAAQCrAKwAAgCsAPd/AgCtAPd/AgDyAPd/AgAeAfd/ AgAfAfd/AgBWAfd/AgBXAfd/AgCYAfd/AgDhAfd/AgArAvd/AgA9Avd/AgA+ Avd/AgBXAvd/AgCRAvd/AgDWAvd/AgARA/d/AgASA/d/AgAsA/d/AgAtA/d/ AgA4A/d/AgA5A/d/AgBBA/d/AgBSA/d/AgCGA/d/AgDDA/d/AgD1A/d/AgD2 A/d/AgD3A/d/AgD4A/d/AgD5A/d/AAAAAAAAAAAAAAAAZAABgAEBAAMBgAQB AAYBgAcBAAkBgAoBAAwBgA0BAA8BgBABABIBgBMBABUAAAAAAAAAAAAAAABk AAGwAQFgAwEQBQHABgFwCAEgCgHQCwGADQEwDwHgEAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-00BC190F-- ------------------------------ Date: Wed, 07 May 1997 17:01:12 -0700 From: Aaron Falk Subject: Re: FWD>RE>Definition of High B Aaron Falk wrote: > > At 09:24 AM 4/24/97 +0000, you (David Mercer) wrote: > >Can anybody put a number on what defines a "High Bandwidth Delay > >Product"? > > > > This might be a good thing to put in the draft, so I'll take a stab > at it: > > > R/ > Dan I've been thinking about this thread. It seems to me what we want to characterize is: what is the delay-bandwidth product where efficiency starts to degrade. Tim showed in Memphis that at 128 kbps TCP works fine over GEO delays and that T1 is very inefficient even with some of the mod's we've been talking about. So, the question is where do things start to fall apart? Aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Thu, 08 May 1997 15:33:39 +0200 From: Frank Zeppenfeldt Subject: Re: Sack over Satellite Renaud, >>> Renaud Bruyeron 05/06/97 03:44am >>> 2) What fraction of the losses come from drops at congested queues ? I suppose you are talking about router queues. We did an experiment with 4700 CISCO's with a sync serial and ethernet over satellite and saw that the session ( at 4 Mbps) did not really come to its maximum speed. In short : slow-start until congestion, drop, and so on Not really useful for a real time data stream. Queue tuning at both serial and ethernet cards was necessary but it was difficult to aportion a fraction of of the loss to this: insufficient queues meant no throughput at all. 3) What is the typical loss pattern for a satellite link ? Check out: "On the error burst properties of Viterbi Decoding" Proc of ICC'93, A. Franchi and R.A. Harris "Burst Error characterisation of FEC coded digital channels", ICDSD-9, D.J. Kennedy M.B. Mahkla Other possible sources: OPNET simulation tool have included now a model from a Stanford company, using all kind of atmospheric models to come to a BER and error distribution. They should have looked into this in detail. Best regards, Frank Zeppenfeldt ==================================================================================== Frank Zeppenfeldt Zeppenfeldt@eumetsat.de tel: +49 6151 807351 fax: +49 6151 807426 EUMETSAT Ground Segment Division Am Kavalleriesand 31 64295 Darmstadt Germany ==================================================================================== ! ! ------------------------------ Date: Thu, 8 May 1997 14:54:00 +0000 From: "Schram, Chris (Greenwich)" Subject: RE: FWD>RE>Definition of High B I agree that part of the question is determining the information rates where performance starts to significantly degrade, but as important for the commercial operators is characterizing the types of traffic where performance may or may not be degraded. We are carrying T1 IP traffic on PanAmSat, and outside of expected satellite delays the end user ISP community is not reporting problems with throughput (on the contrary, they are pleased with the performance compared to inefficient terrestrial routing and multiple ISP hops). As discussed in the Cleveland meeting, it would be useful to test how typical bulk traffic (like subscribers web browsing through an ISP) is carried by satellite, and look at particular inefficiencies. This may have been addressed since Cleveland, and it would be helpful for us to be able to go from telling people the 'end users think it's great' to 'if mixed web traffic of XXX type or XXX typical size is routed over a larger satellite link, there is little perceived degradation to the end user'. We would also be interested in providing capacity and teleport resources (and ISP links and participation) for testing 'real' traffic patterns. ---------- From: afalk[SMTP:afalk@mailsrv1.trw.com] Sent: Thursday, May 08, 1997 12:14 AM To: Dan Glover Cc: David Mercer; tcp-over-satellite@achtung.sp.t Subject: Re: FWD>RE>Definition of High B Aaron Falk wrote: > > At 09:24 AM 4/24/97 +0000, you (David Mercer) wrote: > >Can anybody put a number on what defines a "High Bandwidth Delay > >Product"? > > > > This might be a good thing to put in the draft, so I'll take a stab > at it: > > > R/ > Dan I've been thinking about this thread. It seems to me what we want to characterize is: what is the delay-bandwidth product where efficiency starts to degrade. Tim showed in Memphis that at 128 kbps TCP works fine over GEO delays and that T1 is very inefficient even with some of the mod's we've been talking about. So, the question is where do things start to fall apart? Aaron -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Thu, 08 May 1997 08:12:10 -0700 From: Aaron Falk Subject: Re: FWD>RE>Definition of High B Schram, Chris (Greenwich) wrote: > I agree that part of the question is determining the information > rates > where performance starts to significantly degrade, but as important > for > the commercial operators is characterizing the types of traffic > where > performance may or may not be degraded. Do you mean which types of traffic can tolerate degradation or which types of traffic will experience degradation? > We are carrying T1 IP traffic on > PanAmSat, and outside of expected satellite delays the end user ISP > community is not reporting problems with throughput (on the > contrary, > they are pleased with the performance compared to inefficient > terrestrial > routing and multiple ISP hops). The question is how much bandwidth is dedicated to an individual TCP connection. If you have 24 users at 64kbps on your T1, TCP works fine except for the response time. The thorny problems occur when you have one users who wants to use all that bandwidth. The way it's been described so far, web browsing is not really affected by the high bandwidth-delay product issues since most files are small. This type of traffic is affected by the number of round trip times that slow start imposes. However, I believe that users used to modem access to the Internet are not going to notice this additional latency. Aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Fri, 09 May 1997 14:32:50 -0400 From: Daniel R Glover Subject: B*D examples I thought that describing large windows in the draft would be easy since it is old hat, but there are a few points that should be explained clearly. It is important to show what isn't a problem as well as what is, as Chris and Aaron point out. One point to be made is that just because there is a satellite involved doesn't mean that you have to try to get the largest possible window size. Some low rate users don't even need to change the default, much less get an RFC1323 implementation. A second point is that as bandwidth increases, terrestrial networks will face problems due to large B*D, so this is not a satellite-only problem. How about putting in some tables like the ones below? Now, without worrying about what typical (terrestrial or satellite) values for RTT should be, I'll just use some round numbers like 0.1, 1, and 2 seconds for illustrative purposes. TABLE 1 Optimum Window Sizes for a TCP Connection Rate RTT B*D(optimum window) LFN? (bps) (sec) (bytes) (Yes, No) 33.6k 0.1 420 N 33.6k 1 4200 N 33.6k 2 8400 N 128k 0.1 1600 N 128k 1 16000 N 128k 2 32000 N 1.55M 0.1 19375 N 1.55M 1 193,750 Y 1.55M 2 387,500 Y 10M 0.1 125,000 Y 10M 1 1,250,000 Y 10M 2 2,500,000 Y 155M 0.1 1,937,500 Y 155M 1 19,375,000 Y 155M 2 38,750,000 Y Note that for a 2 sec RTT, you don't need large windows for a rate less than 262,140 bps. For a 0.1 sec RTT you will need large windows when the rate is a little greater than 5.2 Mbps. As Aaron pointed out, these numbers are for individual TCP connections. It doesn't matter if a bunch of TCP connections are trunked together. TABLE 2 Rate RTT for 64KB B*D (bps) (msec) 33.6k 15,600 (15.6 sec) 128k 4,096 (4.1 sec) 1.55M 340 10M 50 155M 3 622M 0.8 Note that a 128kbps connection doesn't need large windows even for several satellite hops, while a 155Mbps or 622Mbps connection would need large windows routinely. Large windows is the least of our problems since it already exists as an option in RFC1323 and more implementors are including it. It is a good idea for us to explain the problem so that people can figure out if they need large windows or not, but that part of the draft should be pretty straightforward. Something we should discuss is how to pick the best window size for a given user. I don't know where to start on that. R/ Dan ------------------------------ Date: Fri, 9 May 1997 15:20:00 -0400 From: matthew halsey Subject: SACK understanding? Form: Memo Text: (29 lines follow) Gentlemen, Thankyou for your comments on SACK. I would like to see if I finally understand it. Please bear with me. Regular TCP: Say a series of packets (#1-5) have been received successfully. Receiver (Rx) now ACKs #6. PAcket #6 is lost. Packets #7 through #10 is received and stored in buffer but Rx still ACKs #6. The RTO is expired. The Rx buffer is lost and transmitter needs to transmit packets # 6 through 10. Regular TCP with Fast Retransmit: As above but after packet #5 gets received the Rx buffers #7,8 and 9. At this point the transmitter has received 3 duplicate ACK #6 messages and retransmits packet #6, 7, 8 and 9. SACK with Fast Retransmit: Packets # 1-5 all received OK, packet #6 lost. packet #7 received. Rx sends SACK #7, ACK #6. packet #8 received. Rx send SACK#8, ACK#6. packet #9 received. Rx send SACK#9, ACK#6. Transmitter resends packet #6. Received OK. Rx sends ACK#10. Indicating that he can now successfully acknowledge receipt of packets up to #9 and is waiting for #10. Is this clear? If so is it correct? Thanks for indulging me with your time. Matt Use Proportional Font: true ------------------------------ Date: Fri, 09 May 1997 17:02:11 -0400 From: Christopher Hayes Subject: Re: SACK understanding? I am trying to get this out before I leave so forgive me for any over simplification or dumb mistakes. >Thankyou for your comments on SACK. I would like to see if I finally >understand it. Please bear with me. > >Regular TCP: >Say a series of packets (#1-5) have been received successfully. Receiver >(Rx) now ACKs #6. Packet #6 is lost. Packets #7 through #10 is received >and stored in buffer but Rx still ACKs #6. The RTO is expired. The Rx >buffer is lost and transmitter needs to transmit packets # 6 through >10. The Rx buffer may or may not be lost, The Rx has the right to renege on the packets in the Rx buffers (so it seems from RFC 2018) but as long as the computer has available buffer space I don't think it would discard them. The RTO will expire and the sender will enter slow start and transmit from packet #6 on up. If a duplicate packet is still in the Rx buffer it would be discarded. > >Regular TCP with Fast Retransmit: >As above but after packet #5 gets received the Rx buffers #7,8 and 9. At >this point the transmitter has received 3 duplicate ACK #6 messages and >retransmits packet #6, 7, 8 and 9. When three duplicate ACKs are received the sender will reset RTO and retransmit #6. When the ACK returns for #6, the ACK will cover #7-#9 which are still in the Rx buffer. A single packet drop is the best situation. However, if the sender sends #5-#12 and packets #6, #8, and #9 are lost. The sender will receive three duplicate ACKs and fast retransmit #6. When the ACK covering #6 and #7 is received the sender exits fast retransmit. All the packets have drained from the connection so there will not be enough duplicate ACKs generated to fast retransmit #8 or #9. The RTO timer will expire and the sender will enter slow start and begin transmitting with packet #8. >SACK with Fast Retransmit: >Packets # 1-5 all received OK, packet #6 lost. packet #7 received. Rx >sends SACK #7, ACK #6. packet #8 received. Rx send SACK#8, ACK#6. packet >#9 received. Rx send SACK#9, ACK#6. Transmitter resends packet #6. >Received OK. Rx sends ACK#10. Indicating that he can now successfully >acknowledge receipt of packets up to #9 and is waiting for #10. This is essentially correct. Rx will send a SACK saying I have #7. The next ACK will contain a SACK saying I have #7-8. SACK is more useful for multiple drops in a window. If #9 was dropped in the above example. The SACK blocks in the ACK for #10 would report #7-8 and #9. Here is a rough outline of the retransmission strategy in the PSC TCP with SACK. This allows the sender to recover without a RTO when you have multiple losses in a window. The listing of received outstanding data gathered from the receivers SACK blocks are stored in the scoreboard data structure. The sender can use the scoreboard to locate segments that have been lost and need retransmitted. The SACK retransmission strategy is outlined in Fall, K. and Floyd,S. Simulation-based Comparisons of Tahoe, Reno, and SACK TCP at http://ftp.ee.lbl.gov/floyd/papers.html This new strategy is designed to solve the performance problems associated with multiple losses in a window of data. If a connection has a single packet loss event then standard Reno recovery algorithms are used. However, if TCP with SACK suffers a multiple packet drop in a single window of data a new recovery scheme is used that uses the SACKed information maintained in the scoreboard. The sender will not exit the Fast Recovery phase until an acknowledgment is received that covers all the outstanding data present when Fast Recovery began. This new scheme maintains an estimated amount of data on the path in the pipe variable. Pipe is decremented by a segment size for each duplicate ACK that is received and two segments for each partial ACK that is received. Partial ACKS receive two because it assumes two packets have left the path, the dropped packet and the retransmitted packet. The sender is allowed to send new data or retransmit data if the pipe falls below cwnd(Congestion window). When retransmitting data the SACK sender will retransmit the next missing data segment maintained in the scoreboard. This scheme guarantees that the sender will never recover slower than slow start. As in TCP Reno a retransmission timer is maintained and if a retransmitted packet is lost then the retransmission timer will expire and the connection will enter Slow Start. >Thanks for indulging me with your time. >Matt No problem. Chris Hayes Graduate Assistant Ohio University chayes@ace.cs.ohiou.edu ------------------------------ Date: Fri, 9 May 1997 17:46:05 -0400 (EDT) From: rohit goyal Subject: Re: SACK understanding? Hello, I haven't been following this discussion, but your description is essentially correct. With regular TCP, it may be the case that the receiver could buffer the out-of-order packets received (7 -- 10 in your example). So when the sender times out, sends the first packet (#6) and waits for the ACK (since the window is now 1), the receiver may send a cumulative ack for 6 -- 10, and the sender can start sending from 11. A modification of fast retransmit and recovery was proposed by Janey Hoe in her MS thesis at MIT. This mechanism keeps track of the highest seq. # (RECOVER) sent out when the fast retransmit is triggered. The sender stays in fast retransmit phase until RECOVER is acked. This has been called "New Reno" in the LBL simulator (Fall, Floyd et. al.), and protect fast retransmit and recovery from the known problem of multiple drops. The mechanism can also easily be adapted to SACK. Look at http://www.cis.ohio-state.edu/~jain/atmf/a97-0423.htm for some SACK descriptions and results. Rohit - -- Rohit Goyal, CIS PhD Student (W) (614)-688-4482 395 Dreese Lab, 2015 Neil Ave (H) (614)-488-5785 The Ohio State University Fax: (614)-292-2911 Columbus OH 43210 goyal@cis.ohio-state.edu http://www.cis.ohio-state.edu/~goyal > > Form: Memo > Text: (29 lines follow) > Gentlemen, > > Thankyou for your comments on SACK. I would like to see if I finally > understand it. Please bear with me. > > Regular TCP: > Say a series of packets (#1-5) have been received successfully. Receiver > (Rx) now ACKs #6. PAcket #6 is lost. Packets #7 through #10 is received > and stored in buffer but Rx still ACKs #6. The RTO is expired. The Rx > buffer is lost and transmitter needs to transmit packets # 6 through 10. > > Regular TCP with Fast Retransmit: > As above but after packet #5 gets received the Rx buffers #7,8 and 9. At > this point the transmitter has received 3 duplicate ACK #6 messages and > retransmits packet #6, 7, 8 and 9. > > SACK with Fast Retransmit: > Packets # 1-5 all received OK, packet #6 lost. packet #7 received. Rx > sends SACK #7, ACK #6. packet #8 received. Rx send SACK#8, ACK#6. packet > #9 received. Rx send SACK#9, ACK#6. Transmitter resends packet #6. > Received OK. Rx sends ACK#10. Indicating that he can now successfully > acknowledge receipt of packets up to #9 and is waiting for #10. > > Is this clear? > If so is it correct? > > Thanks for indulging me with your time. > > Matt > Use Proportional Font: true > ------------------------------ Date: Fri, 09 May 1997 20:27:58 -0400 From: Tim Shepard Subject: Re: [Fwd: B*D examples] > Do you think that only connections that meet the LFN definition are > going to have problems getting full efficiency (neglecting slow start > issues) over a GEO sat link? What I mean is that you showed those plots > of TCP unable to get to the max rate if the rate was too high. Does this > only apply to connections in the LFN catagory? > > [...] please post your response to the mail list. > > Aaron Aaron, I think what I've shown is that when the product of the delay and the bottleneck bitrate is more than the buffer space available for the queue before the bottleneck, then the current TCP congestion related algorithms will prevent even the most cutting-edge TCPs (e.g. the PSC FACK implementation) from getting up to the proper rate, in simple solo demonstrations. This problem is relevant to the satellite case because among those who manage routers in the Internet there is some conventional wisdom that you need to put enough buffering in each router so that it can buffer the full outgoing line rate for about a RTT which everyone knows is about 100 msec. If you take a conventional piece of the Internet and concatenate a satellite hop, you won't get the performance you expect even if you use at each end the best TCPs you can get in May of 1997. (And I do not believe we can arrange to have the amount of buffer space in every router on the net adjusted to accommodate adding a satellite hop.) I believe that there is a problem even when the back-of-the-envelope calculations show that a window size less than 64 kilobytes should be sufficient. But the problem is less severe at the slower rates. I hope I've managed to answer your questions. Also, Dan's calculations are something that everyone should grab a calculator and do for themselves a few times to help calibrate the thinking about whether the window scaling option and a large window is needed. However I don't think it's wise to suggest that the receiving TCP has any responsibility to select a good window size. The receiver should select a window based on it's own availability of buffer memory that it could allocate to the reassembly buffers for the connection. If it has oodles, it should offer a large window. (The use of the word "optimum" in Dan's table could lead to otherwise thinking. Perhaps "minimum required window to achieve full rate" would be a better label for that column.) I did find it somewhat startling that the the "LFN?" column in Dan's table was monotonic. (But indeed it is given the particular numbers that Dan chose to plug in.) > > PS. How does it feel to work for GTE? AF For the time being, I still work for BBN S&T, and GTE is a company with a division or two which perhaps are on the short list of BBN S&T's main competitors. All that's happened so far is that GTE has announced that they intend to offer $29/share for all outstanding shares of BBN, and the BBN board has voted to consider this a friendly offer. It remains to be seen if a sufficient number of the BBN shareholders will part with their shares at that price to give GTE a controlling majority. Even if this goes through, it remains to be seen if we (S&T) will be absorbed into or just owned by GTE. We'll see... -Tim Shepard BBN Systems and Technologies shep@bbn.com +1 617 873 2013 ------------------------------ End of tcp-over-satellite-digest V1 #20 *************************************** tcp-over-satellite-digest Saturday, May 17 1997 Volume 01 : Number 021 In this issues: Re: SACK understanding? Re: SACK understanding? See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- End of tcp-over-satellite-digest V1 #21 *************************************** tcp-over-satellite-digest Saturday, May 24 1997 Volume 01 : Number 022 In this issues: test Mail list problem fixed Interested in cell relay over satellite repost: Interested in cell relay over satellite (fwd) Repost: Prob. Area 1 Re: [Fwd: B*D examples] See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Tue, 20 May 1997 08:31:10 -0700 From: Aaron Falk Subject: test test - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Tue, 20 May 1997 10:06:29 -0700 From: Aaron Falk Subject: Mail list problem fixed Folks- I apologize about the problems with the mail list last week. The sys admin went out on long term sick leave and I didn't know. I spent several days trying to reach him before realizing he wasn't there. We now have a temporary replacement sys admin and the problem should be solved. It appears that messages mail to the list which were not broadcast have been lost. Please resubmit them if you still have copies. Aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Tue, 20 May 1997 08:58:35 -0700 (PDT) From: Thomas Bohn Subject: Interested in cell relay over satellite Hello: This is an initial post from me...I was informed of the tcp-over-satellite group by Cisco engineers out of Anchorage, due to my situation up here on the Seward Peninsula. I currently run an ISP over a frame relay satellite connect (via ATT bird, AuroraII) and am interested in learning more about cell relay opportunities in the future... Anybody have any information? And what sort of ground-based equipment does a third-party system need for interconnects to a cell relay switch at an earth station? Curious. If this post is inappropriate for the forum, please ignore, and reply such to direct e-mail: tbohn@nome.net Thanx, Thomas Bohn -- Systems Specialist Kawerak, Inc: 907.443.4392 -- Direct Desk Line 907.443.3807 -- FAX tbohn@nome.net nome.net: 907.443.5387 -- Direct Desk Line 907.443.5112 -- FAX tbohn@nome.net "By concentrating scientific attention upon a narrow area of trouble and by preparing the scientific mind to recognize experimental anomalies for what they are, crisis often proliferates new discoveries." -- Kuhn, "Structure of Scientific Revolutions" ------------------------------ Date: Tue, 20 May 1997 13:13:17 -0700 (PDT) From: Thomas Bohn Subject: repost: Interested in cell relay over satellite (fwd) This is a re-post, per Aaron's suggestion... Date: Tue, 20 May 1997 08:58:35 -0700 (PDT) From: Thomas Bohn To: tcp-over-satellite@achtung.sp.trw.com Subject: Interested in cell relay over satellite Hello: This is an initial post from me...I was informed of the tcp-over-satellite group by Cisco engineers out of Anchorage, due to my situation up here on the Seward Peninsula. I currently run an ISP over a frame relay satellite connect (via ATT bird, AuroraII) and am interested in learning more about cell relay opportunities in the future... Anybody have any information? And what sort of ground-based equipment does a third-party system need for interconnects to a cell relay switch at an earth station? Curious. If this post is inappropriate for the forum, please ignore, and reply such to direct e-mail: tbohn@nome.net Thanx, Thomas Bohn -- Systems Specialist Kawerak, Inc: 907.443.4392 -- Direct Desk Line 907.443.3807 -- FAX tbohn@nome.net nome.net: 907.443.5387 -- Direct Desk Line 907.443.5112 -- FAX tbohn@nome.net "By concentrating scientific attention upon a narrow area of trouble and by preparing the scientific mind to recognize experimental anomalies for what they are, crisis often proliferates new discoveries." -- Kuhn, "Structure of Scientific Revolutions" ------------------------------ Date: Fri, 23 May 1997 17:11:51 -0400 From: Daniel R Glover Subject: Repost: Prob. Area 1 Eric, How is the draft coming? Are you refining the problem area outline? Problem Area 1 in the draft outline is "Large Bandwidth * Delay Product." To start a list for "currently available (or proposed) mitigation techniques" there are: RFC1323 (or whatever the new draft becomes): D. Borman, R. Braden, V. Jacobson, "TCP Extensions for High Performance", 05/13/1992. I-D "TCP Extensions for High Performance", V. Jacobson, R. Braden, D. Borman, 02/25/1997. "This memo presents a set of TCP extensions to improve performance over large bandwidth*delay product paths and to provide reliable operation over very high-speed paths. It defines new TCP options for scaled windows and timestamps, which are designed to provide compatible interworking with TCP's that do not implement the extensions. The timestamps are used for two distinct mechanisms: RTTM (Round Trip Time Measurement) and PAWS (Protect Against Wrapped Sequences). Selective acknowledgments are not included in this memo. This memo updates and obsoletes RFC-1323, a Proposed Standard, with small clarifications and corrections." RFC2001 which covers Fast Retransmit and Fast Recovery W. Stevens, "TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms", 01/24/1997. RFC2018 SACK M. Mathis, J. Mahdavi, S. Floyd, A. Romanow, "TCP Selective Acknowledgment Options", 10/17/1996. I would guess that RFC1323 is going to be useful in many cases, but that SACK as well as Fast Retransmit and Fast Recovery would be helpful in all cases that include satellites. Under the proposed or research category: Janey C. Hoe: "Improving the Start-up Behavior of a Congestion Control Scheme for TCP" M. Mathis and J. Mahdavi: "Forward Acknowledgment: Refining TCP Congestion Control," 1996 and if we're not limited to discussing TCP directly: Floyd, S., and Jacobson, V.: "Random Early Detection Gateways for Congestion Avoidance," 1993 Is this a "what to look for in a TCP?" Should we discuss various TCP heritages or is that too confusing? It would help me to have the introduction to the draft to look at to help keep me focused. R/ Dan ------------------------------ Date: Fri, 23 May 1997 18:05:53 -0400 From: Greg Miller Subject: Re: [Fwd: B*D examples] Weeks ago, Tim Shepard wrote: > >I think what I've shown is that when the product of the >delay and the bottleneck bitrate is more than the buffer space >available for the queue before the bottleneck, then the current TCP >congestion related algorithms will prevent even the most >cutting-edge TCPs (e.g. the PSC FACK implementation) from getting >up to the proper rate, in simple solo demonstrations. I'm curious, were these simulations, lab tests, or tests across a production network? >This problem is relevant to the satellite case because among those >who manage routers in the Internet there is some conventional >wisdom that you need to put enough buffering in each router so that >it can buffer the full outgoing line rate for about a RTT which >everyone knows is about 100 msec. If you take a conventional piece >of the Internet and concatenate a satellite hop, you won't get the >performance you expect even if you use at each end the best TCPs >you can get in May of 1997. (And I do not believe we can arrange >to have the amount of buffer space in every router on the net >adjusted to accommodate adding a satellite hop.) I agree that it's a commonly-held belief that routers need to be able to buffer line rate for a RTT. However, I don't believe this view reflects reality. The fact is that very few routers in today's internet can buffer line rate (often OC-3) for anywhere near 70 ms (coast-to-coast propagation time). That's primarily because routers with that amount of buffering aren't even available, as opposed to it being a conscious decision by network operators not to deploy deep buffers. Because most paths on the Internet today already traverse lots of "under-buffered" routers, I don't think adding a satellite link to a typical path will cause any significant loss of performance due to router buffering issues (latency and host buffers are another story). As for the impact of small router buffers on TCP performance, I can offer a data point. We routinely get TCP throughput numbers above 125 Mb/s coast-to-coast across the vBNS. We use a vanilla TCP with window scaling (i.e., no SACK) and the path traverses routers with OC-3 interfaces that have less than 2 MB of packet buffering for all their interfaces combined. (We don't get full OC-3 (~136 Mb/s for TCP/IP over ATM) rates because we're limited by the router's ATM SAR rate.) Anyone else have results to share? Greg - -- Gregory J. Miller vBNS Engineering MCI Telecommunications Reston, VA 20191 gmiller@mci.net ------------------------------ End of tcp-over-satellite-digest V1 #22 *************************************** tcp-over-satellite-digest Saturday, May 31 1997 Volume 01 : Number 023 In this issues: Re: [Fwd: B*D examples] Re: [Fwd: B*D examples] Re: Repost: Prob. Area 1 TCPSAT Temporary Editor Re: Repost: Prob. Area 1 Re: Repost: Prob. Area 1 Prob. Area 2 re: TCPSAT Temporary Editor A pointer to J.C Hoe work Re: A pointer to J.C Hoe work See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Sat, 24 May 1997 19:55:50 -0400 From: Tim Shepard Subject: Re: [Fwd: B*D examples] Greg, A better way of putting it would be that with today's normal TCP start up and congestion-related behavior, individual flows going solo through the net will not achieve the performance that the bottleneck would allow if the the queue at the bottleneck is not at least a full RTT's worth at the bottleneck rate. So if the bottleneck is 45 Mbits/second, the RTT is 100ms, and there's only 800 kilobits (100 kilobytes) of buffer space at the queue before the 45 Mbit/second bottleneck, then a single TCP (with today's normal congestion related behavior) will see about 8 Mbits/second performance. The reason is that the TCP's slow start (which is perhaps mis-named) will experience a loss when only 800 kilobits are outstanding, and will henceforth have an ssthresh set at 800 kilobits, and only try to grow it by one packet per round trip time (which will take a long time to creep up to the bottleneck rate). -Tim ------------------------------ Date: Tue, 27 May 1997 13:26:32 -0400 From: Curtis Villamizar Subject: Re: [Fwd: B*D examples] [ warning - this movie has played before - appoligies in advance ] In message <199705232205.SAA22629@postoffice.Reston.mci.net>, Greg Miller write s: > > I agree that it's a commonly-held belief that routers need to be able to > buffer line rate for a RTT. However, I don't believe this view reflects > reality. The fact is that very few routers in today's internet can buffer > line rate (often OC-3) for anywhere near 70 ms (coast-to-coast propagation > time). That's primarily because routers with that amount of buffering aren't > even available, as opposed to it being a conscious decision by network > operators not to deploy deep buffers. Because most paths on the Internet > today already traverse lots of "under-buffered" routers, I don't think adding You're wrong here. Bay BCN and BLN routers offer 8 MB of buffering per HSSI, FDDI, or OC3 interface. There is 64MB and RISC processors available per forwarding card. Many providers have "under-buffered" routers and routers that drop packets prematurely for other (not at all legitimate) reasons. For some of these providers claiming "real fast" backbones independent measurements have shown fairly high loss. Some with the most impressive backbone speed claims show high losses. The fact that some providers use "under-buffered" routers does not support you're conclusion that deep buffers are not needed. If they had very low packet loss then it might. > a satellite link to a typical path will cause any significant loss of > performance due to router buffering issues (latency and host buffers are > another story). > > As for the impact of small router buffers on TCP performance, I can offer a > data point. We routinely get TCP throughput numbers above 125 Mb/s > coast-to-coast across the vBNS. We use a vanilla TCP with window scaling > (i.e., no SACK) and the path traverses routers with OC-3 interfaces that have > less than 2 MB of packet buffering for all their interfaces combined. (We > don't get full OC-3 (~136 Mb/s for TCP/IP over ATM) rates because we're > limited by the router's ATM SAR rate.) Correct me if I'm wrong, but isn't 125 Mb/s times 70 msec about 1.1 Mb/s? 2MB is already more than RTT*BW for OC3 so if the router is otherwise fairly idle you should (could if configured for it) get close to RTT*BW and possibly more. Also, if the window is optimized for this transfer, then you won't need much buffering. Optimizing the window for 1 flow won't work if there are 2 or more simultaneous flows (without reducing the window size). > Anyone else have results to share? > > Greg Sure. If we're only talking 70 msec delay, ftp://engr.ans.net/pub/papers/tcp-performance.ps Only 44.2 Mb/s (about 40-41 Mb/s goodput). Used SGI IRIX 5.3 workstations and 1-8 TCP flows, varrying buffer size and TCP window size. If you have only one flow and can guess the optimal window size accurately, you can get away with a small buffer. Look at figure 1 at the small buffer lines on the plots. One way flow optimal buffer size for a 70 msec path was about a bit over 384 KB. If there was two way traffic, optimal send/recv buffer size for a small buffer router was about 256 KB, acheiving about 25 kb/s rather than 40 kb/s. If you used 384 KB and had two way traffic (figure 2) you'd only get about 12 kb/s. If you used 384 KB and had a 20 msec path, you'd get about 15 mb/s for one way traffic (just beyond "the cliff") and you'd get about 12 mb/s for 2 way. If you had four flows (figure 3) at 384 KB window you'd get a total of 25 mb/s goodput. So what is the magic number that everyone should set their send and receive buffer sizes to? It is clear that if buffer sizes are less than RTT*BW that guessing wrong can seriously impact performance so if we are to use these routers guessing right is critical. The point is that it is impossible to guess this number (except for cases were you expect to have the entire wire and know the RTT ahead of time, which the vBNS for now fits into). The idea behind using buffers > RTT*BW is to make guessing the optimal send/recv buffer size less critical. RED makes a further improvdement in this regard. Sprint, ICM, and ANS each have evidence (far from conclusive) that increasing buffers from < RTT*BW to buffers > RTT*BW reduces loss in cases where there are very large numbers of TCP flows. The evidence with Sprint was gained when replacing 512KB cards with 2MB cards. With ANS it was deploying the code in the NSS routers that allocated memory better, bringing effective buffering from about 200 KB to 300 KB per (HSSI/DS3) interface (the code described in the above paper). With ICM, improvement was seen when using one router per direction to double buffer capacity (this was reported at NANOG). In these cases, fixing an operational problem (loss) was the priority so carefull measurement was not made before and after, however in each case reduction in loss was described as substantial. Curtis ps- Sorry to bore all of the people that have already heard this. ------------------------------ Date: Tue, 27 May 1997 11:32:37 -0700 From: touch@ISI.EDU Subject: Re: Repost: Prob. Area 1 > From owner-tcp-over-satellite@achtung.sp.trw.com Fri May 23 14:41:27 1997 > From: Daniel R Glover > Subject: Repost: Prob. Area 1 ... > How is the draft coming? Are you refining the problem area outline? > Problem Area 1 in the draft outline is "Large Bandwidth * Delay Product." > To start a list for "currently available (or proposed) mitigation > techniques" there are: > > Under the proposed or research category: > > Janey C. Hoe: "Improving the Start-up Behavior of a Congestion Control > Scheme for TCP" > > M. Mathis and J. Mahdavi: "Forward Acknowledgment: Refining TCP Congestion > Control," 1996 I'd like to offer my own as relevant here: J. Touch, "TCP Control Block Interdependence", USC/ISI, RFC-2140, April 1997. as well as some information on the fundamental problem and issues: J. Touch, "Defining `High Speed' Protocols : Five Challenges & an Example That Survives the Challenges", IEEE JSAC., special issue on Applications Enabling Gigabit Networks, Vol. 13, No. 5, June 1995, pp. 828-835. This is a discussion of the fundamental issues in high-speed systems, which are the large BW-delay product AND the unpredictability of the prototol state system. Predictable systems are easy. Much of my work has been focused on high bw*delay systems, and there is more information, including my dissertation, which was a fundamental discussion on latency and communication, at: http://www.isi.edu/~touch/pubs/ also http://www.isi.edu/lowlat/ Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: Tue, 27 May 1997 15:55:15 -0700 From: Aaron Falk Subject: TCPSAT Temporary Editor Folks- It seems Eric Travis has been having some problems with his email and I've been unable to reach him reliably for the last 4-6 weeks. Since we have a very tight schedule to meet, I've asked Mark Allman to act as editor until I can reach Eric. I've just snail mailed him a note as a last ditch effort. Mark can be reached at . He is defending his master's thesis on Wednesday and I suspect he'll be a little preoccupied for the next couple of days but I'm sure we'll hear more from him afterwords. Our schedule has draft 1 sent to mail list on June 30. To that end I'd like to ask folks to review the outline below (Eric submitted this following the discussion on the document at the Memphis IETF) and provide some paragraphs on sections that haven't been discussed so far. Aaron ====================================================== Draft Outline as Presented/Discussed in Memphis: I. Purpose a. Ensure that any and all recommendations resulting from this effort will not adversely impact other traffic on the shared (non-satellite) media. b. Provide a comprehensive document describing the problems (and potential solutions) associated with TCP performance in the satellite environment. c. Identify, explain *and* wherever possible quantify impacts of satellite environment on TCP performance Quantification: Simulation OK, but attempts should be made to validate simulations Emulations better, same desires regarding validation apply Live measurements most desirable, but probably most elusive Bottom line, whatever is available is desirable; Please contribute any relevant results and efforts wherever possible. Quantifications must *not* be conducted in a vendor specific manner. Impacts of base algorithms and common (or proposed) implementation techniques are acceptable (as are pointers to published work covering this area). I'm of the opinion that discussions/categorizations along the lines of 4.3BSD Tahoe, 4.3BSD Reno, 4.4BSD, Net/1, Net/2, Net/3, etc. are OK. I'm willing to revise my opinion here if necessary. In no uncertain terms are proprietary algorithms, implementations, etc. going to be acceptable. d. Categorize the problems according to those that can be addressed through engineering (applying what we already know) and those that require further research. For the latter, this is simply a tagging/identification effort. Anything else is out of scope of this group. e. Identify and explain what mitigation techniques exist and (in general) how to use them. Efforts should concentrate on the more mature, somewhat well understood techniques/options such as those in RFC 1323, RFC 2018, as well as others that have already been discussed within this group (J.C. Hoe's work, the proposed "4K initial cwnd", etc.). I'm not going to attempt to provide a comprehensive listing here. Explanations should cover not only the specific performance problems that a mitigation technique (attempts) to overcome, but also the why, how and where applicable the problems involved. As above, *nothing* proprietary is going to be acceptable. Along those lines, is the "how to use" coverage. This is likely to be difficult and if it proves too difficult then it should fall off the end of the document. Pointers/discussion to things like using socket-options is probably OK, but not things like how to use ndd on Solaris to tweak things. II. Overview of the problem domain A broad overview of the differences in the satellite environment that cause difficulties for TCP. There should be not effort to go into great detail here, that will be done in the next section. I envision a main element of this section to be a categorization of different architectures/topologies (common) to satellite applications. This should prove to be useful for the discussion of specific issues and mitigation techniques in section 3. Furthermore, such a categorization will allow for comparisons/contrasts to be made by those in non-satellite environments. [Please see the note in the next section regarding scope.] III. Specific Issues =================================================== Note regarding scope: ----------------------- There was some good discussion during Friday's session in Memphis as to whether or not we should address the performance impacts that applications and "link" technologies have on TCP in the satellite environment; As an example at the application level, the (unnecessary) interrogative nature of some applications, such as FTP does impact perceived user-level performance in long delay environments. I don't need to provide examples related to link(ish) technologies. Although this is a gray area (for us), I'd like to restate that *I* will be sympathetic to related contributions or pointers on these topics. I won't promise that these items will make it into the final document - we'll have to see how things fall together. Even if the contributions don't make it into this Informational RFC, I'm sure we can prevent them from going to waste. Also, let me be explicit that network layer topic *are* within the scope of this effort. I expect to see lots of content to be from this area. =================================================== a. Identify and describe the problems related to TCP performance in satellite environments. In addition, we wish to provide some validation as to the level to which the problem is understood. This will help sort the areas where engineering is appropriate/possible and those areas that require more research. A first cut at an ordered categorization of problem *areas* is as follows: 1. Large Bandwidth * Delay Product 2. Long Feedback Delay Path 3. Asymmetric Channels 4. Errors Before anyone starts complaining about this, there *are* environments where you get error-rates worse than 10E-8 *after* application of strong FEC; 5. ??? b. Identify, describe and *assess* the applicability of currently available (or proposed) mitigation techniques keyed to their associated problem areas. c. Tag those areas where engineering will not suffice as areas promising for research. d. For each specific issue identified: 1. Problem Statement: 2. Semi-detailed explanation of the problem 3. Mitigation techniques identified 4. Discussion of mitigation techniques a. 1st Order discussion on technique (isolated, no interactions) Assess the benefits and potential pitfalls of this technique keyed to the different topologies provided in section 2. This is will prove to be important. b. If applicable, tag this area as one needing further research. IV. Interacting Mitigations This section should *attempt* to address the potential impacts of mixing the different mitigation techniques identified/discussed above. V. Areas Requiring Further Engineering Pull together those areas identified in Section 3 that can be handled through engineering, but which fall out of scope of this group. V. Identified Research Areas Collect all the tagged research areas identified in as requiring further research. VI. Recommendations - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Wed, 28 May 1997 10:18:48 -0400 From: Curtis Villamizar Subject: Re: Repost: Prob. Area 1 In message <199705271832.AA15558@ash.isi.edu>, touch@ISI.EDU writes: > > J. Touch, "TCP Control Block Interdependence", USC/ISI, > RFC-2140, April 1997. There seems to be consensus that initializing TCBs based on information from a cache is a good thing for large D*BW. I should point out that this RFC is informational and was produced without review. When it was published a few caveats were pointed out that were missed in the RFC. For example, the RFC failed to mention that the cache information may age very quickly and may provide little benefit once HTTP 1.1 is in use. Still worth citing IMO. > J. Touch, "Defining `High Speed' Protocols : Five Challenges & an > Example That Survives the Challenges", IEEE JSAC., special issue on > Applications Enabling Gigabit Networks, Vol. 13, No. 5, June 1995, pp. > 828-835. > > This is a discussion of the fundamental issues in high-speed systems, > which are the large BW-delay product AND the unpredictability of the > prototol state system. Predictable systems are easy. What is the point of this paper and how is it relevant to the IP over Sat draft? The paper seems to arm wave about gigabit networks then wander off into an sketchy analysis of HTML/HTTP and some suggestions for improving them. I don't think this is worth citing. If we want to mention HTTP or other applications layer caching, there may be better references specific to caching. The HTTP 1.1 RFC-2068 might be a much better candidate. Does that mean we need to reference something on ftp mirroring, NNTP distribution, and other application layer caching. None of this is satellite specific, it is just that their benefits are more pronounced given limited bandwidth or long delay. Some brief mention is probably appropriate. Curtis ------------------------------ Date: Wed, 28 May 1997 08:13:38 -0700 From: touch@ISI.EDU Subject: Re: Repost: Prob. Area 1 > > J. Touch, "TCP Control Block Interdependence", USC/ISI, > > RFC-2140, April 1997. ... > I should point out that this RFC is informational and was produced > without review. When it was published a few caveats were pointed out > that were missed in the RFC. For example, the RFC failed to mention > that the cache information may age very quickly and may provide little > benefit once HTTP 1.1 is in use. When and where were these caveats pointed out? SOP is to inform the author when the RFC is at the I-D stage. FYI, it appears to be 'conventional wisdom' that this information ages quickly, but there appears to be evidence that this is not the case. Also, the RFC clearly addresses the issue of using these values as starting parameters only. > > J. Touch, "Defining `High Speed' Protocols : Five Challenges & an > > Example That Survives the Challenges", IEEE JSAC., special issue on > > Applications Enabling Gigabit Networks, Vol. 13, No. 5, June 1995, pp. > > 828-835. > > > > This is a discussion of the fundamental issues in high-speed systems, > > which are the large BW-delay product AND the unpredictability of the > > prototol state system. Predictable systems are easy. > > What is the point of this paper and how is it relevant to the IP over > Sat draft? The paper seems to arm wave about gigabit networks then > wander off into an sketchy analysis of HTML/HTTP and some suggestions > for improving them. I don't think this is worth citing. Seems to me there was a large discussion on BW*delay issues and how they related to TCP at one point in the stream of this mail list. This document clearly discusses how high speed and high latency combined affect protocols in general, and what sorts of accommodations are required. In particular, the main point is that, for a high BW*delay link to be effective, the info in transit has to fill the pipe (which we all know). However, what seems be overlooked is a SINGLE connection can fill the pipe when the response is not long enough to do so (HTTP is one example of this case). FYI, the criteria therein ('arm waving') has become part of the acceptance criteria for the Gigabit Networks Workshop, the flagship meeting of the IEEE Techical Committee on Gigabit Networking. I would hope the rest of the satellite community is interested in how gigabit networks change the requirements of protocols. In particular, to observe how pointless the window increase discussions are to support single high-speed satellite channels if the data sent (i.e., size of the data of the connection) is smaller than the BW*delay product. That is, unless some additional structure is added to the TCP stream, beyond that of byte ordering alone. And I agree, none of that is satellite specific. Just high bw*delay. In that case, exactly WHAT *is* satellite specific about this group? Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: Wed, 28 May 1997 14:41:52 -0400 From: Daniel R Glover Subject: Prob. Area 2 Mark (and Eric), More on the outline. I haven't seen Eric's response on Prob. Area 1 on the list yet, probably due to the server outage. I just want to get started on the next level of the outline to get the draft going, so I'm listing the easy stuff, there is more out there that we need to capture. Problem Area 2 is "Long Feedback Delay Path." This one is more satellite specific than Problem Area 1. The main problem is that slow start and congestion avoidance are very slow when the RTT is long. (A second problem could be that real-time applications may not tolerate long delays. I don't know how much of a problem the second one really is, but I don't know what to do about it anyway.) For mitigation sections, a paragraph or two on: T/TCP: Braden, R., "T/TCP -- TCP Extensions for Transactions: Functional Specification," RFC-1644, USC/ISI, July 1994 HTTP/1.1 (and Cascading Style Sheets and Portable Network Graphics) for web traffic. This avoids going through slow start for lots of little file transfers. The RFC also mentions caches and compression, which we should also expand on as more general mitigation methods. RFC2068 R. Fielding, J. Gettys, J. Mogul, H. Frystyk, T. Berners-Lee: "Hypertext Transfer Protocol -- HTTP/1.1," 01/03/1997 and http://www.w3.org/pub/WWW/Protocols/NL-PerfNote.html In the proposed/research area, there is S. Floyd: "Increasing TCP's Initial Window," end2end list, 2/7/1997 along with Mark Allman, Chris Hayes: "An Evaluation of TCP Slow Start Modifications," draft February 1997 http://jarok.cs.ohiou.edu/papers/slowstart.ps) Some one might want to list the TCP research paths out there today in a separate section of the draft. R/ Dan /************************** Dan Glover dglover@lerc.nasa.gov NASA Lewis Research Center MS 54-2 Cleveland, OH, USA 44135 (216)433-2847 **************************/ ------------------------------ Date: Fri, 30 May 1997 08:55:00 -0400 From: matthew halsey Subject: re: TCPSAT Temporary Editor This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-00008572 Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Reply Text: (7 lines follow) I will soon be providing some graphs of tests we have performed in our technical labs regarding various Internet applications. More soon. Matt INTELSAT Original text: (227 lines follow) From AFALK@SMTPGATE (Aaron Falk) {afalk%mailsrv1.trw.com.@intelsat1.intelsat.int}, on 27/5/97 6:55 PM: To: TCP-OVER@SMTPGATE (tcp-over-satellite) {tcp-over-satellite@achtung.sp.trw.com} Folks- It seems Eric Travis has been having some problems with his email and I've been unable to reach him reliably for the last 4-6 weeks. Since we have a very tight schedule to meet, I've asked Mark Allman to act as editor until I can reach Eric. I've just snail mailed him a note as a last ditch effort. Mark can be reached at . He is defending his master's thesis on Wednesday and I suspect he'll be a little preoccupied for the next couple of days but I'm sure we'll hear more from him afterwords. Our schedule has draft 1 sent to mail list on June 30. To that end I'd like to ask folks to review the outline below (Eric submitted this following the discussion on the document at the Memphis IETF) and provide some paragraphs on sections that haven't been discussed so far. Aaron ====================================================== Draft Outline as Presented/Discussed in Memphis: I. Purpose a. Ensure that any and all recommendations resulting from this effort will not adversely impact other traffic on the shared (non-satellite) media. b. Provide a comprehensive document describing the problems (and potential solutions) associated with TCP performance in the satellite environment. c. Identify, explain *and* wherever possible quantify impacts of satellite environment on TCP performance Quantification: Simulation OK, but attempts should be made to validate simulations Emulations better, same desires regarding validation apply Live measurements most desirable, but probably most elusive Bottom line, whatever is available is desirable; Please contribute any relevant results and efforts wherever possible. Quantifications must *not* be conducted in a vendor specific manner. Impacts of base algorithms and common (or proposed) implementation techniques are acceptable (as are pointers to published work covering this area). I'm of the opinion that discussions/categorizations along the lines of 4.3BSD Tahoe, 4.3BSD Reno, 4.4BSD, Net/1, Net/2, Net/3, etc. are OK. I'm willing to revise my opinion here if necessary. In no uncertain terms are proprietary algorithms, implementations, etc. going to be acceptable. d. Categorize the problems according to those that can be addressed through engineering (applying what we already know) and those that require further research. For the latter, this is simply a tagging/identification effort. Anything else is out of scope of this group. e. Identify and explain what mitigation techniques exist and (in general) how to use them. Efforts should concentrate on the more mature, somewhat well understood techniques/options such as those in RFC 1323, RFC 2018, as well as others that have already been discussed within this group (J.C. Hoe's work, the proposed "4K initial cwnd", etc.). I'm not going to attempt to provide a comprehensive listing here. Explanations should cover not only the specific performance problems that a mitigation technique (attempts) to overcome, but also the why, how and where applicable the problems involved. As above, *nothing* proprietary is going to be acceptable. Along those lines, is the "how to use" coverage. This is likely to be difficult and if it proves too difficult then it should fall off the end of the document. Pointers/discussion to things like using socket-options is probably OK, but not things like how to use ndd on Solaris to tweak things. II. Overview of the problem domain A broad overview of the differences in the satellite environment that cause difficulties for TCP. There should be not effort to go into great detail here, that will be done in the next section. I envision a main element of this section to be a categorization of different architectures/topologies (common) to satellite applications. This should prove to be useful for the discussion of specific issues and mitigation techniques in section 3. Furthermore, such a categorization will allow for comparisons/contrasts to be made by those in non-satellite environments. [Please see the note in the next section regarding scope.] III. Specific Issues =================================================== Note regarding scope: ----------------------- There was some good discussion during Friday's session in Memphis as to whether or not we should address the performance impacts that applications and "link" technologies have on TCP in the satellite environment; As an example at the application level, the (unnecessary) interrogative nature of some applications, such as FTP does impact perceived user-level performance in long delay environments. I don't need to provide examples related to link(ish) technologies. Although this is a gray area (for us), I'd like to restate that *I* will be sympathetic to related contributions or pointers on these topics. I won't promise that these items will make it into the final document - we'll have to see how things fall together. Even if the contributions don't make it into this Informational RFC, I'm sure we can prevent them from going to waste. Also, let me be explicit that network layer topic *are* within the scope of this effort. I expect to see lots of content to be from this area. =================================================== a. Identify and describe the problems related to TCP performance in satellite environments. In addition, we wish to provide some validation as to the level to which the problem is understood. This will help sort the areas where engineering is appropriate/possible and those areas that require more research. A first cut at an ordered categorization of problem *areas* is as follows: 1. Large Bandwidth * Delay Product 2. Long Feedback Delay Path 3. Asymmetric Channels 4. Errors Before anyone starts complaining about this, there *are* environments where you get error-rates worse than 10E-8 *after* application of strong FEC; 5. ??? b. Identify, describe and *assess* the applicability of currently available (or proposed) mitigation techniques keyed to their associated problem areas. c. Tag those areas where engineering will not suffice as areas promising for research. d. For each specific issue identified: 1. Problem Statement: 2. Semi-detailed explanation of the problem 3. Mitigation techniques identified 4. Discussion of mitigation techniques a. 1st Order discussion on technique (isolated, no interactions) Assess the benefits and potential pitfalls of this technique keyed to the different topologies provided in section 2. This is will prove to be important. b. If applicable, tag this area as one needing further research. IV. Interacting Mitigations This section should *attempt* to address the potential impacts of mixing the different mitigation techniques identified/discussed above. V. Areas Requiring Further Engineering Pull together those areas identified in Section 3 that can be handled through engineering, but which fall out of scope of this group. V. Identified Research Areas Collect all the tagged research areas identified in as requiring further research. VI. Recommendations - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com Use Proportional Font: true Previous From: AFALK@SMTPGATE (Aaron Falk) {afalk%mailsrv1.trw.com.@intelsat1.intelsat.int} Previous To: TCP-OVER@SMTPGATE (tcp-over-satellite) {tcp-over-satellite@achtung.sp.trw.com} Original to: TCP-OVER@SMTPGATE (tcp-over-satellite) {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-00008572 Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGje80CgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQADQBcBgAAAAAAAAAAAAAAAAAA AAAAAAAAT3JpZ2luYWwgdGV4dKsgRgoAAAAAAAAAAAAASgYDAKsgRAYCAAIA AADhAAIAAQABALsAAAAAAAAAAgC8APAfAAAAAAAAOP8AAAAAAACQAQAAAAAA AAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8AAAAA AACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAGgAAQBoALsAAQC7ALwAAgC8APd/AgDDAPd/AgDEAPd/AgAKAfd/ AgBSAfd/AgCXAfd/AgDdAfd/AgDwAfd/AgDxAfd/AgA4Avd/AgB5Avd/AgC+ Avd/AgDTAvd/AgDUAvd/AgAbA/d/AgBeA/d/AgCgA/d/AgDpA/d/AgDqA/d/ AgDwA/d/AgDxA/d/AgAoBPd/AgBZBPd/AgBaBPd/AgBbBPd/AgBmBPd/AgBn BPd/AgCeBPd/AgDUBPd/AgAGBfd/AgAHBfd/AgA8Bfd/AgB0Bfd/AgCnBfd/ AgCoBfd/AgDYBfd/AgAKBvd/AgAfBvd/AgAgBvd/AgA1Bvd/AgBqBvd/AgCG Bvd/AgCNBvd/AgC+Bvd/AgDWBvd/AgDXBvd/AgANB/d/AgAhB/d/AgAiB/d/ AgBaB/d/AgCUB/d/AgCuB/d/AgCvB/d/AgDqB/d/AgAoCPd/AgBmCPd/AgCl CPd/AgCtCPd/AgDpCPd/AgAmCfd/AgBfCfd/AgCbCfd/AgDaCfd/AgD5Cfd/ AgD6Cfd/AgA2Cvd/AgBzCvd/AgCnCvd/AgCpCvd/AgDpCvd/AgAlC/d/AgAm C/d/AgBhC/d/AgCIC/d/AgCJC/d/AgDIC/d/AgAEDPd/AgBDDPd/AgCCDPd/ AgC+DPd/AgD2DPd/AgD3DPd/AgA6Dfd/AgB+Dfd/AgC+Dfd/AgDPDfd/AgDQ Dfd/AgAWDvd/AgAcDvd/AgBcDvd/AgCeDvd/AgDmDvd/AgApD/d/AgBnD/d/ AgBoD/d/AgCLD/d/AgCMD/d/AgDJD/d/AgAEEPd/AgBJEPd/AgBtEPd/AgBu EPd/AgCoEPd/AgDkEPd/AgAhEfd/AgBgEfd/AgCfEfd/AgDdEfd/AgAUEvd/ AgAVEvd/AgBXEvd/AgBYEvd/AgBtEvd/AgBuEvd/AgCpEvd/AgDGEvd/AgDl Evd/AgAkE/d/AgBiE/d/AgCbE/d/AgDZE/d/AgDhE/d/AgAjFPd/AgBiFPd/ AgCfFPd/AgDbFPd/AgAFFfd/AgANFfd/AgBIFfd/AgCAFfd/AgC7Ffd/AgD3 Ffd/AgA1Fvd/AgBuFvd/AgCqFvd/AgDBFvd/AgDCFvd/AgD7Fvd/AgA1F/d/ AgBmF/d/AgChF/d/AgCiF/d/AgDcF/d/AgALGPd/AgAMGPd/AgBHGPd/AgB/ GPd/AgC5GPd/AgDzGPd/AgAJGfd/AgAKGfd/AgBGGfd/AgBkGfd/AgBlGfd/ AgCPGfd/AgCQGfd/AgCzGfd/AgC0Gfd/AgDSGfd/AgDTGfd/AgDkGfd/AgAd Gvd/AgBQGvd/AgCBGvd/AgCoGvd/AgCpGvd/AgC3Gvd/AgC4Gvd/AgD0Gvd/ AgAzG/d/AgBjG/d/AgBkG/d/AgChG/d/AgDGG/d/AgDHG/d/AgDPG/d/AgD6 G/d/AgD7G/d/AgAYHPd/AgAZHPd/AgBMHPd/AgBNHPd/AgB5HPd/AgB6HPd/ AgCoHPd/AgCpHPd/AgDZHPd/AgACHfd/AgAQHfd/AgBJHfd/AgCAHfd/AgC3 Hfd/AgDgHfd/AgDhHfd/AgAbHvd/AgA6Hvd/AgA7Hvd/AgBXHvd/AgBYHvd/ AgCTHvd/AgDNHvd/AgDtHvd/AgDuHvd/AgAWH/d/AgAXH/d/AgBSH/d/AgCN H/d/AgCpH/d/AgCqH/d/AgDJH/d/AgDKH/d/AgADIPd/AgAmIPd/AgArIPd/ AgA/IPd/AgBAIPd/AgBBIPd/AgBFIPd/AgBiIPd/AgBrIPd/AgCVIPd/AgCs IPd/AAAAAAAAAAAAAAAAZAABgAEBAAMBgAQBAAYBgAcBAAkBgAoBAAwBgA0B AA8BgBABABIBgBMBABUAAAAAAAAAAAAAAABkAAGwAQFgAwEQBQHABgFwCAEg CgHQCwGADQEwDwHgEAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-00008572-- ------------------------------ Date: Fri, 30 May 1997 08:28:53 -0500 (CDT) From: Biaz Saad Subject: A pointer to J.C Hoe work Hi! Can anybody provide a pointer to the work of J.C. Hoe ? Thanks. Saad Biaz ------------------------------ Date: Fri, 30 May 97 09:42:44 PDT From: Keith Scott Subject: Re: A pointer to J.C Hoe work Maybe a pointer to http://ana-www.lcs.mit.edu/anaweb/papers.html should be added to the IPOS home page? --keith >Hi! > Can anybody provide a pointer to the work of J.C. Hoe ? > > Thanks. Saad Biaz - ----------------------------------------------------------------------------- Keith Scott kscott@zorba.jpl.nasa.gov Jet Propulsion Laboratory 4800 Oak Grove MS 161-260 (Voice) +1.818.354.9250 Pasadena, CA 91109-8099 (FAX) +1.818.393.4643 - ----------------------------------------------------------------------------- ------------------------------ End of tcp-over-satellite-digest V1 #23 *************************************** tcp-over-satellite-digest Monday, June 2 1997 Volume 01 : Number 024 In this issues: TCPSAT: Draft Outline Re: Prob. Area 1 Repost: Questions (1 of 2) Repost: Questions (2 of 2) Re: Mail list problem fixed Re: TCPSAT Temporary Editor Re: TCPSAT Temporary Editor Mark's outline re: Repost: Questions (1 of 2) Re: Mark's outline Re: TCPSAT Temporary Editor See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Sun, 01 Jun 1997 21:56:15 -0400 From: Mark Allman Subject: TCPSAT: Draft Outline Hello! I have just revised Erik's outline into something that I think looks like a rough outline for the draft. So, any good ideas in the following can be attributed to Mr. Travis and all mistakes should be sent to me. Unfortunatly I do not have the entire mailing list archive saved. So, if you sent comments which are not reflected in this outline previously, please drop me a short private note and I will make sure your input is included next time. One thing that may seem noticably missing is any mention of non-TCP mechanisms that might help. I couldn't decide where or if to include these mechanisms. On the one hand, mechanisms like RED, HTTP 1.1, MTU discovery, etc. may be a big help over satellite channels (as well as others). On the other hand, the name of the BOF was *TCP* Over Satellites. It seems to me that we could easily degenerate into "here is how to build a good network", which seems well beyond the scope of the draft. So, for the time being, I have left out such mechanisms. I would like to focus on TCP first and worry about these other mechanisms later (if at all). Of course, if you think this is way off the mark, don't be shy... According to the timeline defined in Memphis we have about a month to release a first cut of this draft (due on 6/30). So, I would appreciate it if you folks would give some feedback on this outline. This feedback might include (but is not limited to): (1) defining the problem/solution in a better manner than I have (not hard), (2) alerting me of references to work that I forgot, (3) ripping apart those sections you disagree with, (4) expanding on something I have said to make sure that we are all on the same page, (5) adding sections that I forgot, (6) home made brownies with a note that says "nice job Mark". So, esstentially lets just hash this thing out. Thanks in advance, allman - --- 1. Introduction Comparison of the differences between satellite and non-satellite networks. Example topologies. Don't want to introduce TCP mechanisms that will degrade performance in non-satellite environments. 2. Problems 2.1 Large Delay*Bandiwdth Product 2.1.1 Discussion of the problem. Problem is "long" networks require bigger windows and satellite channels are very long. 2.1.2 Possible mitigation techniques [JBB92] (RFC 1323) [JBB97] (Current I-D updating 1323) (these TCP extensions allow bigger windows) 2.1.3 Experimntal Studies [AHKO97] (bigger windows help over satellite channels) [VS95] (shows that it works well in terrestrial networks too) 2.1.4 Results Recommend that large window extensions be added to TCP. 2.2 Long Feedback Channel 2.2.1 Slow Start and Congestion Avoidance 2.2.1.1 Discussion of the problem. Increasing the window is always in response to an ACK. Since ACKs take a long time to come back the increase is much slower than over "shorter" networks. 2.2.1.2 Possible mitigation techniques [Flo97] (suggests using a larger initial window) [AHO97] (suggests using a new window increase algorithm) [Bra94] (defines T/TCP which requires less startup time) 2.2.1.3 Experimental Studies [All97] (tests of the above suggestions over emualted satellite links) [AHO97] (tests of the above suggestions over the Internet) 2.2.1.4 Results Nothing can be recommended at this time. These are still research proposals, not standards track mechanisms. 2.2.2 Loss Recovery 2.2.2.1 Discussion of the problem. ACKs are used to detect loss (in many cases). Since it takes a long time to get ACKs back, it takes the sender a long time to figure out what has been lost and retransmit it. 2.2.2.2 Possible mitigation techniques [MMFR96] (provides selective acknowledgement extensions to TCP) [Hoe96] (provides new mechanisms for detecting which segments have been dropped without SACK) [FF96] (provides a fast recovery replacement based on SACKs) [MM96a], [MM96b] (provides another fast recovery replacement (FACK) based on SACKs) [Hoe96], [BOP94] (provide mechanisms for loss prevention) 2.2.2.3 Experimental Studies [FF96] [MM96a], [MM96b] [Hoe96] [Hoe96], [BOP94] (these papers provide evaluations of the mechanisms they introduced; these tests are over terrestrial or simulated networks, not satellite networks) [AHKO97] (evaluation of the mechanism introduced in [FF96] over satellite circuits) 2.2.2.4 Results SACKs should be implemented. The recovery mechanisms are still research issues. 2.4 Asymmetric Channels 2.4.1 Discussion of the problem. 2.4.2 Possible mitigation techniques 2.4.3 Experimental Studies 2.4.4 Results [I am not in tune with research specific to TCP over asymmetric channels on this area... Please help!] 2.5 Errors 2.5.1 Discussion of the problem. 2.5.2 Possible mitigation techniques 2.5.3 Experimental Studies 2.5.4 Results [I am not in tune with this this area much at all... Please help!] 2.6 Others??? 3. Interacting Mitigations Discussion of how the above mitigations might interfere with one another. 4. Conclusions and Future Work 4.1 Recommendations Large Window TCP Extensions SACK TCP Extensions 4.2 Areas Requiring Further Study Slow Start Modifications Congestion Avoidance Modifications Loss Recovery Techniques Asymmetric Channel mechanisms (???) High bit error mechanisms (???) References [AHKO97] Mark Allman, Chris Hayes, Hans Kruse and Shawn Ostermann. TCP Performance Over Satellite Links. In Proceedings of the 5th International Conference on Telecommunication Systems, March 1997. [AHO97] Mark Allman, Chris Hayes and Shawn Ostermann. An Evaluation of TCP Slow Start Modifications. In preparation. [All97] Mark Allman. Improving TCP Performance Over Satellite Channels. Master's Thesis, Ohio University, June 1997. [BOP94] Lawrence Brakmo, Sean O'Malley and Laryy Peterson. TCP Vegas: New Techniques for Congestion Detection and Avoidance. In ACM SIGCOMM, pages 24-35, August 1994. [Bra94] Robert Braden. T/TCP - TCP Extensions for Transactions: Functional Specification, July 1994. RFC 1644. [FF96] Kevin Fall and Sally Floyd. Simulation-based Comparisons of Tahoe, Reno, and SACK TCP. Computer Communications Review, July 1996. [Flo97] Sally Floyd, February 1997. Note to end2end-interest mailing list. [Hoe96] Janey Hoe. Improving the Start-up Behavior of a Congestion Control Scheme for TCP. In ACM SIGCOMM, August 1996. [JBB92] Van Jacobson, Robert Braden and David Borman. TCP Extensions for High Performance, May 1992. RFC 1323. [JBB97] Van Jacobson, Robert Braden and David Borman. TCP Extensions for High Performance, February 1992. Internet-Draft draft-ietf-tcplw-high-performance-00.txt (work in progress). [MMFR96] Matt Mathis, Jamshid Mahdavi, Sally Floyd and Allyn Romanow. TCP Selective Acknowledgement Options, October 1996. RFC 2018. [MM96a] Matt Mathis and Jamshid Mahdavi. Forward Acknowledgement: Refining TCP Congestion Control. In ACM SIGCOMM, August 1996. [MM96b] Matt Mathis and Jamshid Mahdavi. TCP Rate-Halving with Bounding Parameters. Technical Report, Pittsburgh Supercomputer Center, October 1996. [VS95] Curtis Villamizar and Cheg Song. High Performance TCP in ANSNET. Computer Communications Review, 24(5):45-60, October 1995. ------------------------------ Date: Sun, 1 Jun 1997 22:51:46 -0400 (EDT) From: Eric Travis Subject: Re: Prob. Area 1 [ Please send responses to this and any future correspondence to: e.j.travis@ieee.org regardless of what my headers say] Dan, I'm guessing my first round of responses on this fell into the mailing list's black hole (shortly followed by me :o( ); It *was* a well timed prod; Things are a bit more desperate now; So... >Is this a "what to look for in a TCP?" Should we discuss various TCP >heritages or is that too confusing? It would help me to have the >introduction to the draft to look at to help keep me focused. Well, that's the crux of my problem - scope. I'm slightly troubled. I've been following the traffic on the list that I've gotten, and I guess it's necessary to ask a few probing questions (of everyone); I'll formalize the responses into the new document outline and introduction. The direction will be driven by the consensus of the responses and the scope will reflect what effort folks volunteer. We need to get this mule in gear. The document needs to be rather concise, so my inclination is to *drop* any explicit discussion of experimental results (they are far too topology dependent) and concentrate on the problems and mitigations. Experimental results can be referenced as seperate documents. This is pretty much anarchy, so articulate dissension is encouraged. With the exception of Mark Allman and yourself, we've got one other volunteer that I'm aware of - me; If I've missed someone, I apologize, but it is no longer time to be shy. Nothing needs to be polished prose within the next two weeks, but it does need to have content. Please see (and respond to) the following *two* (2) messages; Eric (e.j.travis@ieee.org) ------------------------------ Date: Sun, 1 Jun 1997 22:54:40 -0400 (EDT) From: Eric Travis Subject: Repost: Questions (1 of 2) Folks, In order to get inputs as to the scope and direction of our collective effort, I'd appreciate input on the following three questions. How about a deadline of Friday 6/5? Thanks! As you can probably glean from the below, I've got some biases... o What do you envision the outcome of this effort to be? o What do you hope to get out of the resulting document? For Example: - What to look for in a TCP implementation (and demand from your vendor)? - What *isn't* broken in TCP and how to provide network services over a long-delay path? - Identification of the aspects of TCP (as well as IP, applications, etc.) that are inherently unfair to, or perform suboptimally in long-delay networks? - What can be done in the near-term to mitigate any unfairness without degrading non-long-delay path performance? - What can't be done with respect to speed-of-light issues? - Why should hosts not generally subject to long delay paths (such as a popular web-server) bother in participating in implementing any identified mitigations? - What looks promising for further down the road? o What can you sign up for based upon your above answers? The last question is a call for more volunteers, not a requisite for answering the first two questions. ------------------------------ Date: Sun, 1 Jun 1997 22:59:26 -0400 (EDT) From: Eric Travis Subject: Repost: Questions (2 of 2) In order to better gauge what the problems people are encountering in using TCP over satellite links, I'd appreciate if folks could respond to the following simple questions ASAP (by Friday 5/16 is best); What I'm looking for right now, is a quick response, not an detailed analysis. The folks that *really* need to respond are: o Operational providers (VSATs and above!) o Soon to be operational providers o The TIA "problem statement" that spawned this effort o Those who have "different" environments (such as various Space Agencies, non-satellite high-latency, etc) Questions: A. What are the characteristics of your environment? 1. Bandwidth (forward and reverse channel data rates available on a per-connection basis) 2. Delay of long-delay path (satellite, wireless, fiber, whatever) 3. Guesstimate of the end-to-end delay 4. What is the closest configuration depicted below (1->5) or provide a new/fixed configuration 5. General comments/observations B. What, in your opinion, is the problem? o Application domain problems? o Performance related (difficult to tune end-to-end)? o Protocol related (problems with TCP, IP or something else)? C. What mitigations are you aware of that *might* help? o Are the mitigations symmetric (must be applied at both hosts) or asymmetric (need only be applied at one side of the connection)? o If the mitigation is asymmetric, how likely is it that you can influence it occurring? If anyone would like their name and organization filtered from responses, send them directly to me. I will anonymize them and keep things confidential. ======================================================== Representative Configurations ======================================================== Note: I have *not* been terribly careful with my verbage below; I'd appreciate corrections, but would prefer them to be offline (e.j.travis@ieee.org). Conventions HA/HB: arbitrary hosts; may be single hosts or represent a number of hosts on a LAN )O( : Representative of the satellite hop in the end-to-end path CCC : A subnetwork "cloud" "clouds" with an "i" represent subnetworks managed/owned by the same entity providing connectivity to a depicted host. The network infrastructure within these clouds is within the control of the service provider. It should be possible to apply mitigations within this cloud; "clouds" with an "I" represent traffic routed over a common carrier/Internet backbone. The network infrastructure within these clouds is not within the control of either provider. It is not realistic to require mitigations here. R : select or significant routers; There are other routers implied in the cartoons below; Case 1: Single satellite hop, no routed intermediate subnets +--+ +--+ |HA|-----)O(------|HB| +--+ +--+ Case 2A: Single satellite hop, traffic is bridged (filtered, but not routed) to the intermediate subnet/intranet on one side of long-delay path CCC CC CC +--+ CC CC |HA|-----)O(---CC i CC +--+ CC CC C CC CC CC CC CC CC | | | +--+ |HB| +--+ Case 2B: Single satellite hop, traffic is routed to the intermediate subnet/intranet on the other side of the long-delay path CCC CC CC +--+ +--+ CC CC |HA|-----)O(--|R |-CC i CC +--+ +--+ CC CC C CC CC CC CC CC CC | | +--+ |HB| +--+ Case 3: Single satellite hop, intermediate subnets (intranets) on either side of long-delay path CCC CC CC +--+ CC CC |HA|---CC CC +--+ +--+ CC i CC-|R |--)O( C CC +--+ | CCC CC CC CC | CC CC CC CC +--+ CC CC |R |---CC i CC +--+ CC CC C CC CC CC CC CC CC | +--+ |HB| +--+ Case 4: Single satellite hop, intermediate subnets/intranets) on either side of the long delay path and traffic is routed over the "Internet" CCC CC CC +--+ CC CC |HA|---CC CC +--+ +--+ CC i CC-|R |---)O(--+ C CC +--+ | CC CC CC | CC CC | +--+ |R | +--+ | | | CCC CCC CC CC CC CC CC CC +--+ CC CC CC i CC---|R |---CC I CC CC CC +--+ CC CC C CC C CC CC CC CC CC CC CC CC CC CC CC | | +--+ |HB| +--+ Case 5: (included for "completeness" (?)) Unbeknownst to any of the *either* service provider involved, the traffic over the backbone provider networks is being routed over a satellite hop. CCC CCC CC CC CC CC CC CC +--+ CC CC CC i CC---|R |---CC I CC CC CC +--+ CC CC C CC C CC CC CC CC CC CC CC CC CC CC CC | | | | +--+ +--+ |HA| |R | +--+ +--+ | | )O( | +--+ |R | +--+ | CCC CCC CC CC CC CC CC CC +--+ CC CC CC i CC---|R |---CC I CC CC CC +--+ CC CC C CC C CC CC CC CC CC CC CC CC CC CC CC | | +--+ |HB| +--+ ------------------------------ Date: Sun, 1 Jun 1997 23:03:26 -0400 (EDT) From: Eric Travis Subject: Re: Mail list problem fixed Aaron, Is there (he asked hopefully) an archive of the mailing list available? Eric (e.j.travis@ieee.org) ------------------------------ Date: Mon, 2 Jun 1997 00:14:39 -0400 (EDT) From: Eric Travis Subject: Re: TCPSAT Temporary Editor I suppose it is fairly schizophrenic for me to disagree with myself below, but: > ====================================================== > Draft Outline as Presented/Discussed in Memphis: > > > I. Purpose > > a. Ensure that any and all recommendations resulting > from this effort will not adversely impact other > traffic on the shared (non-satellite) media. > > b. Provide a comprehensive document describing the > problems (and potential solutions) associated with > TCP performance in the satellite environment. b. Provide a comprehensive document describing the problems (and potential mitigations) associated with the performance of TCP based applications in satellite and other long-delay environments. Although the primary focus of this effort is on TCP, cause/effect and mitigation at the IP and application layers must also be considered. > > c. Identify, explain *and* wherever possible > quantify impacts of satellite environment on > TCP performance Strike out (or at least strongly down-play) all references/discussions related to quantification. This is too topology, application and flow dependent to be able to provide a solid, validated and defensible generalized treatment in the time available. We can speak in generalities, but I think anything more is pushing it. Pointers to supporting published experimental results in the list of references is (in my opinion) the best way to provide quantifications. > Quantification: > Simulation OK, but attempts should be made to > validate simulations > > Emulations better, same desires regarding > validation apply > > Live measurements most desirable, but probably > most elusive > > Bottom line, whatever is available is desirable; > Please contribute any relevant results and efforts > wherever possible. > > Quantifications must *not* be conducted in a vendor > specific manner. Impacts of base algorithms and common > (or proposed) implementation techniques are acceptable > (as are pointers to published work covering this area). > > I'm of the opinion that discussions/categorizations > along the lines of 4.3BSD Tahoe, 4.3BSD Reno, 4.4BSD, > Net/1, Net/2, Net/3, etc. are OK. I'm willing to > revise my opinion here if necessary. In no uncertain > terms are proprietary algorithms, implementations, etc. > going to be acceptable. > > d. Categorize the problems according to those that can be > addressed through engineering (applying what we already > know) and those that require further research. > > For the latter, this is simply a tagging/identification > effort. Anything else is out of scope of this group. > > e. Identify and explain what mitigation techniques exist > and (in general) how to use them. > > Efforts should concentrate on the more mature, somewhat > well understood techniques/options such as those in > RFC 1323, RFC 2018, as well as others that have already > been discussed within this group (J.C. Hoe's work, the > proposed "4K initial cwnd", etc.). I'm not going to > attempt to provide a comprehensive listing here. > > Explanations should cover not only the specific performance > problems that a mitigation technique (attempts) to overcome, > but also the why, how and where applicable the problems > involved. > > As above, *nothing* proprietary is going to be acceptable. > > Along those lines, is the "how to use" coverage. This is > likely to be difficult and if it proves too difficult then > it should fall off the end of the document. Pointers/discussion > to things like using socket-options is probably OK, but not > things like how to use ndd on Solaris to tweak things. > > II. Overview of the problem domain > > A broad overview of the differences in the satellite > environment that cause difficulties for TCP. There > should be not effort to go into great detail here, that will > be done in the next section. > > I envision a main element of this section to be a > categorization of different architectures/topologies > (common) to satellite applications. This should prove > to be useful for the discussion of specific issues and > mitigation techniques in section 3. Furthermore, such a > categorization will allow for comparisons/contrasts to > be made by those in non-satellite environments. > > [Please see the note in the next section regarding scope.] > > III. Specific Issues > > =================================================== > Note regarding scope: > ----------------------- > There was some good discussion during Friday's session > in Memphis as to whether or not we should address the > performance impacts that applications and "link" > technologies have on TCP in the satellite environment; > > As an example at the application level, the (unnecessary) > interrogative nature of some applications, such as FTP > does impact perceived user-level performance in long > delay environments. I don't need to provide examples > related to link(ish) technologies. > > Although this is a gray area (for us), I'd like to > restate that *I* will be sympathetic to related > contributions or pointers on these topics. I won't > promise that these items will make it into the final > document - we'll have to see how things fall together. > Even if the contributions don't make it into this > Informational RFC, I'm sure we can prevent them from > going to waste. > > Also, let me be explicit that network layer topic > *are* within the scope of this effort. I expect to > see lots of content to be from this area. > =================================================== > > a. Identify and describe the problems related to TCP > performance in satellite environments. > > In addition, we wish to provide some validation as > to the level to which the problem is understood. > This will help sort the areas where engineering is > appropriate/possible and those areas that require > more research. Discussion of each of the below should address the relevance of the problem area to not only TCP, but also IP and applications. > A first cut at an ordered categorization of problem > *areas* is as follows: 0a. How either side of a connection can autonomously determining whether or not a connection includes a Long Feedback Delay Path. > 1. Large Bandwidth * Delay Product > > 2. Long Feedback Delay Path > > 3. Asymmetric Channels > > 4. Errors > Before anyone starts complaining about this, > there *are* environments where you get > error-rates worse than 10E-8 *after* > application of strong FEC; > > 5. ??? > > b. Identify, describe and *assess* the applicability of > currently available (or proposed) mitigation techniques > keyed to their associated problem areas. > > c. Tag those areas where engineering will not suffice as > areas promising for research. > > d. For each specific issue identified: > > 1. Problem Statement: > > 2. Semi-detailed explanation of the problem > > 3. Mitigation techniques identified > > 4. Discussion of mitigation techniques > > a. 1st Order discussion on technique > (isolated, no interactions) > > Assess the benefits and potential pitfalls > of this technique keyed to the different > topologies provided in section 2. This is > will prove to be important. > > b. If applicable, tag this area as one needing > further research. > > IV. Interacting Mitigations > > This section should *attempt* to address the potential > impacts of mixing the different mitigation techniques > identified/discussed above. > > V. Areas Requiring Further Engineering > > Pull together those areas identified in Section 3 that > can be handled through engineering, but which fall out > of scope of this group. > > V. Identified Research Areas > > Collect all the tagged research areas identified in > as requiring further research. > > VI. Recommendations VI. Observations and Conclusions (I've got reservations about this document providing *anything* labeled Recommendations). Eric (e.j.travis@ieee.org) BTW: Given the pending 6/30 deadline for the first draft, contributions really should be posted to the mailing list (and editors) at least 2 weeks prior (6/16) ------------------------------ Date: Mon, 2 Jun 1997 09:04:22 -0400 From: hkruse1@ohiou.edu (Hans Kruse) Subject: Re: TCPSAT Temporary Editor >> >> c. Identify, explain *and* wherever possible >> quantify impacts of satellite environment on >> TCP performance > > Strike out (or at least strongly down-play) all > references/discussions related to quantification. > This is too topology, application and flow dependent > to be able to provide a solid, validated and defensible > generalized treatment in the time available. We can > speak in generalities, but I think anything more is > pushing it. > I disagree. Quantitative results are needed and possible, without them we have really not accomplished much. I see one goal of this effort as convincing implementors (of both stacks and satellite networks) to "do the right thing" so users get the best service possible. Iimplementors will look for quantifiable results before taking the discussion seriously. As to the variables involved, we need to use the concept of a reference model, where the various experimental results can be categorized. I have started on something like that; if the group wants I can clean that up some more and post it. >> >> VI. Recommendations > > VI. Observations and Conclusions > >(I've got reservations about this document providing *anything* labeled >Recommendations). > Again, I think without recommendations (maybe in the spirit of BCP) this effort won't accomplish much. Hans Kruse, Associate Professor McClure School of Communication Systems Management, Ohio University 9 S. College Street Athens, OH 45701 614-593-4891 voice, 614-593-4889 fax, hkruse1@ohiou.edu ------------------------------ Date: Mon, 2 Jun 1997 08:58:00 -0400 From: matthew halsey Subject: Mark's outline Form: Memo Text: (10 lines follow) Overall I think the outline is fine, I would like to suggest that we address RFC 2001 Fast Recovery/Retransmit under section 2.5 as I believe they essentially address error recovery. Even if the satellite environment is relatively error-free, there will often be errors associated with adjoining terrestrial networks. The recovery from these errors will depend upon the rtt including the satellite portion. Implementation of RFC2001 will enable faster recovery from terrestrial based errors - in my humble opinion? Thanks Matt Use Proportional Font: true ------------------------------ Date: Mon, 2 Jun 1997 09:12:00 -0400 From: matthew halsey Subject: re: Repost: Questions (1 of 2) This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-003FF6BA Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7BIT Form: Reply Text: (33 lines follow) Here is my first pass: o What do you envision the outcome of this effort to be? I had envisioned that this Informational RFC could be sent to vendors to say; "This is what we want implemented in your off-the-shelf TCP/IP stack software". I feel that, in order to accomplish this, we need to explain why we want the features that we eventually recommend and explain how they are possibly of benefit to the terrestrial world also, or at least not detrimental. o What do you hope to get out of the resulting document? I would like to see a brief explanation of satellite networks, their major benefits such as quick deployment, large area coverage, terrain independant etc. etc. In this section we can explain the fundamental delay associated with GEOs (assuming we are limiting this to GEOs?). I would like to emphasize how we feel that TCP/IP works OK over satellite today but, in some areas, improvements could be made. E.g. Corporate high bandwidth Intranets could benefit from large windows. Public interfaced networks (terrestrial-to-satellite) could benefit from Fast Retransmit and Fast Recovery. There could also be benefit from Fast Start (Floyd) but as this is not standardized I am not sure we should recommend it at this time. In explaining how these features benefit satellite, we could also explain how they would potentially benefit similar terrestrial networks and why it is therefore important that the recommended RFCs be implemented. I hope this helps discussion. Matt Original text: (52 lines follow) From TRAVIS@SMTPGATE (Eric Travis) {travis%clark.net.@intelsat1.intelsat.int}, on 1/6/97 10:54 PM: To: TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com}, E@SMTPGATE {e.j.travis@ieee.org} Folks, In order to get inputs as to the scope and direction of our collective effort, I'd appreciate input on the following three questions. How about a deadline of Friday 6/5? Thanks! As you can probably glean from the below, I've got some biases... o What do you envision the outcome of this effort to be? o What do you hope to get out of the resulting document? For Example: - What to look for in a TCP implementation (and demand from your vendor)? - What *isn't* broken in TCP and how to provide network services over a long-delay path? - Identification of the aspects of TCP (as well as IP, applications, etc.) that are inherently unfair to, or perform suboptimally in long-delay networks? - What can be done in the near-term to mitigate any unfairness without degrading non-long-delay path performance? - What can't be done with respect to speed-of-light issues? - Why should hosts not generally subject to long delay paths (such as a popular web-server) bother in participating in implementing any identified mitigations? - What looks promising for further down the road? o What can you sign up for based upon your above answers? The last question is a call for more volunteers, not a requisite for answering the first two questions. Use Proportional Font: true Previous From: TRAVIS@SMTPGATE (Eric Travis) {travis%clark.net.@intelsat1.intelsat.int} Previous To: E@SMTPGATE {e.j.travis@ieee.org}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Original to: E@SMTPGATE {e.j.travis@ieee.org}, TCP-OVER@SMTPGATE {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-003FF6BA Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: BASE64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjZdYCgAAAAAAQmV5b25kIFBy b3ByaWV0YXJ5IERhdGEaAAAAABEAAAAAAAQADQBCAgAAAAAAAAAAAAAAAAAA AAAAAAAAT3JpZ2luYWwgdGV4dDIGRgoAAAAAAAAAAAAAMAIDADIGKgICAAIA AAAyAAIAAQABAMQAAAAAAAAAAgDFAG4FAAAAAAAAOP8AAAAAAACQAQAAAAAA AAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOP8AAAAA AACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAGQAAQBkAMQAAQDEAMUAAgDFAPd/AgDGAPd/AgDNAPd/AgDOAPd/ AgAKAfd/AgBJAfd/AgB4Afd/AgB5Afd/AgCBAfd/AgCCAfd/AgDEAfd/AgDF Afd/AgAAAvd/AgABAvd/AgA8Avd/AgA9Avd/AgBOAvd/AgCLAvd/AgCmAvd/ AgCnAvd/AgDdAvd/AgAOA/d/AgAPA/d/AgBMA/d/AgCHA/d/AgC/A/d/AgDA A/d/AgD6A/d/AgAzBPd/AgBIBPd/AgBJBPd/AgCDBPd/AgCTBPd/AgCUBPd/ AgDRBPd/AgAIBfd/AgA9Bfd/AgBSBfd/AgBTBfd/AgCLBfd/AgCMBfd/AgDI Bfd/AgDJBfd/AgAKBvd/AgAxBvd/AgAyBvd/AgAzBvd/AAAAAAAAAAAAAAAA ZAABgAEBAAMBgAQBAAYBgAcBAAkBgAoBAAwBgA0BAA8BgBABABIBgBMBABUA AAAAAAAAAAAAAABkAAGwAQFgAwEQBQHABgFwCAEgCgHQCwGADQEwDwHgEAGQ EgFAFAHwFQGgFwAAEQAAAAAABAAEAE8CAAAAAAAAAAAAAAAAAAAAAAAAAABU ZXh03AVIdAAAAAAAAAAAAAA9AgMA3AU2AgkABAAAABQAAgABAAEAGAAAAAAA AAACABkAAQAAAAAAAAADABoAOAAAAAAAAAACAFIAAQAAAAAAAAAEAFMAfAEA AAAAAAACAM8BAQAAAAAAAAADANABOgAAAAAAAAACAAoCAQAAAAAAAAAEAAsC 0gMAAAAAAAA4/wAAAAAAAJABAAAAAAAAAABNUyBTYW5zIFNlcmlmAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAA4/wAAAAAAAJABAAAAAAAAAABDb3VyaWVyAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAQ/wAAAAAAAJABAAEAAAECATFD b3VyaWVyAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAgAA4/wAAAAAAAJAB AAAAAAECATFDb3VyaWVyAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB AAEAFwABABgAGAACABkAUgACAFMAUwACAFQAzwECANAB0AECANEBCgICAAsC CwICAAwCIwMCACQDowMCAKQDpAMCAKUD7gMCAO8DWgQCAFsE4QQCAOIE4gQC AOMEuAUCALkFuQUCALoF1wUCANgF2AUCANkF3QUAAAAAAAAAAAAAAABkAAGA AQEAAwGABAEABgGABwEACQGACgEADAGADQEADwGAEAEAEgGAEwEAFQAAAAAA AAAAAAAAAGQAAbABAWADARAFAcAGAXAIASAKAdALAYANATAPAeAQAZASAUAU AfAVAaAXaQAA - --mime-boundary-iaanaabhmb-003FF6BA-- ------------------------------ Date: Mon, 02 Jun 1997 10:57:16 -0400 From: Mark Allman Subject: Re: Mark's outline Please disregard the outline I posted last night. If there are good things in the outline I am sure that Eric will re-introduce them at an appropriate time, now that he is back in the drivers seat. allman ------------------------------ Date: Mon, 02 Jun 1997 11:20:15 -0400 From: Mark Allman Subject: Re: TCPSAT Temporary Editor > > Strike out (or at least strongly down-play) all > > references/discussions related to quantification. > > This is too topology, application and flow dependent > > to be able to provide a solid, validated and defensible > > generalized treatment in the time available. We can > > speak in generalities, but I think anything more is > > pushing it. > > > > I disagree. Quantitative results are needed and possible, without > them we have really not accomplished much. I see one goal of this > effort as convincing implementors (of both stacks and satellite > networks) to "do the right thing" so users get the best service > possible. Iimplementors will look for quantifiable results before > taking the discussion seriously. I am afraid we are going to get mired down in trying to write down *everything* one needs to know about satellite communications. I vote for Eric's idea of keeping this document as short as possible. I think we can say things that indicate that a given mechanism has been tested in the satellite environment and works well (or not) and then cite the work. That is good. Re-running tests and/or restating results (other than in somewhat high level terms) seems like a waste of effort to me. > > VI. Observations and Conclusions > > > >(I've got reservations about this document providing *anything* > >labeled Recommendations). > > Again, I think without recommendations (maybe in the spirit of > BCP) this effort won't accomplish much. I think recommendations are needed, as well. However, there are precious few we can really make. I count three: 1323, 2001, 2018 (as far as TCP goes). The other mechanisms are still research issues that the IETF has not decided upon (more aggressive slow start, fast recovery replacement that uses SACKs, etc.). Of course, we can help with the process of deciding these issues. But, it seems to me that we cannot recommend things that the IETF does not, or we risk "competing" TCP implementations. allman ------------------------------ End of tcp-over-satellite-digest V1 #24 *************************************** tcp-over-satellite-digest Tuesday, June 3 1997 Volume 01 : Number 025 In this issues: Re: Mail list problem fixed Proposed WG Charter Re: TCPSAT Temporary Editor re: Repost: Questions (1 of 2) Re: Proposed WG Charter Re: Proposed WG Charter Re: Mark's outline Re: Proposed WG Charter comments: TCPSAT: Draft Outline Re: comments: TCPSAT: Draft Outline Re: Proposed WG Charter Re: comments: TCPSAT: Draft Outline Re: comments: TCPSAT: Draft Outline Re: Proposed WG Charter Re: Proposed WG Charter See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 02 Jun 1997 09:49:02 -0700 From: Aaron Falk Subject: Re: Mail list problem fixed Eric Travis wrote: > Aaron, > > Is there (he asked hopefully) an archive of the mailing > list > available? Here is a response from sending 'info tcp-over-satellite' to majordomo@listserv.trw.com: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Welcome to the TCP-over-Satellite Listserv. This email list is the online discussion mechanism for the Internet protocols working group. This group operates within the Telecommunications Industry Association (TIA) as follows: Telecommunications Industry Association, Satellite Communications Division (TR34) Communications Interoperability Section (TR34.1) Internet Procotols Working Group (TR34.1.1) Participation in the email discussion is open, however discussion should relate to issues relating to the working group's goal of seamless interoperability between satellites and the Internet. The working group co-chairs are Aaron Falk (Aaron.Falk@trw.com) and Scott Corson (corson@Glue.umd.edu) To send to the email discussion list, send email to: tcp-over-sattelite@listserv.trw.com To view and get files from the TCP-over-Sattelite digest, use the following commands by sending email to: majordomo@listserv.trw.com commands: "index tcp-over-sattelite-digest" (to retrieve an index of the digest) "get tcp-over-satellite-digest file_name" (where 'file_name' is the name of the file being requested) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ When I sent the request 'index tcp-over-sattelite-digest' I received the following response (note that the digest files are created once per week): >>>> index tcp-over-satellite-digest total 507 -rw-rw---- 1 majordom 4402 Jan 25 01:05 v01.n001 -rw-rw---- 1 majordom 706 Feb 1 01:05 v01.n002 -rw-rw---- 1 majordom 33865 Feb 8 01:05 v01.n003 -rw-rw---- 1 majordom 40653 Feb 13 22:09 v01.n004 -rw-rw---- 1 majordom 13509 Feb 15 01:05 v01.n005 -rw-rw---- 1 majordom 16271 Feb 22 01:05 v01.n006 -rw-rw---- 1 majordom 3237 Mar 8 01:05 v01.n007 -rw-rw---- 1 majordom 8665 Mar 15 01:05 v01.n008 -rw-rw---- 1 majordom 13032 Mar 29 01:05 v01.n009 -rw-rw---- 1 majordom 3821 Apr 5 01:05 v01.n010 -rw-rw---- 1 majordom 11174 Apr 12 01:05 v01.n011 -rw-rw---- 1 majordom 41290 Apr 14 13:54 v01.n012 -rw-rw---- 1 majordom 41495 Apr 17 09:56 v01.n013 -rw-rw---- 1 majordom 926 Apr 19 01:05 v01.n014 -rw-rw---- 1 majordom 41168 Apr 23 11:33 v01.n015 -rw-rw---- 1 majordom 44756 Apr 24 11:52 v01.n016 -rw-rw---- 1 majordom 41345 Apr 25 12:54 v01.n017 -rw-rw---- 1 majordom 17625 Apr 26 01:05 v01.n018 -rw-rw---- 1 majordom 12024 May 3 01:05 v01.n019 -rw-rw---- 1 majordom 29575 May 10 01:05 v01.n020 -rw-rw---- 1 majordom 667 May 17 01:05 v01.n021 -rw-rw---- 1 majordom 9564 May 24 01:05 v01.n022 -rw-rw---- 1 majordom 37753 May 31 01:05 v01.n023 -rw-rw---- 1 majordom 40531 Jun 2 08:19 v01.n024 >>>> Hope this helps. Aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Mon, 02 Jun 1997 09:59:47 -0700 From: Aaron Falk Subject: Proposed WG Charter Since there has been some discussion regarding whether we should make recommendations in the RFC, I thought I should post the WG charter that will be submitted to the IESG. Allyn and I worked this out last Friday. And it should be submitted in the next couple of weeks. (She's out of town this week) So, this has yet to be approved and the IESG may recommend some changes. Aaron - ------------------------------------------- TCP over Satellite (TCPSAT) - ------------------------------------------- Chair(s): Aaron Falk Transport Area Directors: Scott Bradner Allyn Romanow Area Advisor: Allyn Romanow Mailing lists: General Discussion: tcp-over-satellite@listserv.trw.com To Subscribe: majordomo@listserv.trw.com, with "subscribe tcp-over-satellite" in the message body Archive: majordomo@listserv.trw.com, with "index tcp-over-satellite-digest" in the message body HTTP: http://www.isr.umd.edu/CSHCN/Links/IPoS/ Description of Working Group: Satellites are being used to extend the Global Internet geographically and will be more ubiquitous in the next decade with the deployment of several proposed services capable of providing greater than T1 access to individual users anywhere in the world. Yet, satellite links have a unique combination of characteristics that can affect the throughput of TCP traffic. Because of the high-bandwidth delay product, slow start and congestion control algorithms behave much differently when the path includes a satellite link than exclusively terrestrial ones. The TCP over Satellite Working Group shall produce an informational RFC which describes the issues affecting TCP throughput over satellite links. It identifies the domains in which each issue applies; fixes, both protocol and implementation, that ameliorate reduced throughput; and areas for further research. The purpose of including this information is to direct the research community to the areas in which show promise, not to perform the research or even advocate the results. Scope: The scope of this working group will include consideration of the following for inclusion in the RFC: o Transport layer issues affecting TCP over satellite links o Existing TCP options o Compliant implementations which have some known improved performance over satellite links o Recommendation of well understood protocol changes o Identification of protocol changes that are potentially promising but require more further investigation in order to be well understood SECURITY ##Allyn to provide boilerplate## Goals and Milestones: The major goal of this group is to produce a document that describes the problems involved in running TCP over satellite links, known solutions, and areas for further research. Key Milestones: May 1997 Solicit of volunteers June 1997 Post first draft July 1997 Post second draft August 1997 Munich IETF: Review Draft 2 September 1997 Post third draft October 1997 Post fourth draft November 1997 Post fifth draft December 1997 DC IETF: Achieve consensus for final Internet Draft January 1997 Submit Internet Draft to the IESG for consideration as an Informational RFC - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Mon, 2 Jun 1997 14:04:13 -0400 (EDT) From: Eric Travis Subject: Re: TCPSAT Temporary Editor Hans, I was hoping for discussion :o) > I disagree. Quantitative results are needed and possible, > without them we have really not accomplished much. I see > one goal of this effort as convincing implementors (of both > stacks and satellite networks) to "do the right thing" so > users get the best service possible. Implementors will look > for quantifiable results before taking the discussion seriously. I think we can accomplish plenty even without providing quantifications within the text of the document (pointers to existing quantifications are what I am advocating). Any new experimental results will need supporting documentation and analysis. I think much of this is beyond the scope of this document. If this is done elsewhere and we can provide a reference to it, fine. Having been in similar positions before, I'm confident we can spend the next six months working at reaching consensus on the appropriate Reference Model and the set of parameters to be used in testing. Unless you intend to address a point-solution, the topologies and applications involved are going to be widely varied. Furthermore, I no longer believe that quantitative results are *required* to get implementors to start taking this *discussion* seriously. In the long-run, quantification will be required to effect real change in the IETF (prove your solutions work and are at least not detrimental to other traffic; For bonus points, show how it helps other traffic). Like I said in Memphis, I view this WG to be the *beginning* of a solution, perhaps providing short-term mitigations but not long-term solutions. Evolution takes longer than 9-months. What motivates vendors is market-share. What drives market-share (other than good marketing) is capability and performance. The problem space that is being addressed is nothing new, implementors are aware of the general performance problem. I believe that the vendors are already acting in ways that support the cause. For example RFC-1323 options are becoming more readily available. I believe that *if* there are implementors who are not already paying attention either don't see a problem (they view performance problems to be due to application inefficiencies, misconfiguration or insufficient engineering of the network infrastructure) or the satellite market as being big enough (profitable) for them to dedicate resources. There is merit in the former view and addressing the latter is up to the satellite industry, not the IETF to convince them of their folly. Quantifications will be absolutely necessary when the time comes to reach consensus on which *mitigations* are to be pursued and recommended. In my opinion, that's going to take longer than the time alloted to this working group. Rather than scatter efforts, I'd prefer to concentrate on what we can document and reach rough consensus on prior to December. > As to the variables involved, we need to use the concept of a > reference model, where the various experimental results can be > categorized. I have started on something like that; if the group > wants I can clean that up some more and post it. Developing a reference model that can address the generic satellite (or long-delay) environment is going to be hard. My reposted questions and topologies of last evening are my first efforts to develop a workable framework for discussion. If this can be used for the design of experiments for obtaining quantitative results, that would be excellent. That was one of the goals I stated back in Memphis. I'd like to work with you and see if we can't come up with a single framework that suits both discussion and experimentation. If you don't post to the group, could you at least send me (e.j.travis@ieee.org) a copy when you are comfortable? I don't want to discourage experimentation, just the opposite. However, I also don't want to fracture the available brain-power just yet. At this stage, if people are trying to decide between documentation of what they already know and performing more experiments, I'd encourage them to do the documentation *first*. I *do* want to see some good experiment design done (and this list is probably a great place to do it), but I don't want to mortgage the document on the hopes that we can design, conduct and evaluate all the tests (or if need be simulations) necessary prior to December. I felt differently in January, but that was over 4 months ago... > >> VI. Recommendations > > > > VI. Observations and Conclusions > > > >(I've got reservations about this document providing *anything* > >labeled Recommendations). > > > > Again, I think without recommendations (maybe in the spirit of BCP) > this effort won't accomplish much. Again, I think we can accomplish a great deal without providing a set of things labeled as "Recommendations". Given that this is going to be (at most) an informational RFC, I honestly believe that Recommendations that already are not IETF Recommendations are inappropriate. If something doesn't meet a litmus test of being "well-understood", I'm against putting a label of "Recommendation" anywhere near it. There danger of providing a "recommendation" in an informational RFC is that an otherwise well meaning implementor might not notice the *informational* and assume that the recommendation has the endorsement of the IETF. Paranoid? Yes, but I do worry about such things. Mark and I had this discussion well over a month ago, and although it had slipped my mind last night, he did convince me to include a recommendation for requiring RFC 1323, 2001, 2018. He was (and still is) absolutely correct. These are certainly things one should look for in a vendor's TCP. In my mind, these should be "recommended" there, but this (and everything else) is open to discussion. On the other-hand, in terms of being a motivating force, conclusions and observations should be a sufficient force for effecting change. Calling them "recommendations" is a matter of semantics, but I see nothing positive in providing such a label (see my paranoia above). This document (being informational) will *not* be undergoing a rigorous peer-review outside of this group. The time-line is sufficiently short (and the participants relatively uncoordinated) that I'm already insecure about the absolute correctness of any conclusions. If something isn't already an IETF Recommendation, I'd rather not call it a recommendation in this document. Let's have further discussion on this point (please). In the interest of further discussion, I see the main goals of this effort to be as follows: o Documentation of the problems space o Providing documentation of "best-practices" in configuring end-systems and intermediate systems for operation in the satellite environment o Providing guidance to system designers/managers as to what to look for in a TCP implementation. o Being a catalyst for the long-delay problems which are not due to misconfiguration and which have no standards track proposals in the IETF. My staged view of how to proceed in attacking the problem we are addressing is as follows: 1. Understanding the causes of performance degradation in long-delay environments 1a. Drawing in parallels/similarities to *other* environments (terrestrial radio, wireless, mobile, long-fiber, etc.) This helps in solving the bigger/more fundamental problems and increasing the size of the constituency. 2. Documentation of the above phenomenon 3. Identification of workable mitigations addressing the long-delay environment. These mitigations will *not* be segregated to just the transport protocol. 4. Evaluation of the mitigations identified in 3. This process must include assessing the effects of the mitigations on other (non-long-delay) traffic as well as the possible interactions between different mitigations. 5. Extended testing of the mitigations to validate the results of step 4. 6. Convincing *all* implementors to include the successful mitigations in their products. 7. Convincing *everyone* (clients, servers and intermediate infrastructure) to upgrade so that they support the mitigations. I see the main role of this effort being one of documentation and first order evaluation. My scoping is mainly due to the short time-line available. The world doesn't end in December, so view this working group as a step on the longer trail. Comments please. Eric (e.j.travis@ieee.org) ------------------------------ Date: Mon, 2 Jun 1997 14:32:46 -0400 (EDT) From: Eric Travis Subject: re: Repost: Questions (1 of 2) Matt, Terrific - the first of what I hope will be many more. I'll be collecting and will summarize them all; > with GEOs (assuming we are limiting this to GEOs?). Well, I'd prefer to include LEOs (and HEOs) if possible. I'm aware of some planned/proposed LEOs that are *not* going to be part of a constellation, but are going to be doing cool science; These beasts have a different set of problems with TCP (and maybe IP) but I'd like to include them if possible (read: responsibility in generating text is mine, the group decides whether it fits in with everything else). The corporate line on LEO constellations *seems* to be that they don't have problems with long-delays - if that is true (yes, I'm skeptical) then LEO constellations need not be addressed. Although, as the number of birds in a constellation drops (economics) I'm imagining all sorts of things that change from original specs. If there are no contributions, there will be no coverage. I'm hoping for contributions. > I would like to emphasize how we feel that TCP/IP works OK over satellite > today but, in some areas, improvements could be made. Bless you! I like the way you think :o) > I hope this helps discussion. Thanks! Eric ------------------------------ Date: Mon, 2 Jun 1997 14:43:02 -0400 (EDT) From: Eric Travis Subject: Re: Proposed WG Charter > o Recommendation of well understood protocol changes I read the "well understood protcol changes" to be things that are already standards track items. Are there other items that I should be putting in the basket with these? ------------------------------ Date: Mon, 02 Jun 1997 12:00:35 -0700 From: Aaron Falk Subject: Re: Proposed WG Charter Eric Travis wrote: > > o Recommendation of well understood protocol > changes > > I read the "well understood protcol changes" to be things > that > are already standards track items. Are there other items > that > I should be putting in the basket with these? My original wording was "small" instead of "well understood" and I changed it to make it more clear (I thought). The distinction is between protocol changes a working group could develop and ones that imply a new TCP, which, for a variety of reasons, we don't want to advocate. If we can find changes that we feel are well understood and meet the other constraints of the working group (terrestrially benign, etc) then we can recommend them. I think this is the catagory of small changes that includes raising the initial slow start window to 4k, although we haven't reached a consensus yet on that one. It's important to remember that what we are producing is an Informational RFC not the protocol changes themselves. Based on our recommedation, a working group could be created to make those changes to the protocol. aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Mon, 2 Jun 1997 14:38:28 -0400 (EDT) From: Eric Travis Subject: Re: Mark's outline > > Please disregard the outline I posted last night. If there are good > things in the outline I am sure that Eric will re-introduce them at > an appropriate time, now that he is back in the drivers seat. > Don't listen to him - comment on it, Mark and I will work out a revised outline based on all comments. Mark has received a calling from a higher power and just can't walk away from it that easily :o) Eric ------------------------------ Date: Mon, 2 Jun 1997 15:48:23 -0400 (EDT) From: Eric Travis Subject: Re: Proposed WG Charter Aaron, > My original wording was "small" instead of "well understood" > and I changed it to make it more clear (I thought). The > distinction is between protocol changes a working group > could develop and ones that imply a new TCP, which, for a > variety of reasons, we don't want to advocate. I think the wording is appropriate. "Well understood" is very different than "small"; Either can imply a new TCP (off topic but: new is not necessarily bad if it is interoperable...) but I think the important aspect is having confidence that we know the impacts of the change. Small changes (sometimes disguising themselves as programming errors) can have profound effects, if we don't understand the impacts prior to making such a recommendation then I feel that we would be acting in an irresponsible manner. > > If we can find changes that we feel are well understood and > meet the other constraints of the working group > (terrestrially benign, etc) then we can recommend them. I > think this is the catagory of small changes that includes > raising the initial slow start window to 4k, although we > haven't reached a consensus yet on that one. Because we're not sure we understand the impacts in a system with many flows? > It's important to remember that what we are producing is an > Informational RFC not the protocol changes themselves. > Based on our recommedation, a working group could be created > to make those changes to the protocol. Are the "Recommendations" that (may) be part of the Informational RFC one in the same as the recommendations of the working group? In other words, if the document had a small set of recommendations that said something like "Use RFC-1323 and RFC-2008 on both sides of the connection" would it then be appropriate for the working group recommendations to be something like "Optional mechanisms that provide the following functionalities { X,Y,Z } but that do not have detrimental impacts on other traffic need to be developed and standardized"? If the above is appropriate, then what I would feel comfortable calling "Conclusions and Observations" in the Informational RFC would then become the Working Group Recommendations. If we (meaning the IETF community, not just the Working Group) understand it, then it is OK to make it a recommendation in the document; If we don't fully understand it, but we think it merits further work/development it is OK to make it a Working Group recommendation. How far off into the weeds am I? Eric ------------------------------ Date: Mon, 02 Jun 1997 16:04:28 -0400 From: Dan Shell Subject: comments: TCPSAT: Draft Outline >1. Introduction >2. Problems > > 2.4 Asymmetric Channels > > 2.4.1 Discussion of the problem. Note that current versions of Router protocols OSPF,RIP,IGRP,EIGRP assume symmetrical networks. To get asymetric networks to run you need to set up static routes. Does anyone have experience with BGP? Does it support asymmetic networks? If we do not have a ready answer I can do some research and find out. Also DVMRP and PIM assume symmetrical multicast networks and in current version will not work over asymmetric networks. > > 2.4.2 Possible mitigation techniques Have an option for asymmetric routing protocols. One thing to watch out for would be split horizon problems. Also some sort of asymmetrical PIM. > > 2.4.3 Experimental Studies We need them > > 2.4.4 Results > > [I am not in tune with research specific to TCP over asymmetric > channels on this area... Please help!] ADSL type communication is also asymmetirical. Would you like me to do some digging around the ADSL areas and see what they have or have not come up with? > > 2.5 Errors > > 2.5.1 Discussion of the problem. One of the main problems is mitigating the high latency for the end to end congestion control. What I would like to see is some sort of mechanism for error control just over the high-latency link. > > 2.5.2 Possible mitigation techniques Maybe a rate-based congestion control mechanism? > > 2.5.3 Experimental Studies > > 2.5.4 Results > > [I am not in tune with this this area much at all... Please > help!] > >4. Conclusions and Future Work >References > Dan Shell CISCO Systems Engineer Federal Division ------------------------------ Date: Mon, 2 Jun 1997 13:43:34 -0700 From: touch@ISI.EDU Subject: Re: comments: TCPSAT: Draft Outline > > 2.4 Asymmetric Channels > > > > 2.4.1 Discussion of the problem. > > Note that current versions of Router protocols OSPF,RIP,IGRP,EIGRP assume > symmetrical networks. > To get asymetric networks to run you need to set up static routes. Which is like not running them. It can be worse than that. Even with static routes, if the other properties of the links are very different (shared vs. dedicated, high vs. low loss, etc), some protocols (routing, transport, or other) can also have problems. > > 2.4.2 Possible mitigation techniques > Have an option for asymmetric routing protocols. One thing to watch out for > would be split horizon problems. And cycles, esp. 'indirect' ones. - correct asymmetric routing _requires_ what can, in some protocols, be misinterpreted as a routing cycle violation. (I have experience with asymmetric link failures on a wired point-to-point network, and how such failures are compounded by running RIPv2 to do dynamic path selection) - this under the http://www.isi.edu/atomic2 project > > 2.5 Errors > > > > 2.5.1 Discussion of the problem. > > One of the main problems is mitigating the high latency for the end to end > congestion control. > What I would like to see is some sort of mechanism for error control just > over the high-latency link. High constant latency is not as bad as highly variable latency is. Otherwise, the convergence is just slower (that's a problem as well). > > 2.5.2 Possible mitigation techniques > Maybe a rate-based congestion control mechanism? We're working on one for TCP that avoids the slow-start restart after source data gaps, i.e., "slam the window if the source has been idle". Seems like similar pacing would be useful for a satellite link, pushing the flow control out to a second derivative. (this under the http://www.isi.edu/lsam/ project) Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ Date: Mon, 02 Jun 1997 16:18:52 -0700 From: Aaron Falk Subject: Re: Proposed WG Charter Eric Travis wrote: > > > > If we can find changes that we feel are well understood > and > > meet the other constraints of the working group > > (terrestrially benign, etc) then we can recommend them. > I > > think this is the catagory of small changes that > includes > > raising the initial slow start window to 4k, although we > > > haven't reached a consensus yet on that one. > > Because we're not sure we understand the impacts in a > system > with many flows? That is my understanding. A while back some folks raised that issue. I'd like to see some discussion on this list to either quantify where the uncertainty lies or put it to bed so we can advocate it. > > It's important to remember that what we are producing is > an > > Informational RFC not the protocol changes themselves. > > Based on our recommedation, a working group could be > created > > to make those changes to the protocol. > > Are the "Recommendations" that (may) be part of the > Informational > RFC one in the same as the recommendations of the working > group? > > In other words, if the document had a small set of > recommendations > that said something like "Use RFC-1323 and RFC-2008 on > both sides > of the connection" would it then be appropriate for the > working > group recommendations to be something like "Optional > mechanisms > that provide the following functionalities { X,Y,Z } but > that do > not have detrimental impacts on other traffic need to be > developed > and standardized"? My intuitive answer is yes. But I'd like Allyn to clarify when she returns (next week). In the spirit of "recommendation" and not "dictum" I'd change the word "need" to "should" in the preceding paragraph. > If the above is appropriate, then what I would feel > comfortable > calling "Conclusions and Observations" in the > Informational RFC > would then become the Working Group Recommendations. That would be OK with me except I think that the working group recommendations should be captured in an RFC. Since the sole purpose of the working group is to create *this* RFC, it seems to me that it should be done here. If anyone with more experience in IETF working groups has a better suggestion, I'm glad to entertain it. > If we (meaning the IETF community, not just the Working > Group) > understand it, then it is OK to make it a recommendation > in > the document; If we don't fully understand it, but we > think it > merits further work/development it is OK to make it a > Working Group > recommendation. I thought we'd have a section in the RFC that captured areas that merit further work as research areas. All the knowledge that we can collect though this activity should end up in this RFC. We can caveat those areas where there is uncertainty due to immature understanding. I see two basic catagories of recommendations: those we can advocate without reservation and those that need more work. I'd like to avoid a second document containing our "dirty laundry." > How far off into the weeds am I? Not far.... Aaron > Eric - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Tue, 03 Jun 1997 19:24:34 -0400 From: Curtis Villamizar Subject: Re: comments: TCPSAT: Draft Outline In message <199706022043.AA00771@ash.isi.edu>, touch@ISI.EDU writes: > > (I have experience with asymmetric link failures on a wired point-to-point > network, and how such failures are compounded by running RIPv2 to > do dynamic path selection) - this under the http://www.isi.edu/atomic2 projec > t Real networks don't run RIP2. :-) Curtis ------------------------------ Date: Tue, 03 Jun 1997 19:22:14 -0400 From: Curtis Villamizar Subject: Re: comments: TCPSAT: Draft Outline In message <2.2.32.19970602200428.006f2918@lint.cisco.com>, Dan Shell writes: > > Note that current versions of Router protocols OSPF,RIP,IGRP,EIGRP assume > symmetrical networks. > To get asymetric networks to run you need to set up static routes. Does > anyone have experience > with BGP? Does it support asymmetic networks? If we do not have a ready > answer I can do some research > and find out. This is not true for OSPF. You can set the costs independently in either direction. This is also not true for BGP. You can play all sorts of games with policy (and get yourself in some real trouble if you don't know what you are doing, but that's another issue). Assymetric paths are already very common in the Internet, assymetric routes that is. I think you are talking about assymetric bandwidth links. > Also DVMRP and PIM assume symmetrical multicast networks and in current > version will not work > over asymmetric networks. They build a route in the reverse direction of the unicast route. Here you would have to get creative. You can force the multicast routing control traffic over the path you want it to take to create a reverse path where you want it, but you'll have to resort to rather ugly methods. Set TOS or precedence on entry and use TOS routing or precedence (ie: Cisco tag switching). > > 2.4.4 Results > > > > [I am not in tune with research specific to TCP over asymmetric > > channels on this area... Please help!] > ADSL type communication is also asymmetirical. Would you like me to do some > digging around the ADSL areas > and see what they have or have not come up with? If there is a faster back channel, then OSPF can quite easily use it. If there is not a faster back channel, things don't get too bad with the 552:40 (or greater with MTU discovery) data to ACK ratio and delayed ACK. When the data flow is in the forward direction, the slow return of ACKs should not be a problem for less than 10:1 speed difference. If transfers are in the back channel direction, then TCP congestion avoidance control back channel flow. Is there any evidence that TCP does not perform well over assymetric paths? > > 2.5 Errors > > > > 2.5.1 Discussion of the problem. > > One of the main problems is mitigating the high latency for the end to end > congestion control. > What I would like to see is some sort of mechanism for error control just > over the high-latency link. > > > > > 2.5.2 Possible mitigation techniques > Maybe a rate-based congestion control mechanism? There are much better ways to do this. There is no way to make a link rate control apply end2end since you don't have control over parts of the path. There is no concensus on any end system rate-based congestion control mechanism and there doesn't seem to be any promising work that will behave desirably with existing TCPs and over exisitng paths. I think the restriction in the charter to stick with "well understood" changes more than adequately excludes this. Curtis ------------------------------ Date: Tue, 03 Jun 1997 19:53:03 -0400 From: Curtis Villamizar Subject: Re: Proposed WG Charter In message <339354DC.CA158AAF@mailsrv1.trw.com>, Aaron Falk writes: > Eric Travis wrote: > > > > > > > If we can find changes that we feel are well understood > > and > > > meet the other constraints of the working group > > > (terrestrially benign, etc) then we can recommend them. > > I > > > think this is the catagory of small changes that > > includes > > > raising the initial slow start window to 4k, although we > > > > > haven't reached a consensus yet on that one. > > > > Because we're not sure we understand the impacts in a > > system > > with many flows? > > That is my understanding. A while back some folks raised > that issue. I'd like to see some discussion on this list to > either quantify where the uncertainty lies or put it to bed > so we can advocate it. This has come up on end2end-interest. IMHO- Raising the TCP slow start initial value would result in an unqualified disater for slower links and in particular for some older customer premise routers on very slow links and for links with a very large number of flows. Others have expressed similar opinions. At the very least this falls under the "not well understood" category and probably falls into the potential dangerous category which would mean that it cannot be responsibly advocated. Different TCP implementations for different conditions has never been considered a desirable option. TCB caching, with the caveat that TCB caches must be expired quickly (for example, initial window might be the max of one or last-cwnd >> rtt-since-active). This can be mentioned as promising but it is not a WG produced RFC and is not a standards track RFC (informational and submitted by an individual). > > > It's important to remember that what we are producing is > > an > > > Informational RFC not the protocol changes themselves. > > > Based on our recommedation, a working group could be > > created > > > to make those changes to the protocol. > > > > Are the "Recommendations" that (may) be part of the > > Informational > > RFC one in the same as the recommendations of the working > > group? > > > > In other words, if the document had a small set of > > recommendations > > that said something like "Use RFC-1323 and RFC-2008 on > > both sides > > of the connection" would it then be appropriate for the > > working > > group recommendations to be something like "Optional > > mechanisms > > that provide the following functionalities { X,Y,Z } but > > that do > > not have detrimental impacts on other traffic need to be > > developed > > and standardized"? > > My intuitive answer is yes. But I'd like Allyn to clarify > when she returns (next week). In the spirit of > "recommendation" and not "dictum" I'd change the word "need" > to "should" in the preceding paragraph. I think you mean RFC1323 and RFC2001 (not 2008). 2008 BC Y. Rekhter, T. Li, "Implications of Various Address Allocation Policies for Internet Routing", 10/14/1996. (Pages=13) (Format=.txt) 2001 PS W. Stevens, "TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms", 01/24/1997. (Pages=6) (Format=.txt) It would be nice if we could get a RFC2001 up to the status of required standard. Until then, we need to use SHOULD. > > If the above is appropriate, then what I would feel > > comfortable > > calling "Conclusions and Observations" in the > > Informational RFC > > would then become the Working Group Recommendations. > > That would be OK with me except I think that the working > group recommendations should be captured in an RFC. Since > the sole purpose of the working group is to create *this* > RFC, it seems to me that it should be done here. If anyone > with more experience in IETF working groups has a better > suggestion, I'm glad to entertain it. The document could become a BCP (best current practices) RFC. There seems to be clear consensus behind recommendations such as RFC2001 (fast retransmit and fast recovery) and RFC1323, probably RFC2018 (SACK) as well. In mentioning RFC2001, it would be appropriate to mention the work of Hoe regarding multiple drops within the fast recovery stage as something that has not entered the standards track yet but for which there is growing consensus (btw- I've yet to hear anyone object to the newreno changes). Anything else for which there is controversy should not be recommended unless you want to bog down the draft until their is enough experience and consensus. > > If we (meaning the IETF community, not just the Working > > Group) > > understand it, then it is OK to make it a recommendation > > in > > the document; If we don't fully understand it, but we > > think it > > merits further work/development it is OK to make it a > > Working Group > > recommendation. > > I thought we'd have a section in the RFC that captured areas > that merit further work as research areas. > > All the knowledge that we can collect though this activity > should end up in this RFC. We can caveat those areas where > there is uncertainty due to immature understanding. I see > two basic catagories of recommendations: those we can > advocate without reservation and those that need more work. > I'd like to avoid a second document containing our "dirty > laundry." > > > How far off into the weeds am I? > > Not far.... > > Aaron Sounds good. Curtis ------------------------------ Date: Tue, 3 Jun 1997 17:15:22 -0700 From: touch@ISI.EDU Subject: Re: Proposed WG Charter > From: Curtis Villamizar > > This has come up on end2end-interest. IMHO- Raising the TCP slow > start initial value would result in an unqualified disater for slower > links and in particular for some older customer premise routers on > very slow links and for links with a very large number of flows. > Others have expressed similar opinions. I agree - but for different reasons. I think it would increase the burstiness seen at routers shared on higher speed links (above T1), but claim (with emperical proof) that slow links are not affected, since the max possible window isn't important if the window never really opens up (for slow links, with 1-2 packets in transit, this is the case). This just underscores Curtis's point about 'not well understood'. > TCB caching, with the caveat that TCB caches must be expired quickly > (for example, initial window might be the max of one or last-cwnd >> > rtt-since-active). This can be mentioned as promising but it is not a > WG produced RFC and is not a standards track RFC (informational and > submitted by an individual). Given that I am the individual, I'd like to second this point. And, whether the product of a WG or not, it is not emperically proven (rough consensus AND running code, remember?), so it is mentioned only as a related issue. Note however, that any reason TCB sharing is not 'ready for prime time' is also a reason playing with the initial conditions of TCP isn't ready either. Neither has broad emperical evidence of doing no harm. (in fact, they are somewhat equivalent, in that 'playing with initial conditions' is the issue in either case). Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ End of tcp-over-satellite-digest V1 #25 *************************************** tcp-over-satellite-digest Saturday, June 7 1997 Volume 01 : Number 026 In this issues: Re: Proposed WG Charter TCB caching [was Re: Proposed WG Charter] Re: TCB caching [was Re: Proposed WG Charter] See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Tue, 3 Jun 1997 21:47:13 -0400 (EDT) From: Eric Travis Subject: Re: Proposed WG Charter >I think you mean RFC1323 and RFC2001 (not 2008). Actually, I meant 2018 *and* 2001 but I didn't proof read :o( ------------------------------ Date: Thu, 05 Jun 1997 17:14:17 -0700 From: Aaron Falk Subject: TCB caching [was Re: Proposed WG Charter] touch@ISI.EDU wrote: > > TCB caching, with the caveat that TCB caches must be > expired quickly > > (for example, initial window might be the max of one or > last-cwnd >> > > rtt-since-active). This can be mentioned as promising > but it is not a > > WG produced RFC and is not a standards track RFC > (informational and > > submitted by an individual). > > Given that I am the individual, I'd like to second this > point. And, > whether the product of a WG or not, it is not emperically > proven (rough > consensus AND running code, remember?), so it is mentioned > only as a > related issue. OK, I'm going to be brave and show my ignorance: what is TCB caching? I searched RFC titles and the Internic I-D database and didn't find anything. aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Thu, 5 Jun 1997 17:16:34 -0700 From: touch@ISI.EDU Subject: Re: TCB caching [was Re: Proposed WG Charter] > From afalk@mailsrv1.trw.com Thu Jun 5 17:14:09 1997 > Date: Thu, 05 Jun 1997 17:14:17 -0700 > From: Aaron Falk > Organization: TRW, Electronics Systems & Technology Division > To: touch@ISI.EDU > Cc: curtis@ans.net, travis@clark.net, tcp-over-satellite@achtung.sp.trw.com, > e.j.travis@ieee.org > Subject: TCB caching [was Re: Proposed WG Charter] > X-Priority: 3 (Normal) > References: <199706040015.AA03702@ash.isi.edu> > > touch@ISI.EDU wrote: > > > > TCB caching, with the caveat that TCB caches must be > > expired quickly > > > (for example, initial window might be the max of one or > > last-cwnd >> > > > rtt-since-active). This can be mentioned as promising > > but it is not a > > > WG produced RFC and is not a standards track RFC > > (informational and > > > submitted by an individual). > > > > Given that I am the individual, I'd like to second this > > point. And, > > whether the product of a WG or not, it is not emperically > > proven (rough > > consensus AND running code, remember?), so it is mentioned > > only as a > > related issue. > > OK, I'm going to be brave and show my ignorance: what is TCB > caching? I searched RFC titles and the Internic I-D database > and didn't find anything. RFC 2140. Joe - ---------------------------------------------------------------------- Joe Touch - touch@isi.edu http://www.isi.edu/~touch/ ISI / Project Leader, ATOMIC-2, LSAM http://www.isi.edu/atomic2/ USC / Research Assistant Prof. http://www.isi.edu/lsam/ ------------------------------ End of tcp-over-satellite-digest V1 #26 *************************************** tcp-over-satellite-digest Saturday, June 14 1997 Volume 01 : Number 027 In this issues: Re:TRW Milstar Reponse to Questions (2 of 2) Re: Repost: Questions (1 of 2) TCPSAT meeting schedule for Munich IETF Re: TCPSAT meeting schedule for Munich IETF Re: TRW Milstar Reponse to Questions (2 of 2) See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 09 Jun 1997 10:37:51 -0700 From: Aaron Falk Subject: Re:TRW Milstar Reponse to Questions (2 of 2) Here is a response from TRW regarding our architecture for the next generation of Milstar, a satellite constellation which provides extremely reliable communications to the US military. TRW is building an engineering model of the on-board processor that will do packet switching. The actual system is scheduled for launch in 2006. While the architecture design is still fluid, it is likely that the satellite based switching will not be IP routing. Only the simplest switching will be provided on board meaning that, at a minimum, packets will be routed to the correct downlink beam. One reason I think there has been little response to your request for information on architectures is that the application set that is driving the design of these systems is highly speculative. Each system architecture needs to make certain assumptions about the desired user rates and what applications will be used by the system. Satellite hardware design lags terrestrial hardware in speed and complexity since satellite hardware has the additional constraints of limited bandwidth, size, weight, and power which are driven by such concerns as spectrum, launch vehicles and radiation hardness. For this reason, terrestrial network architectures which sacrifice effieciency for flexibility may not work in a satellite environment. Therefore, each satellite system has to make certain optimizations trying to anticipate the future of the market of data service users in the 5 to 20 year horizon. The correctness of the guess can have an impact as to whether a billion dollar system works as a business. So, one can understand the reluctance of the commercial system providers to reveal information about their systems. However, I can guarentee that they are interested and want to support this group. (This can be seen by the list of subscribers to the mail list) We may need to try to find ways that these folks can help without asking them to give up their proprietary information. Now I've digressed some from what I wanted to say originally. What you are asking for in this posting is not really proprietary but more speculative. These data services (& I'm really referring to the Ka band filers, if that wasn't clear) won't be offered for several years (2-5, I'd say) and the satellite part of the architecture is the only part that will remain fixed (for around 15 years). I don't think anyone _knows_ what the future services are that far out. If they are smart, they're not designing for any particular topology that excludes other likely ones. TCP will work over any topology that you've described, therefore it's possible that a service provider could see someone wanting to use their satellite system to support that topology. So, maybe you can see a little of the bind that these system developers are in. On one hand they need to make some simplifying assumptions to build something reasonably efficient over satellite, on the other hand, no one can be too sure about what those assumptions should be. Having said that, here is some information on our proposed architecture. Eric Travis wrote: > Questions: > > A. What are the characteristics of your environment? > 1. Bandwidth (forward and reverse channel data rates > available on a per-connection basis) Per connection rates will be from 70 bps (not a typo) - 8 Mbps > 2. Delay of long-delay path (satellite, wireless, fiber, > > whatever) Geosynchronous satellite delay (one way ~125ms), much longer delays are possible due to intersatellite crosslinks > 3. Guesstimate of the end-to-end delay Depends on whats connected at the terrestrial interface. We are only designing the satellite portion. > 4. What is the closest configuration depicted below > (1->5) > or provide a new/fixed configuration Since we are doing packet switching on-board packets are statistically multiplexed into the downlink beam. The downlink is a high rate broadcast to all the users in the beam area and a particular user picks out his data from the downlink stream. This resembles case 2A where the downlink-terminal interaction resembles a bridge/filter. However, whether there are routers, intranets, or Internets connected at either end, at this time I cannot say. It is very likely that the military TCP applications will be similar to commercial applications. > 5. General comments/observations See above.-- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Mon, 09 Jun 1997 11:00:14 -0700 From: Aaron Falk Subject: Re: Repost: Questions (1 of 2) Eric Travis wrote: > o What do you envision the outcome of this effort to be? I'm not a TCP expert and I'd like to see the RFC act as a tutorial to help the satellite community understand what the inefficiencies are in running TCP over satellite, why they occur, which types of systems they apply to, and what can be done about them. Since I've become involved in this activity, I've discovered that there are a number of independant groups conducting research in this area, and I'd like to see this document act as a guide so that the right problems are solved (and existing wheels are not reinvented). There has been some confusion regarding the viability of TCP over GEO satellites. If we can release a document that helps to reduce some of that confusion by showing what works *today* we'll be helping the satellite industry (and helping the expand the Internet globally via the use of satellites). > o What do you hope to get out of the resulting document? All the points below are excellent. > For Example: > - What to look for in a TCP implementation (and > demand > from your vendor)? > > - What *isn't* broken in TCP and how to provide > network services over a long-delay path? > > - Identification of the aspects of TCP (as well as > IP, > applications, etc.) that are inherently unfair to, > > or perform suboptimally in long-delay networks? > > - What can be done in the near-term to mitigate any > unfairness without degrading non-long-delay path > performance? > > - What can't be done with respect to speed-of-light > issues? > > - Why should hosts not generally subject to long > delay > paths (such as a popular web-server) bother in > participating in implementing any identified > mitigations? > > - What looks promising for further down the road? > > o What can you sign up for based upon your above > answers? > Still thinking about this one.... aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Wed, 11 Jun 1997 08:20:52 -0700 From: Aaron Falk Subject: TCPSAT meeting schedule for Munich IETF > Date: 6/10/97 12:37 PM > From: Marcia Beaulieu > > > Aaron, > > This is to confirm one session for TCPSAT as follows: > > Monday, August 8 at 0930-1130 (opposite roamops, idr) > - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Wed, 11 Jun 1997 09:17:23 -0700 From: Aaron Falk Subject: Re: TCPSAT meeting schedule for Munich IETF The message sent to me seems to include the wrong date. The published agenda for the IETF has us scheduled for Monday, August 11. aaron > Draft Agenda of the Thirty-ninth IETF > (August 11-15, 1997) > As of June 10, 1997 > > MONDAY, August 11, 1997 > > 0800-0900 IETF Registration and Continental Breakfast > 0900-0930 Introductions > 0930-1130 Morning Sessions > > OPS roamops Roaming Operations WG > RTG idr Inter-Domain Routing WG > TSP tcpsat TCP Over Satellite BOF > > 1130-1300 Break > 1300-1500 Afternoon Sessions I > > INT ion Internetworking over NBMA WG > OPS disman Distributed Management WG > SEC tls Transport Layer Security WG > TSV ippm IP Performance Metrics BOF > USV uswg User Services WG > > 1500-1530 Break (Refreshments provided) > 1530-1730 Afternoon Sessions II > > INT ion Internetworking over NBMA WG > OPS roamops Roaming Operations WG > SEC spki Simple Public Key Infrastructure WG > > 1730-1930 Break > 1930-2200 Evening Sessions > > OPS snmpv3 SNMP Version 3 WG > RTG udlr Unidirectional Link Routing WG > SEC cat Common Authentication Technology WG > - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Wed, 11 Jun 1997 14:46:02 -0700 From: Aaron Falk Subject: Re: TRW Milstar Reponse to Questions (2 of 2) Ari Ollikainen wrote: > > > >Geosynchronous satellite delay (one way ~125ms), much > longer > >delays are possible due to intersatellite crosslinks > > > > Umm...I know you described on-board packet > switching, which > would account for your ~125ms oneway figure, but > usually the > satellite hop consists of the sum of up + down, > ~250ms. > > Was the above interpretation what you intended? Good catch. Of course, the up + down is ~250ms. We haven't found a way to beat the laws of physics,... yet ;-). aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ End of tcp-over-satellite-digest V1 #27 *************************************** tcp-over-satellite-digest Saturday, June 21 1997 Volume 01 : Number 028 In this issues: Poll: MBONE interest? Reminder of upcoming workshop Re: Poll: MBONE interest? Re: Poll: MBONE interest? re: Poll: MBONE interest? See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Wed, 18 Jun 1997 08:21:36 -0700 From: Aaron Falk Subject: Poll: MBONE interest? We have the opportunity to get our IETF working group meeting broadcast on the MBONE but there is a conflict with another group. Let's take a poll: 1. Would you watch the meeting on the MBONE? and 1. How important is that to you? aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Wed, 18 Jun 1997 11:34:21 -0400 (EDT) From: romaniak@lerc.nasa.gov (Greg Romaniak) Subject: Reminder of upcoming workshop Subject: Workshop Announcement and CFP WORKSHOP ANNOUNCEMENT A one-day workshop entitled "Transport Protocols for Long-Delay Broadband Networks" will be held at Globecom '97, in Phoenix, Arizona, USA, Monday, November 3, 1997 from 8:00 am to 5:00 pm. The location is the Hyatt Regency Hotel, Phoenix Civic Plaza. The workshop is sponsored by the IEEE Communications Society Technical Committee on Gigabit Networking and Technical Committee on Satellite and Space Communications. CALL FOR PAPERS The full performance of emerging high-speed satellite and optical networks is often unavailable to applications running on hosts connected to networks with such extremely large bandwidth-delay characterisitics. Although large bandwidth-delay transport-layer issues have been previously dealt with, and solutions to transport-layer bottlenecks have been demonstrated, these solutions have typically been environment-specific. Furthermore, both the individual and synergistic effects of operating system and protocol changes interacting with the computing and network environment are still not well understood. This workshop will examine the state of the art in deliverable network application and performance possible in modern long-delay high speed networks, identify and quantify performance limiters, and investigate possible solutions. Work toward a unified approach to optimization is of particular interest. The workshop will focus on work with networks with delay-bandwidth products greater than 50 Mb, and delays greater than those typically found in land networks (LEO satellite systems are NOT excluded). Papers on the wide range of application and protocol performance implications are of interest but papers investigating solutions that unify these approaches with those of current gigabit transport (high bw*delay) are of particular interest are especially sought. In particular, papers dealing with the following are solicited: - --a broad focus to TCP optimization, - --networked applications over wideband, high-delay networks, - --applications using native ATM, and - --applications using multiple, geographically separate hosts Experimental results observed on ATM testbeds and satellite systems are of interest, along with prototyping results of integrated systems. Focus should be on current research and implementation issues pertaining to data transmission over high bandwidth-latency paths. - ---------------------------------------------------------------------------- Workshop Chair: Douglas J. Hoder National Aeronautics & Space Administration Lewis Research Center ms 54-6 21000 Brookpark Road, Cleveland, OH 44135 216-433-3438 (voice) 216-433-8705 (fax) dhoder@lerc.nasa.gov Program Chair: Patrick Dowd Department of Defense 9800 Savage Road, MS R53 Ft. Meade, MD 20755 301-688-0347 (voice) 301-688-0588 (fax) p.dowd@ieee.org Workshop Secretary: Gregory W. Romaniak Sterling Software NASA Lewis Research Center ms 142-1 21000 Brookpark Road Cleveland, OH 44135 216-433-8289 (voice) 216-433-8000 (fax) romaniak@lerc.nasa.gov Program Committee: Andrew Campbell, Columbia University, USA Geoff Coulson, Lancaster University, UK Daniel Daly, Bellcore, USA Patrick Dowd, DOD, USA Serge Fdida, Laboratoire MASI, France Dan Glover, NASA Lewis Research Center, USA Vijay Konangi, Cleveland State University, USA Hans Kruse, Ohio University, USA Craig Partridge, BBN, USA Ira Richer, CNRI, USA Isil Sebuktekin, Bellcore, USA Saragur Srinidhi, Fore Systems, India James Sterbenz, GTE Laboratories, USA Joe Touch, ISI, USA Arun Welch, Compuserve, USA Important dates: Paper Submission: July 1, 1997* Notification of Acceptance: August 15, 1997 Camera Ready Copy: September 15, 1997 *Papers will be accepted for submission until July 15, and although every attempt will be made to properly review these submissions, there is no guarantee. Submission Policy: To speed up the reviewing process an electronic version of the paper should be submitted by e-mail to . The preferred formats for electronic submission are: Adobe Acrobat Microsoft Word Postscript If electronic version cannot be generated then four copies should be mailed to the workshop secretary. The submission of double-spaced full papers is preferred, but extended abstracts up to four pages will be considered by the program committee as well. The length of double-spaced papers should not exceed 20 pages, including figures and references. Publication Policy: All submitted papers will be reviewed by at least three members of the program committee. WWW: or - -------------------------------------------------------------------- Greg Romaniak | phone: 216-433-8289 Sterling Software | fax: 216-433-8000 NASA Lewis Research Center | 21000 Brookpark Rd, MS 142-2 | Cleveland, Ohio 44135 | email: romaniak@lerc.nasa.gov - -------------------------------------------------------------------- ------------------------------ Date: Wed, 18 Jun 1997 11:51:16 -0400 (EDT) From: Eric Travis Subject: Re: Poll: MBONE interest? Yes and plenty. On Wed, 18 Jun 1997, Aaron Falk wrote: > We have the opportunity to get our IETF working group > meeting broadcast on the MBONE but there is a conflict with > another group. > > Let's take a poll: > > 1. Would you watch the meeting on the MBONE? > > and > > 1. How important is that to you? > > aaron > > -- > Aaron Falk (310) 814-4932 > TRW, Inc > Electronics Systems & Technology Division > afalk@mailsrv1.trw.com > > ------------------------------ Date: Thu, 19 Jun 1997 08:04:03 +0800 From: Barry Raveendran Greene Subject: Re: Poll: MBONE interest? Hello Aaron, If it is on, I will be watching from Singapore. Barry Aaron Falk wrote: >We have the opportunity to get our IETF working group >meeting broadcast on the MBONE but there is a conflict with >another group. > >Let's take a poll: > > 1. Would you watch the meeting on the MBONE? > >and > > 1. How important is that to you? > >aaron > >-- >Aaron Falk (310) 814-4932 >TRW, Inc >Electronics Systems & Technology Division >afalk@mailsrv1.trw.com > > > - -- - -- - -- Barry Raveendran Greene | || || | Senior Consultant | || || | Corporate Consulting Engineering | |||| |||| | tel: +65 738-5535 ext 235 | ..:||||||:..:||||||:.. | e-mail: bgreene@cisco.com | c i s c o S y s t e m s | ------------------------------ Date: Wed, 18 Jun 1997 11:28:00 -0400 From: matthew halsey Subject: re: Poll: MBONE interest? This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-00046500 Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7bit Form: Reply Text: (6 lines follow) Question 1. I would like to but my organization's IS doesn't support, consequently the second question 1 doesn't apply. Matt Original text: (27 lines follow) From AFALK@SMTPGATE (Aaron Falk) {afalk%mailsrv1.trw.com.@intelsat1.intelsat.int}, on 18/6/97 11:21 AM: To: TCP-OVER@SMTPGATE (tcp-over-satellite) {tcp-over-satellite@achtung.sp.trw.com} We have the opportunity to get our IETF working group meeting broadcast on the MBONE but there is a conflict with another group. Let's take a poll: 1. Would you watch the meeting on the MBONE? and 1. How important is that to you? aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com Use Proportional Font: true Previous From: AFALK@SMTPGATE (Aaron Falk) {afalk%mailsrv1.trw.com.@intelsat1.intelsat.int} Previous To: TCP-OVER@SMTPGATE (tcp-over-satellite) {tcp-over-satellite@achtung.sp.trw.com} Original to: TCP-OVER@SMTPGATE (tcp-over-satellite) {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-00046500 Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: base64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjQc1CgAAAAAAQmV5b25kIFByb3ByaWV0YXJ5 IERhdGEaAAAAABEAAAAAAAQADQCsAQAAAAAAAAAAAAAAAAAAAAAAAAAAT3JpZ2luYWwgdGV4 dB4CRgoAAAAAAAAAAAAAmgEDAB4ClAECAAIAAAAZAAIAAQABALwAAAAAAAAAAgC9AGIBAAAA AAAAOP8AAAAAAACQAQAAAAAAAAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAOP8AAAAAAACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAGkAAQBpALwAAQC8AL0AAgC9APd/AgDzAPd/AgAvAfd/AgA+Afd/AgA/Afd/AgBS Afd/AgBTAfd/AgCCAfd/AgCDAfd/AgCHAfd/AgCIAfd/AgCrAfd/AgCsAfd/AgCyAfd/AgCz Afd/AgC2Afd/AgDTAfd/AgDcAfd/AgAGAvd/AgAdAvd/AgAeAvd/AgAfAvd/AAAAAAAAAAAA AAAAZAABgAEBAAMBgAQBAAYBgAcBAAkBgAoBAAwBgA0BAA8BgBABABIBgBMBABUAAAAAAAAA AAAAAABkAAGwAQFgAwEQBQHABgFwCAEgCgHQCwGADQEwDwHgEAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-00046500-- ------------------------------ End of tcp-over-satellite-digest V1 #28 *************************************** tcp-over-satellite-digest Saturday, June 28 1997 Volume 01 : Number 029 In this issues: IMSC/SCGII comments Re: IMSC/SCGII comments Vegas vs. Reno & Spoofing Re: Vegas vs. Reno & Spoofing Re: IMSC/SCGII comments Re: Vegas vs. Reno & Spoofing Re: IMSC/SCGII comments Re: Spoofing Re: IMSC/SCGII comments Reference for Vegas vs. Reno Re: Vegas vs. Reno & Spoofing Re: IMSC/SCGII comments congestion avoidance issue Paper announcements from end2end-interest... See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 23 Jun 1997 08:46:46 -0700 From: Aaron Falk Subject: IMSC/SCGII comments I received some interesting comments at a conference on Satellites and the GII last week. I presented some charts describing the problems we've identified and alerting the group to our existance. Here are some questions/comments posed by some folks looking at similar issues elsewhere in the satellite industry: People would like to know which of the various flavors of TCP (reno, tahoe, vegas) are best to use over satellites. We should include a section on the pros and cons of spoofing. Are there any IPv6/IPNG or http1.1 impacts to anything we're discussing? aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division afalk@mailsrv1.trw.com ------------------------------ Date: Mon, 23 Jun 1997 12:48:51 -0400 (EDT) From: Eric Travis Subject: Re: IMSC/SCGII comments > People would like to know which of the various flavors of > TCP (reno, tahoe, vegas) are best to use over satellites. Assuming that you can not influence the TCP deployed on both sides of the connection, are you in control of the side most likely to be sourcing the data rather than consuming it? I know I'm not going to get an answer to this but... :o) Assuming that you can not influence the TCP deployed on both sides of the connection, are you in control of the side most likely to be sourcing the data rather than consuming it? What's your network look like? Are you going to be competing for non-satellite link bandwidth with connections using a different "flavor" of TCP? > We should include a section on the pros and cons of > spoofing. Can we get a "pro" position for the proponents of spoofing? I'll gladly provide the cons; I suppose if no pro-spoofing platform shows up *soon* I'll make an attempt to cover both sides myself. > Are there any IPv6/IPNG or http1.1 impacts to anything we're > discussing? Sorry, but since I can read this two ways, I must ask: Which side of the equation are the impacts in question ? :o) If I read it as {IPv6/http1.1} impacts on satellite performance, (which is how I think the question was intended) I'd open the *discussion* by offering: - HTTP1.1 should provide a positive impact on http performance in our environment. - IPV6 addresses are painful if bit-efficiency is important to you (maybe this is irrelevant to a large segment of the community); Elements of IPV6 might make spoofing far more difficult. If I flip the impacts around I'd have to say we don't influence *them* at all at this point. Eric (who is still happily accepting *any* submissions [or ideas/concepts] for the draft document) ------------------------------ Date: Mon, 23 Jun 97 13:32:03 PDT From: Keith Scott Subject: Vegas vs. Reno & Spoofing There's an interesting paper in ICC '97 comparing Vegas and Reno. It uses a fluid flow model to show that under heavy congestion, Vegas behaves like Reno. That is, when many connections share a bottleneck link, the Vegas connections will each continue to increase their congestion windows until a loss occurs, drop back, increase until another loss, etc. Regarding Eric's request for a pro-spoofing position, I might try to claim that if I am sending over a path that contains a satellite link which spoofs at my end and has big buffers, I can increase the utilization of the link (since I'll end up sending more data and not waiting for the satellite RTT) and, from my point of view, the transmission will end sooner. Unfortunately, the receiver will have to pay the RTT cost for retransmissions unless we spoof at both ends and have a reliable link layer over the satellite? What else can be said for spoofing? The worse the spoofed link is, the more good spoofing would do, I suppose... There's the Berkeley SNOOP paper, but that's targeted more at the last hop to a wireless node than a link in the middle of a path. Are there arguments for spoofing in an asymmetric environment maybe? [I have trouble supporting spoofing with a straight face, as I am strongly opposed to breaking the end-to-end acknowledgement scheme. I'm really just trying to play devil's advocate.] --keith - ----------------------------------------------------------------------------- Keith Scott kscott@zorba.jpl.nasa.gov Jet Propulsion Laboratory 4800 Oak Grove MS 161-260 (Voice) +1.818.354.9250 Pasadena, CA 91109-8099 (FAX) +1.818.393.4643 - ----------------------------------------------------------------------------- ------------------------------ Date: Mon, 23 Jun 1997 16:49:20 -0400 From: Paul Ferguson Subject: Re: Vegas vs. Reno & Spoofing At 01:32 PM 06/23/97 PDT, Keith Scott wrote: > >There's an interesting paper in ICC '97 comparing Vegas and Reno. It uses >a fluid flow model to show that under heavy congestion, Vegas behaves like >Reno. That is, when many connections share a bottleneck link, the Vegas >connections will each continue to increase their congestion windows until a >loss occurs, drop back, increase until another loss, etc. > Pointer, please? Also, did the paper happen to mention any congestion avoidance mechanism, such as RED, and it's effect on this behavior? - - paul ------------------------------ Date: Mon, 23 Jun 1997 17:24:40 -0400 From: Curtis Villamizar Subject: Re: IMSC/SCGII comments In message <33AE9A66.7C5B3BF4@mailsrv1.trw.com>, Aaron Falk writes: > > People would like to know which of the various flavors of > TCP (reno, tahoe, vegas) are best to use over satellites. Good to best is - reno, newreno, reno+sack, newreno+sack - the name newreno is unofficial and currently you need a BSD base to get reno (I'm calling bsd44 approx equal to reno in that they both have fast retransmit and fast recovery) and then you can get patches for newreno and sack. In all case you must have rfc1323 if you want to support high speed end to end flow. BSD44 is clearly the best TCP you can get. Some of the worst are a few commercial TCPs. I don't think we want to name commercial products except ones that are better than the norm. The tcplw WG had a list of RFC1323 implementation but I don't remember if there was a URL or whether you'll have to dredge the mailing list archive. This sort of information is constantly changing so it makes little sense to enshrine a snapshot. > We should include a section on the pros and cons of > spoofing. I don't think we want to consider this a viable option at all. Application level proxies on virtually all major services are available which do not rely on protocol violations. (web proxies, mail relays, normal nntp forwarding, http proxy of ftp, etc). > Are there any IPv6/IPNG or http1.1 impacts to anything we're > discussing? Using http1.1 capable proxies that take advantage of persistent connections would provide further improvement. > aaron Curtis ------------------------------ Date: Mon, 23 Jun 1997 16:47:00 -0500 (CDT) From: Biaz Saad Subject: Re: Vegas vs. Reno & Spoofing On Mon, 23 Jun 1997, Keith Scott wrote: > > There's an interesting paper in ICC '97 comparing Vegas and Reno. It uses > a fluid flow model to show that under heavy congestion, Vegas behaves like > Reno. That is, when many connections share a bottleneck link, the Vegas > connections will each continue to increase their congestion windows until a > loss occurs, drop back, increase until another loss, etc. Please, can you give a pointer to this paper or more precise references (Authors, Title) ? Saad ------------------------------ Date: Mon, 23 Jun 1997 17:52:45 -0400 From: Curtis Villamizar Subject: Re: IMSC/SCGII comments In message , Eric Travis wr ites: > > > People would like to know which of the various flavors of > > TCP (reno, tahoe, vegas) are best to use over satellites. > > Assuming that you can not influence the TCP deployed on both sides > of the connection, are you in control of the side most likely to be > sourcing the data rather than consuming it? > > I know I'm not going to get an answer to this but... :o) > > Assuming that you can not influence the TCP deployed on both sides > of the connection, are you in control of the side most likely to be > sourcing the data rather than consuming it? > > What's your network look like? > > Are you going to be competing for non-satellite link bandwidth > with connections using a different "flavor" of TCP? The sender matters unless you have a thoroughly broken receiver that does something really stupid with delayed ACK or throws away out of sequences. Such stupid implementations do exist. A full implementation of RFC2001 is a big help at any speed. This essentially specifies TCP Reno. This rules out quite a few implementations. RFC1323 is needed for reasonable speed end to end. With a 250 msec RTT, not having RFC1323 limits you to 512KB/sec (4 Mb/s) where you might otherwise do better on long transfers. RFC1323 and SACK have to be on both ends. If you use a proxy on either end, they have to be on both proxies, but not on the client or the server. Newreno on the source is an improvement over Reno and doesn't require a change on the receiving side. If the satellite connection is used to access the Internet probably the best practical thing you can do is get a proxy on the sending side with a good TCP (BSD). Even better is a proxy on both sides for whatever service is most heavily used. Curtis ------------------------------ Date: Mon, 23 Jun 1997 18:11:33 -0400 From: hkruse1@ohiou.edu (Hans Kruse) Subject: Re: Spoofing At 16:32 6/23/97, Keith Scott wrote: >[I have trouble supporting spoofing with a straight face, as I am strongly >opposed to breaking the end-to-end acknowledgement scheme. I'm really just >trying to play devil's advocate.] I guess the above is the standard disclaimer for me too; I generally do not think a break in the transport layer flow is a good thing. My "pro" point would be that spoofing can certainly be introduced in a controlled way if the satellite is at the edge of the network, near the end-user. That is the DirecPC model as I understand it (and while DirecPC is asymmetric, this argument applies to symmetric satellite links). The assumption here is that the _information flow_ is asymmetric, towards the user at the network edge. The information sender (e.g. web server is unaware of the satellite link and "unenhanced", so the server to user flow cannot use a fast satellite link, even if the receiver requests a large window and allows SACK, because the server can't use these options). The spoofing gateway sits directly in front (from the senders perspective) of the satellite link. In this case I would argue that the spoffing gateway in fact is the proper endpoint of the interaction (maybe like a proxy server?). A new transport layer connection then sends the information on to the end user, with transport options appropriate to the satellite link. I would argue that spoofing in this case works, and scales, because the gateway implementor controls the number of users behind each physical gateway, and the end-user can be resonably expected to realize that they are connecting via a spoofing server, and therefore react at least in an informed manner to all the ways in which a spoofing gateway can fail. Does this help towards the draft? Hans Kruse, Associate Professor McClure School of Communication Systems Management, Ohio University 9 S. College Street Athens, OH 45701 614-593-4891 voice, 614-593-4889 fax, hkruse1@ohiou.edu ------------------------------ Date: Mon, 23 Jun 1997 15:34:37 -0700 (PDT) From: Kacheong Poon Subject: Re: IMSC/SCGII comments > Good to best is - reno, newreno, reno+sack, newreno+sack - the name > newreno is unofficial and currently you need a BSD base to get reno > (I'm calling bsd44 approx equal to reno in that they both have fast > retransmit and fast recovery) and then you can get patches for newreno > and sack. In all case you must have rfc1323 if you want to support > high speed end to end flow. > > BSD44 is clearly the best TCP you can get. Some of the worst are a FYI, SunOS 5.6 (in Solaris 2.6) TCP has both RFC1323 support and some features suggested by J. Hoe, as in newreno. And of course, RFC2001 features are always in. Poon. kcpoon@eng.sun.com ------------------------------ Date: Mon, 23 Jun 97 15:48:03 PDT From: Keith Scott Subject: Reference for Vegas vs. Reno To clarify: The paper comparing TCP Vegas and TCP Reno is: Omar Ait-Hellal and Eitan Altman, "Analysis of TCP Vegas and TCP Reno", ICC '97 Vol 1 pp. 495, 1997 Also of interest might be a paper by Azer Bestavros and Gitae Kim, "Implementation and Performance Evaluation of TCP Boston[:] A Fragmentation-Tolerant TCP Protocol for ATM Networks", ICC '97 Vol 2 pp. 1049, 1997 The pdf files should be available through my old ee account at: http://www.ee.ucla.edu/people/kscott/TCP I might also point out that the snoop protocol preserves the end-to-end symantics of TCP, which was not clear from my previous post. Maybe someone could provide a precise definition of "spoofing", I've been using: "The most general definition of spoofing is when somebody in the middle of a connection inserts control information that is designed to appear to have come from one of the endpoints." --keith - ----------------------------------------------------------------------------- Keith Scott kscott@zorba.jpl.nasa.gov Jet Propulsion Laboratory 4800 Oak Grove MS 161-260 (Voice) +1.818.354.9250 Pasadena, CA 91109-8099 (FAX) +1.818.393.4643 - ----------------------------------------------------------------------------- ------------------------------ Date: Mon, 23 Jun 1997 17:27:57 -0700 (PDT) From: hugo jacques Subject: Re: Vegas vs. Reno & Spoofing Hi, A more complete version of the paper that was presented at the ICC'97 in Montreal can be found at: http://zenon.inria.fr:8003/mistral/personnel/Omar.Ait-Hellal/pub_list.html Hugo Jacques M.A.Sc. Student, Electrical Engineering The University of British Columbia e-mail: hugoj@ee.ubc.ca Vancouver, B.C. On Mon, 23 Jun 1997, Biaz Saad wrote: > On Mon, 23 Jun 1997, Keith Scott wrote: > > > > > There's an interesting paper in ICC '97 comparing Vegas and Reno. It uses > > a fluid flow model to show that under heavy congestion, Vegas behaves like > > Reno. That is, when many connections share a bottleneck link, the Vegas > > connections will each continue to increase their congestion windows until a > > loss occurs, drop back, increase until another loss, etc. > > > Please, can you give a pointer to this paper or more precise > references (Authors, Title) ? > > > Saad > ------------------------------ Date: Mon, 23 Jun 1997 21:54:25 -0400 (EDT) From: rohit goyal Subject: Re: IMSC/SCGII comments > > People would like to know which of the various flavors of > > TCP (reno, tahoe, vegas) are best to use over satellites. > > Good to best is - reno, newreno, reno+sack, newreno+sack - the name > Curtis > I would argue a different order especially for large delay-bandwidth cases with congestion losses. From poor performance, to Good performance: Reno, Vanilla, NewReno, SACK with NewReno or Reno. Vanilla TCP : Defined as TCP with only slow start and congestion avoidance but no fast retransmit and recovery. In small delay networks, Vanilla TCP results in poor performance because for every segment loss it waits for a timeout. This could be a very long time because of the coarse granularity timer granularity (500ms). However, when the delay is large (of the order of the timer granularity), then the effect of timer granularity is mitigated. Reno TCP : Add Fast Retransmit and Recovery to Vanilla TCP. Reno is bad for multiple segment losses, especially for large delay-bandwidth networks. When multiple segments are lost back to back, Reno could incur a timeout at a very low window, resulting in a congestion avoidance phase at a very small window. The linear increase at the low window for a large delay-bandwidth network results in very low utilization. In fact, for large delay-bandwidth networks which have even a moderate degree of congestion (so that multiple packets are dropped close to one another), Vanilla TCP performs better than Reno. NewReno (from LBL): When multiple segments are lost from a window, NewReno retransmits one segment every RTT until all lost segments are recovered. Janey Hoe's version: For a single block loss within a window, both NewReno and Janey Hoe's Reno will recover one segment per RTT. When loss is scattered through the window, Hoe's reno could recover a little faster than NewReno because it does slow start and multiple segments could be retransmitted in one RTT. However, I will stick to NewReno for this discussion. Newreno improves performance over Reno because of its ability to recover from multiple losses. But it is debatable if NewReno alone is better than Vanilla because NewReno can only retransmit a single lost segment in one RTT. So if many multiple segments are lost, trying to recover from them (with a large RTT) might take longer than waiting for the timeout. The addition of SACK buys a lot for obvious reasons. I'm not sure what the difference would be between Reno+SACK and NewReno+SACK. So, for long delay-bandwidth networks, I would argue for the following order from poor performance to good performance. Reno, Vanilla, NewReno, SACK with NewReno or Reno. However, to ensure good performance for both multiple packet losses as well as isolated packet losses, I would recommend at least NewReno if not SACK also. Certain caveats with SACK must also be pointed out. There is certainly the processing and the space overhead (both and the sender and the receiver). (I'm trying to get some results on how big the scoreboards and stacks grow, but my results are only going to be based on simulation). Also, if a retransmitted packet is dropped with SACK, then the SACK sender assumes receiver reneging and times out to start over again from snd_una. - -rohit ------------------------------ Date: Mon, 23 Jun 1997 22:02:37 -0400 (EDT) From: rohit goyal Subject: congestion avoidance issue This is an implementation issue that I ran into some time ago. Eric has this for possible inclusion in the draft. There may be some other who have encountered similar issues. - -rohit > During the congestion avoidance phase, CWND is incremented by 1 segment > every RTT. Most TCP implementations follow the recommendations in > Jacobson's paper, and increment by CWND by 1/CWND segments for each ACK > received during the congestion avoidance. Since CWND is maintained in bytes, > this increment translates to an increment of MSS*MSS/CWND bytes on the > receipt of each new ACK. All operations are done on integers, and this > expression avoids the need for floating point calculations. However, > in the case of large delay-bandwidth paths where the window scale factor > is used, MSS*MSS may be less than CWND. For example, with > MSS = 512 bytes, > MSS*MSS = 262144, and when CWND is larger than this value, > the expression > MSS*MSS/CWND yields zero. As a result, CWND is never increases during > the congestion avoidance phase. > > There are several solutions to this problem. The most intuitive > is to use floating point calculations. This increases the processing > overhead of the TCP layer and is thus undesirable. A second option is > to not increment CWND for each ACK, but to wait for N ACKs such that > N*MSS*MSS > CWND and then increment CWND by N*MSS*MSS/CWND. Let's call > this the ACK counting option. > > Another option would be to increase MSS to a larger value so that > MSS*MSS would be larger than CWND at all times. The MSS size > of the connection is limited by the smallest MTU of the connection. > Most TCPs are expected to use Path-MTU discovery to find out > the largest possible MSS that can be used. This value of MSS may or > may not be sufficient to ensure the correct functioning of congestion > avoidance without some sort of ACK counting or other fix . > > The code during the congestion avoidance phase with ACK counting looks like: > > { > INCREMENT = MSS; > > > IF (CWND > SSTHRESH) { > N++; > IF (N * INCREMENT * INCREMENT > CWND) { > INCREMENT = N * INCREMENT * INCREMENT / CWND ; > N = 0; > } > } ELSE { > N = 0; > Do exponential increase; > } > } > > ---------------- > > > > - -- Rohit Goyal, CIS PhD Student (W) (614)-688-4482 395 Dreese Lab, 2015 Neil Ave (H) (614)-488-5785 The Ohio State University Fax: (614)-292-2911 Columbus OH 43210 goyal@cis.ohio-state.edu ------------------------------ Date: Tue, 24 Jun 97 14:47:32 PDT From: Keith Scott Subject: Paper announcements from end2end-interest... For those who don't check end2end-interest regularly... I've appended two postings. --keith >To: end2end-interest@ISI.EDU >Subject: paper on analyzing packet traces of TCP behavior available >Date: Tue, 24 Jun 1997 12:03:09 PDT >From: Vern Paxson >Sender: owner-end2end-interest@ISI.EDU >Precedence: bulk > >[My apologies to those of you who will receive multiple copies of this >announcement from other mailing lists ...] > >The paper "Automated Packet Trace Analysis of TCP Implementations", >to appear in SIGCOMM '97, is now available from: > > ftp://ftp.ee.lbl.gov/papers/vp-tcpanaly-sigcomm97.ps.Z > >I've appended the abstract. > > Vern > > >Automated Packet Trace Analysis of TCP Implementations > >Vern Paxson >Network Research Group >Lawrence Berkeley National Laboratory >vern@ee.lbl.gov > >We describe "tcpanaly", a tool for automatically analyzing a TCP >implementation's behavior by inspecting packet traces of the TCP's >activity. Doing so requires surmounting a number of hurdles, including >detecting packet filter measurement errors, coping with ambiguities due to >the distance between the measurement point and the TCP, and accommodating a >surprisingly large range of behavior among different TCP implementations. >We discuss why our efforts to develop a fully general tool failed, and >detail a number of significant differences among 8 major TCP implementations, >some of which, if ubiquitous, would devastate Internet performance. The >most problematic TCPs were all independently written, suggesting that >correct TCP implementation is fraught with difficulty. Consequently, it >behooves the Internet community to develop testing programs and reference >implementations. > > To: end2end-interest@ISI.EDU > Subject: paper on end-to-end Internet packet dynamics available > Date: Tue, 24 Jun 1997 11:59:41 PDT > From: Vern Paxson > Sender: owner-end2end-interest@ISI.EDU > Precedence: bulk > > [My apologies to those of you who will receive multiple copies of this > announcement from other mailing lists ...] > > The paper "End-to-End Internet Packet Dynamics", to appear in SIGCOMM '97, > is now available from: > > ftp://ftp.ee.lbl.gov/papers/vp-pkt-dyn-sigcomm97.ps.Z > > I've appended the abstract. > > Vern > > > End-to-End Internet Packet Dynamics > > Vern Paxson > Network Research Group > Lawrence Berkeley National Laboratory > vern@ee.lbl.gov > > We discuss findings from a large-scale study of Internet packet dynamics > conducted by tracing 20,000 TCP bulk transfers between 35 Internet sites. > Because we traced each 100 Kbyte transfer at both the sender and the > receiver, the measurements allow us to distinguish between the end-to-end > behaviors due to the different directions of the Internet paths, which > often exhibit asymmetries. We characterize the prevalence of unusual > network events such as out-of-order delivery and packet corruption; discuss > a robust receiver-based algorithm for estimating ``bottleneck bandwidth'' > that addresses deficiencies discovered in techniques based on ``packet > pair''; investigate patterns of packet loss, finding that loss events are > not well-modeled as independent and, furthermore, that the distribution of > the duration of loss events exhibits infinite variance; and analyze > variations in packet transit delays as indicators of congestion periods, > finding that congestion periods also span a wide range of time scales. - ----------------------------------------------------------------------------- Keith Scott kscott@zorba.jpl.nasa.gov Jet Propulsion Laboratory 4800 Oak Grove MS 161-260 (Voice) +1.818.354.9250 Pasadena, CA 91109-8099 (FAX) +1.818.393.4643 - ----------------------------------------------------------------------------- ------------------------------ End of tcp-over-satellite-digest V1 #29 *************************************** tcp-over-satellite-digest Saturday, July 5 1997 Volume 01 : Number 030 In this issues: Introduction Re: Introduction re: introduction For Discussion Grassroots Inputs Re: Grassroots Inputs SAT TCP versus BER/ Number Session Warn: long post Re: Grassroots Inputs Re: Grassroots Inputs Re: SAT TCP versus BER/ Number Session See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Wed, 2 Jul 1997 16:53:00 -0400 (EDT) From: Deborah Bakin Subject: Introduction Last week at SCGII Aaron Falk suggested we present an overview of our DARPA funded work on TCP/UDP enhancements for wireless environments to the TCPSAT group. We are currently implementing these modules (called protocol boosters) in Linux (FreeBSD implementations exist as well) and have been experimenting with them over simulated satellite links. Initial tests show excellent results when using boosters with both TCP and UDP (any solution for satellites must include RTP and UDP as well as TCP because multicast is a natural use for satellites). We are actively seeking to test these prototypes over live satellite networks and are seeking partners. Protocol boosters are robust protocol adaptors designed to dynamically improve (boost) the performance of protocols in heterogeneous environments. One environment they are particularly suited for is for networks with satellite links. Protocol boosters are different from other adaptation techniques because they are parasitic and transparent to the protocol being boosted. Boosters: - Keep a protocol's message semantics, unlike protocol termination (e.g., converting from TCP to UDP). - Keep a protocol's message syntax, unlike protocol conversion (e.g., TCP/IP header compression). - Can use information from other protocols, unlike independent link layer protocols (e.g., LAPB). - Can be applied selectively (anywhere in the network or end systems). - Can be used with IP-layer authentication and security methods. More details can be found at: http://gump.bellcore.com:8000/boosters We would appreciate any feedback. Deborah Bakin, Bill Marcus, Tony McAuley, Tom Raleigh. Bellcore ------------------------------ Date: Wed, 2 Jul 1997 19:11:54 -0400 (EDT) From: Eric Travis Subject: Re: Introduction Deborah, Since I'm currently thrashing away on the initial draft of this group's document and I happen to be bogged down in the area regarding "spoofing", I'll take you up on the request for feedback :o) [ To everyone else, my apologies. The draft is coming; For some reason I was expecting slightly more editing and not as much original text generation on my part, but I'm working on getting the wheels back on and things the effort back on schedule ] >Last week at SCGII Aaron Falk suggested we present an overview of our >DARPA funded work on TCP/UDP enhancements for wireless environments to >the TCPSAT group. Actually, I've really got questions mostly related to TCP and boosters; If the answers are in one of the papers asked about below, just point me where appropriate :o) Besides providing all of us with more insight into the concept of protocol boosters, I hope these questions help to foster wider discussion (whine, whine, whine). o Are there (will there be) soft-copies available for the papers identified on your Web site? o This might be covered in one of the papers; Why, in particular, do you feel boosters are particularly suited for networks with satellite links. Does this still hold true if the satellite links are not lossy? o Have any other "boosters" been designed and tested for environmental effects other than channel errors? o How are protocol boosters different than other things that would fall into the "spoofing" bucket, and why are they better than using full-fledged application proxies to separate significantly different subnetwork environments? Aren't most (if not all) spoofing techniques parasitic (and use information from other protocols)? o How do you foresee protocol boosters *transparently* mitigating the satellite environments problems such as long-delay paths and asymmetry? How about non-transparently but without violating TCP's message semantics or syntax? Does the definition of transparency you are using also include not having a negative impact on other flows in the network (are you fair, or does the parasitic nature include eating the bandwidth of other flows)? o How do boosters interact with TCP's congestion avoidance and control mechanisms? For example, say I've got a path that is experiencing congestion, what is to stop a FEC booster pair (running somewhere in the path) from doing it's best to correct for the *congestion* based losses and preventing a traffic source from reducing it's transmission rate? Can boosters distinguish losses due to corruption (erasure seems tougher) and congestion? While I can conceptualize how boosters can improve the performance of a particular connection, do they "work and play well with others"? o How much a priori information about a network topology (or a particular connections path) must be available prior to using boosters? o You note that boosters can "be used with IP-layer authentication and security methods", but if the transport header and payload were encrypted (say, I trust you to route my packets, but I don't trust you to look at the contents), can boosters still boost performance? You also raise an interesting topic in your message; I really had not considered including UDP or RTP within the scope of this draft, mainly because no one has raised concerns about using UDP or RTP. Actually, I believe that there are concerns about people using UDP to avoid TCP's congestion avoidance and control mechanisms. :o) You state that *any* solution for the satellite environment *must* include UDP and RTP; What special problems do UDP and RTP have that need to be addressed? Thanks! Eric ------------------------------ Date: Wed, 2 Jul 1997 16:50:27 -0700 (PDT) From: Thomas Bohn Subject: re: introduction This message intended for: Deborah Bakin, Bill Marcus, Tony McAuley, Tom Raleigh. Bellcore Your post to tcp-over-satellite (titled "Introduction") is of *extreme* interest to me. This past summer (July of 1996) I installed a TCP/IP Internet server in Nome, Alaska, and it runs over a frame relay satellite link through ATT Alascom, touching down in Anchorage before being routed to Seattle via optic fiber. I 'do' systems maintenance for this server site in Nome, which currently runs SunOS 4.X (latest releases, binaries and patches) and am in the process of re-compiling a recent release of Linux source code on a computer which I have built at home, for sysadmin tasks after hours. I would be willing to run tests over these systems, and I have full root access, so compilation of any special binary code onboard would be no problem. I am currently planning an upgrade migration to Solaris 2.51 (and 2.6 when it is released in August) so this system's configuration will change from BSD-style to SVR4 style UNIX very shortly. I can, however, continue to maintain the Linux site on my own. Please let me know if I can be of assistance. My primary job responsibilities are currently linked into a regional WAN project which will, of necessity, be implemented over satellite links. The northwestern Alaskan region is remote, and will continue to be so for decades to come. (Nome, my place of residence, is roughly 500 air miles from the nearest connecting road system.) We have no ground links to the data network. I am interested in learning more about your implementations so that our region may stay abreast of current development at the grassroots level. I will belabor this a bit further... We are currently 'served' by two long distance carriers -- AT&T Alscom and General Communications Incorporated (GCI). Both GCI and ATT are in process of troubleshooting bandwidth on demand multiple access systems utilizing CDMA strategies for ground-to-space links -- top speed on current system configurations through both demand assigned systems is 14.4. Residents within our region would like to see improved speeds. It is, in short, in my best interests, and in my region's best intersts, to be involved in your project. Please feel free to contact me via telephone. Those interested in learning specifics regarding telecom and TCP/IP networking in northwestern Alaska may be interested in a paper I am currently producing, "TCP/IP in the Bering Strait Region," which identifies and describes a frame relay/thin POP (router/integrated modem rack) topology for a satellite based WAN designed to cover roughly 22,000 square miles. The paper is scant on technical detail regarding our TCP/IP implementation (details available from Greg Moore, Sr. Eng. at ATT Alascom, contact info in paper) but describes our networking context at length. I would be happy to forward an e-copy to anyone interested when the paper is finished. And please excuse this lengthy post... Thomas Bohn -- Systems Specialist Kawerak, Inc: 907.443.4392 -- Direct Desk Line 907.443.3807 -- FAX tbohn@nome.net ________________________________________________________________________ Kawerak, Inc. is an Alaska Native owned-and operated non-profit organization serving the Bering Strait Region of northwestern Alaska. ------------------------------ Date: Wed, 2 Jul 1997 23:15:19 -0400 (EDT) From: Eric Travis Subject: For Discussion Thomas Bohn's message is a reality check for the list. If we were to take a step back from things and remind ourselves what the main *applications* (different context) of TCP over satellite are (and will likely be), I'd offer that the major users (in rough order of magnitude): o Regional internet services/backbone connectivity Connectivity to the larger "Internet" backbone via satellite links, "regional" service may or may include terrestrial connectivity, but that might be via wireless/cellular/land-mobile radio technology. o Corporate Intranets/Extranets/Virtual LANs One or more isolated subnetworks connected to the larger backbone via satellite link o Large "Direct Broadcast" type commercial services Asymmetric connectivity/architecture o Military Uses - more or less the union of the above o Space Sciences/Exploration & Terrestrial Sciences Connectivity between spacecraft to ground (or other spacecraft), remote (temporary?) connectivity to isolated areas. I'll accept (grudgingly) that the large commercial *providers* are remaining quiet for "business reasons"; I'll be bold here and claim that they are not as important as first two communities on the list. I believe that these are going the large population base, especially as the world outside of the US "Lower 48" continues to ramp-up Internet connectivity. Running cable or fiber isn't necessarily the best alternative to provide connectivity everywhere. Much of the talk in Internet research circles is toward increasingly high-speed pipes. Let's not forget about the bandwidth underprivledged, that will probably include a large portion of this community. I think folks will be surprised at the bandwidth available in some cases... Even big pipes satellite pipes (as they come on line) look small when you are aggregating enough traffic through them. The trend is always going to be toward larger/faster pipes, but that doesn't mean the pipes are cheap, easy to provide or readily available as required. Topology and bandwidth are important factors, yet there is little discussion on either. As an example, if the satellite link tends to be the bottleneck, and it approaches chronic congestion during certain periods of the day, then some mitigations *might* do more harm than good; These conditions need to be brought into the larger discussion. Comments (particularly from folks on the front lines)? [Silence != Concensus] ------------------------------ Date: Wed, 2 Jul 1997 23:22:56 -0400 (EDT) From: Eric Travis Subject: Grassroots Inputs This seems like a good time to solicit inputs from the readership of the mailing list once again. We've been getting *terrific* input from folks on the technical side of things - but not necessarily from folks on the "front lines". I consider this to be unfortunate. From the list of subscribers to the mailing list, I get the impression that there is a vast amount of untapped, but highly relevant input out there. What I think would be useful is input from the grass-roots level of things. Folks who provide regional service via satellite, connect corporate nets via satellite or folks who get their service that way. o What are the perceived performance shortcomings about TCP over satellite? Slow Web access or something more troubling :o) o What services are being provided, o Rough topologies, pipe sizes, etc. current and planned o etc. Being technically savvy about TCP internals is not a requirement, we've got lots of people covering that :o) If folks feel uncomfortable posting to the list, feel free to contact me *privately* and I will incorporate the input (anonymously if desired); Similarly, if there are commercial entities that feel they are constrained from publicly contributing, the same service is available to you. I'll do anything short of signing non-disclosure agreements. I'd rather not constrain my future ability to get a paycheck and frankly, if you need a non-disclosure agreement, the input just isn't correct for this forum. Any takers? Eric ------------------------------ Date: Wed, 02 Jul 1997 23:28:33 -0700 From: "Paul Porto, Jr." Subject: Re: Grassroots Inputs Eric, Since you asked, here's some news from the front. Granted, I am fairly new to the world of fighting TCP to squeeze every ounce of efficiency out of a satellite link; my past ten years have been spent providing mainly voice trunks and low speed data links over satellite. In recent years, however, more and more customers are wanting to uproot workstations from the LANs where they have worked so well, deploy them half way around the world, request a satellite shot, and expect everything to work as well as it did back at home. Specifically, my customers attempt to ftp a file at a time between Unix workstations via my point-point T-1 Ku satcom link. Well, it didn't take long for us to discover the inefficiencies; it did take a while to discover that there is research in progress, and even some stop-gap tweaks and application level solutions available. I stand at the point of wondering whether to recommend that customers don band-aids (tweaks, upgrade o/s), or just wait for the cure (new standards). Since I hate to stand around and watch customers bleed, I'm giving the band-aids a good look. Here are some things I've experienced. I am aware that my particular experiences are probably not the norm, but, again, you asked. 1. Customers with small files don't really notice a lame throughput efficiency of 25-30%, but the shameful performance leaves the good provider with the feeling that something must be done. 2. Some customers would rather put up with low throughputs than tweak their o/s and rebuild the kernel -- after all, it's only a temporary job and the computer needs to work when it gets back home to the LAN. They're just happy it works. That doesn't make me happy though; I want efficiency! 3. Some customers have a land line T-1 as primary and don't want to modify any settings just to be assured of efficiency should they need the alt-route satcom link. 4. There is the question of how using different operating systems at each end of the link affects TCP -- say Irix and Solaris. Yes, TCP is compatible, but does it run in some sort of compatibility mode when it meets up with a foreigner? Sort of like how we Americans feel the need to revert to pigeon English when we meet a foreigner with a tenuous grasp of our language. Then we wonder why they speak that way, but they mirror us! Sorry, I digress. 5. Though router settings (buffering) can clearly play a part, it's difficult to find recommendations or optimal settings for satcom router links in the router documentation. 6. This is the biggie for me. The trail of bread crumbs left by the researchers is a little thin; leaves me hungry; makes me wonder if I should just sit down and wait to be rescued. Can't they plant a flag from time to time and say, "This is where we are today. If anyone is experiencing this problem, here are the steps to take to minimize it." The steps might range from specific tweaks to the application (say, ftp window size) to alternate applications (XFTP), to O/S upgrades (Solaris 2.6, when available). I am aware of a pretty good Web page that attempt this, but it needs work. Geez, let me stop here or people will start deleting any message with my name on the top! I'll try to throw out some opinions from the grass roots level from time to time. But let me finish by saying that I don't accuse anyone of not doing a great job in researching and documenting this problem. I am keenly aware that it's much more likely that I simply haven't done all my homework yet. But that's where I am. Keep up the good work! Paul ------------------------------ Date: Fri, 04 Jul 1997 06:05:45 +1000 From: Harry Smith Subject: SAT TCP versus BER/ Number Session I am assisting on the development of a military communications system. In this system, we have to do data transfer over satellite links with as little delay as possible. As a result, I have been following this discussion group. However, I took a different approach to solving some of the problems addressed in this group. After the request for front line input, I decided to ask for comments my approach to solving the SAT TCP problem. I have used a two-fold method to SAT TCP which does not appear to require any modification to TCP but depends on the facts that I have a closed network and some additional hardware. 1. I did simple maths and found that at present bit error rate ( 10 e-6 or 10e-7), the retransmissions would kill the performance. So I added Commercial Reed Solomon error correction to move the BER to better than 10e-10. It took about 10% of the bandwidth. There are some commercial RS boxes at about 20K each. 2. I arranged the data so that I had multiple TCP session between endpoint ( a type of session interleaving ). Whist this would not help the composite TCP performance, on the average the performance of individual session would be improved. OK, what I have failed to understand? Harry Smith ----------------------------------------- Harry Smith hsmith@netlink.com.au 3/847 Burwood Rd hsmith1@netcom.com Hawthorn, VIC 3122 011 613 9813 2590(H) 011 614 1223 0012(M) ------------------------------ Date: Thu, 3 Jul 1997 12:18:38 -0700 (PDT) From: Thomas Bohn Subject: Warn: long post Re: Grassroots Inputs Eric: This post comes in response to both your personal note and your post to the tcp-over-satellite listserv: you have requested 'grassroots' input on internetworking via satellite, including performance reviews, technical issues, descriptions of services offered and sketches of network topologies. You have indicated that my particular network may make an ideal case study. I believe you're right. Allow me to briefly describe this network's context, for the benefit of listserv readers, before touching upon the points you have mentioned. I would then leave myself open for further questions from yourself, and from any of the list's subscribers. I provide base level sysadmin support for a network located in northwestern Alaska; geographically, this area is known as the Bering Strait Region. The region is composed of the bulk of the Seward Peninsula and the northern reaches of the Yukon River delta and also includes Diomede, St. Lawrence and King Islands. These land masses cover an area of approximately 22,000 square miles and the climate varies from maritime Arctic to sub-Arctic. Roughly 10,000 persons now make their homes in the region, scattered throughout twenty static communities, isolated from one another. There are no connecting roads, and transport is carried on via 'air taxi' (Cessna 207s and Piper Navajos!!!) In addition, there are no land-based data network lines connecting these communities. The largest ('hub') community within the region is Nome, with approximately 3800 people. The vast majority of the region's residents are Alaska Natives -- of the 8,991 persons counted in the 1990 census of the region, 6,988 were recorded as Native American. The region is approximately 500 air miles from the nearest connecting road system and is completely isolated from the land-based data network. All communications connectivity (telephone, broadcast/cable television services, I-net, etc.) is provided via satellite. Given this description, allow me to provide information on a TCP/IP internetwork server (ISP-style) installed here by myself roughly one year ago this month. Below find: 1.) description of our network topology and server implementation 2.) services currently offered via this network 3.) brief description of bottleneck issues 4.) brief description of technical issues/current needs, projects 1. Network Topology The Bering Strait Region is served by ATT Alascom's Aurora II satellite. Our servers (Sun Microsystems machines running SunOS 4.X) make their connections to this satellite via frame relay assembled on a Cisco 2514 FRAD through a fractional T-1 (currently at 256CIR/512burst) and we now provide roughly 30 dial-up points and service numerous direct-connect ethernet points (several of which utilize NCR WaveLAN wireless systems for 'metro' area connects, running in the 2.x Ghz band). We connect to IPX/SPX systems, as well as TCP/IP implementations on NT4.0 systems. The FRAD connects to an Adtran CSU/DSU, running to the ATT Alascom earth station to uplink. Downlink is in Anchorage, roughly 700 air miles from Nome, at ATT's Eagle River NOC. Signal is then carried via optic fiber to Seattle, where connect is made to the backbone via T3 taps serviced by Sprintlink. We currently have no WAN POPs within the region, though such systems are under discussion, and will likely be implemented via frame circuits connected to similar FRADs with 'thin servers' (basically, an integrated modem rack/terminal server similar to USR's Netserver8). We do service several distant dial-ups, including one from St. Lawrence Island, and these dial-ups experience significant degradations in signal due to satellite hops, with max speeds of 9600 baud, but averaging 1200 baud. More on this below (3.) 2. Current Service Offerings We currently offer the following network services... - - UNIX Shell, Slip, CSlip and PPP connectivity via Xylogics TSU - - Web (NCSA's httpd 1.5.2) including hosting services - - CGI-based web BBS services (http://www.nome.net -- click WebBBS) - - FTP services, currently limited to access subscribers - - News (INND runs through provider gateway in Seattle, see 3., below) - - Majordomo-based listserv services - - SMTP gateway services (sendmail) 3. Bottleneck Issues Several bottlenecks have appeared within this topology. Most significant are listed below... - - Distance Connects: residents living within villages throughout the region currently experience signal degradation on long distance dial-ups. Satellite transmissions must currently make two 'hops' at the data link layer for such connections (residents of Nome do not experience this problem...they dial-up via local loops and then make a single hop to Anchorage). This double hop situation leads to significant latency, and it is nearly impossible to gain links beyond 1200 baud. The single exception to this rule comes out of Unalakleet, a village of nearly 900 residents on the northern skirts of the Yukon-Kuskokwim River Delta, nearly 120 miles southeast of Nome, which is serviced by General Communications Inc. (GCI), a LDC providing limited CDMA links to ATT transponders, which are currently capable of 14400 baud. Plans are in the works for a region-wide rollout of GCI's CDMA system which they have coded 'Demand Assigned Multiple Access' (or, DAMA) through GCI's network of small earthstations. GCI's network is brand spanking new. ATT Alascom is also working on rollout of a 'DAMA' system, currently on a five year rollout schedule. - - INND services: our usenet links are provided by Wolfe.Net, a Seattle-based ISP with broadband taps into Sprint's backbone. We plan the implementation of INND services *in Nome* about one year from now, when we believe demand for services will justify investment in the broader bands necessary for newsfeed. In the meantime, end users must wait for 'sucking news feed' downloads from Wolfe's INND server in Seattle, and significant latency also comes into play here. While service is fairly reasonable, large news downloads from active Usenet sites can take a maddening amount of time. - - Web: WWW downloads are a problem, with ever-increasing popularity leading to long waits during busy hours (say, 20-30 concurrent dial-ups mixed with heavy direct connect ethernet broadcasts). Some sites (especially those laden with heavy multimedia adornments) take *forever* to download. Streaming protocols work, but are fuzzy. A RealAudio stream, for instance, will work at 28.8 for a Nome-based user, are impossible for a village-based dial-up user. We are considering implementation of Squid, or a similar web-caching strategy, but are reluctant to do so at this point in time. Uploads from our httpd server are fairly dismal. We are currently looking at implementation of a mirror site in Seattle. - - Non-Technical/Demographic: while demand for services is high within the region, the geographic situation, combined with the region's low population density, has created a 'development deadlock.' Capital for further technical development is currently 'frozen.' Both GCI and ATT have assessed the region's market demand and, in the case of ATT, have taken a wait-and-see approach to further infrastructure investment. GCI has invested millions in construction of a rural earth station network, but these remain inoperable due to LEC problems and software difficulties within GCI's developing DAMA system. - - Non-Technical/Demographic: lack of integrated technical curricula within the region's existing educational system has produced a situation where end-users possess poor to non-existent computer skills. Compounding difficulties is the fact that the bulk of the region's populace live within 'two worlds' -- one best characterized as a colonial techno-culture, and one typified by traditional Alaska Native cultural systems. Regional residents have taken a pro-active step toward integration of these 'two worlds,' though context-specific issues will likely continue to shape the development of networking systems here for decades to come. The end result is a 'bottleneck' at the non-GUI technical interface. 4. Brief Description of Tech Needs/Current Projects: nome.net (the above-described ISP) is currently looking to expand services to the villages throughout the region. This initiative comes as a response to grassroots demand...In addition, A distance delivery consortium known as the Bering Strait Regional Communications Consortium (or BSRCC) has formed in Nome, and consists of five regional service providers (healthcare, k-12 and post-secondary education, social services and National Guard military units.) The BSRCC has researched frame WAN topologies and is interested in GCI's and ATT's developing 'DAMA' systems, but is also researching alternative spread spectrum technologies. The most dire technical need at this point is bare bones connectivity (e-mail and FTP at rates beyond 1200 baud). We believe frame is the most efficent, least expensive option, at this point. Our need comes in the form of design and implementation of a regionally-based NOC capable of administering thin POPs in the villages throughout this region. Regional residents are cautious when it comes to the LDC's 'DAMA' systems which are centered on a per-usage design principle which would make connectivity prohibitively expensive, if fast. In addition, these 'DAMA' systems are designed around NOCs located in the Anchorage area -- which means revenue flows, technical knowledge, power brokerages and skills transfer would all center on Anchorage, rather than in our region...a situation which would only further our isolation, if in economic, social and political terms. Obviously we have gone beyond the realm of the purely technical. There you have it. Please excuse the lengthy post. I would be willing to answer any follow-up questions. Proprietary information provided in this post is not for re-broadcast and use is presumed for research purposes only. As I have told Eric: I am currently working on a document which describes the Bering Strait Region's situaiton at length, titled "TCP/IP in the Bering Strait Region." E-copies will be freely available to interested parties upon the document's completion (2-3 weeks from today) and Eric has offered provision of an FTP download site (thanks Eric). Quyana. ("Thanks," in Inupiaq...) Thomas Bohn -- Systems Specialist Kawerak, Inc: 907.443.4392 -- Direct Desk Line 907.443.3807 -- FAX tbohn@nome.net ___________________________________________________________________________ Kawerak Inc. is an Alaska Native owned and operated non-profit corporation based in Nome, Alaska, and provides social services to the Bering Strait Region. ------------------------------ Date: Thu, 03 Jul 1997 18:12:03 -0400 From: Curtis Villamizar Subject: Re: Grassroots Inputs Paul, Some sort of off the cuff answers.. In message <33BB4690.97C38751@thegrid.net>, "Paul Porto, Jr." writes: > > 1. Customers with small files don't really notice a lame throughput > efficiency of 25-30%, but the shameful performance leaves the good > provider with the feeling that something must be done. If making tiny files move faster is not a customer requirement my advise is try not to imagine requirements that don't really exist. > 2. Some customers would rather put up with low throughputs than > tweak their o/s and rebuild the kernel -- after all, it's only a > temporary job and the computer needs to work when it gets back home to > the LAN. They're just happy it works. That doesn't make me happy > though; I want efficiency! The choices are 1) use a proxy (for example, ftp or http proxies, news feed through a relay, mail relay, etc), 2) upgrade the OS (or in BSD world apply readily available patches on the sending end), 3) both 1 and 2, 4) live with it as is (the stated customer preference). > 3. Some customers have a land line T-1 as primary and don't want to > modify any settings just to be assured of efficiency should they need > the alt-route satcom link. No need to make changes. Set the default window in the kernel large and leave it that way for terrestrial and satellite. The fact that this works (TCP congestion avoidance) is one reason TCP has gained such popularity. > 4. There is the question of how using different operating systems at > each end of the link affects TCP -- say Irix and Solaris. Yes, TCP is > compatible, but does it run in some sort of compatibility mode when it > meets up with a foreigner? Sort of like how we Americans feel the need > to revert to pigeon English when we meet a foreigner with a tenuous > grasp of our language. Then we wonder why they speak that way, but > they mirror us! Sorry, I digress. No compatibility issues. Some TCPs, particularly on the sending end perform better than others. With certain TCP enhancements such as SACK, if one end has it and the other end doesn't, the two negociated and don't use the enhancement. There is plenty of proof that this works. > 5. Though router settings (buffering) can clearly play a part, it's > difficult to find recommendations or optimal settings for satcom router > links in the router documentation. The router vendors don't have much of a clue (at least once you get to the people writing that sort of documentation). In general you want as much buffering as you can get for TCP. More than 2 times the delay bandwidth product of buffering provides marginal improvement at the cost of increased delay so the usual recommendation is queueing of 1-2 times delay bandwidth product. > 6. This is the biggie for me. The trail of bread crumbs left by > the researchers is a little thin; leaves me hungry; makes me wonder if > I should just sit down and wait to be rescued. Can't they plant a > flag from time to time and say, "This is where we are today. If anyone > is experiencing this problem, here are the steps to take to minimize > it." The steps might range from specific tweaks to the application > (say, ftp window size) to alternate applications (XFTP), to O/S > upgrades (Solaris 2.6, when available). I am aware of a pretty good Web > page that attempt this, but it needs work. Sorry. Just the mailing lists. Maybe this is one reason why we have this WG (or at least that is what the charter says). :) BTW- I suggest that you don't want to hold your breath waiting for some new protocol to mature that defies the laws of physics. Don't expect an absense of claims about emerging protocols that do. > Paul Curtis ------------------------------ Date: Thu, 3 Jul 1997 18:50:37 -0400 From: hkruse1@ohiou.edu (Hans Kruse) Subject: Re: SAT TCP versus BER/ Number Session At 20:05 7/3/97, Harry Smith wrote: >1. I did simple maths and found that at present bit error rate ( 10 e-6 or >10e-7), the retransmissions would kill the performance. So I added >Commercial Reed Solomon error correction to move the BER to better than >10e-10. It took about 10% of the bandwidth. There are some commercial RS >boxes at about 20K each. This is in agreement with some of our studies. You did not mention the "flavor" of TCP you are using. The BER is not quite has harmful for TCPs that use SACK. However, in general it is better to "clean up" the satcom link. > >2. I arranged the data so that I had multiple TCP session between endpoint >( a type of session interleaving ). Whist this would not help the >composite TCP performance, on the average the performance of individual >session would be improved. I am not sure I understand. If you interleave the data over multiple sessions, you will improve the aggregate performance, not the individual ones; is that what you meant? This is essentially what we did in XFTP (a multi-session version of FTP), and what Netscape does in web requests. We have shown that this works for bulk transfers, just as you have seen. The overall concern, I think, is that this type of solution may be harmful if the multi-session transfer traverses a congested portion of a shared network, and it is hard to implement in general for things other than file transfer. Hans Kruse, Associate Professor McClure School of Communication Systems Management, Ohio University 9 S. College Street Athens, OH 45701 614-593-4891 voice, 614-593-4889 fax, hkruse1@ohiou.edu ------------------------------ End of tcp-over-satellite-digest V1 #30 *************************************** tcp-over-satellite-digest Saturday, July 12 1997 Volume 01 : Number 031 In this issues: Re: SAT TCP versus BER/ Number Session Re: For Discussion Questions re: Questions Re: Questions Reminder & New Deadline (satellite workshop at MobiCom'97) TCPSAT charter has been approved! RE: TCPSAT charter has been approved! Response to Booster Questions Re: TCPSAT charter has been approved! See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Sun, 06 Jul 1997 17:30:59 +1000 From: Harry Smith Subject: Re: SAT TCP versus BER/ Number Session At 06:50 PM 7/3/97 -0400, you wrote: >At 20:05 7/3/97, Harry Smith wrote: >>1. I did simple maths and found that at present bit error rate ( 10 e-6 or >>10e-7), the retransmissions would kill the performance. So I added >>Commercial Reed Solomon error correction to move the BER to better than >>10e-10. It took about 10% of the bandwidth. There are some commercial RS >>boxes at about 20K each. >This is in agreement with some of our studies. You did not mention the >"flavor" of TCP you are using. The BER is not quite has harmful for TCPs >that use SACK. >However, in general it is better to "clean up" the satcom link. I am using a commercial version that is included in the Dec Alpha UNIX. I can not use a non-commercial because of support concerns and the need for compatibility with other commercial products. > >> >>2. I arranged the data so that I had multiple TCP session between endpoint >>( a type of session interleaving ). Whist this would not help the >>composite TCP performance, on the average the performance of individual >>session would be improved. >I am not sure I understand. If you interleave the data over multiple >sessions, you will improve the aggregate performance, not the individual >ones; is that what you meant? I am can in number of tcp session between my two sites. In the initial design, all of the information was sent using a single TCP session. Now, I have divided the information between three of four different TCP session. This should result in the TCP interleaving. >This is essentially what we did in XFTP (a multi-session version of FTP), >and what Netscape does in web requests. > >We have shown that this works for bulk transfers, just as you have seen. >The overall concern, I think, is that this type of solution may be harmful >if the multi-session transfer traverses a congested portion of a shared >network, and it is hard to implement in general for things other than file >transfer. > Since I am dealing with a single point to point session, the congestion problem does not apply, ( I think) > >Hans Kruse, Associate Professor >McClure School of Communication Systems Management, Ohio University >9 S. College Street >Athens, OH 45701 >614-593-4891 voice, 614-593-4889 fax, hkruse1@ohiou.edu > > > ----------------------------------------- Harry Smith hsmith@netlink.com.au 3/847 Burwood Rd hsmith1@netcom.com Hawthorn, VIC 3122 011 613 9813 2590(H) 011 614 1223 0012(M) ------------------------------ Date: Mon, 7 Jul 1997 10:49:50 -0700 From: touch@ISI.EDU Subject: Re: For Discussion > From: Eric Travis > To: tcp-over-satellite@achtung.sp.trw.com > Subject: For Discussion > > Thomas Bohn's message is a reality check for the list. > ... > Much of the talk in Internet research circles is toward > increasingly high-speed pipes. Let's not forget about the > bandwidth underprivledged, that will probably include a large This is because the Internet protocols have already been (or are being) optimized for slower links. (excepting header compression, which existed, but is only recently being standardized). > Topology and bandwidth are important factors, yet there is > little discussion on either. As an example, if the satellite > link tends to be the bottleneck, and it approaches chronic > congestion during certain periods of the day, then some > mitigations *might* do more harm than good; These conditions > need to be brought into the larger discussion. This is why I have favored NOT using larger initial windows for TCP, for example. Especially because of the confluence of: - lots of connections - congestion on a shared link - short connections (10-20 KB) Larger initial windows can amplify these interactions, notably near the congestion point of a network (Sally Floyd reported recent simulation results confirming this). Joe ------------------------------ Date: Tue, 8 Jul 1997 10:05:11 -0500 From: "Goldstein, Robert" Subject: Questions Hello, I am new to this thread and have been monitoring it lately. I am extremely interested in your efforts and would like to understand more about the issues you are trying to solve. My interest stems from an article that Karen Hansen wrote on TCP/IP over satellites and what efforts are currently underway on improving it. I am no EE or IM professional by any stretch of the word, so please accept my apology now if I make light of any assumptions, make incorrect statements or leave important information out. I am currently working on a project that is evaluating broadband communications and one specific technology we are looking into is satellite. From what I have read and seen, the new Ka-Band satellite systems that will be launched over the next three to four years may be ideal for broadband data communications and could provide robust bi-directional capabilities, including, obviously, access to the internet. Hughes, Lockheed Martin, Loral etc.. are banking on this idea. However, my conversations with many network managers have given me an impression that satellite is not looked as a viable way to send large streams of data through versus frame relay and ATM over terrestrial lines. Unless the TCP/IP issue is resolved, satellite may not prove to be as accepted as once believed. Also, many software applications and O/S have buffers set at certain lengths for specific reasons. Just modifying kernels, buffers and windows may not be the right answer. You lose bandwidth in the network and can slow it down tremendously in order to utilize the satellite connection. Moreover, asking network managers to make such modifications is something I personally do not think any vendor, software company or service integrator wants to do. That is why I believe that global modifications to TCP/IP and standards is the only answer. So, here are a few questions I would like your feedback on: Do any "white papers" exist that sum up the issues and state the cost and benefits of TCP/IP over satellite in an easily understandable form? I read a very compelling and somewhat biased article on latency over satellites by people at TELEDESIC (http://www.teledesic.com/overview/latency.html). While I understand the motivation behind the paper, I must say that it lays out the problem and has a compelling argument for LEO satellites. Has anyone else read this? What are you feelings? Finally, I have been told that TCP/IP over satellites works just fine and the military has been using it for quite some time. If this is true, why is it in an issue? I appreciate your feedback and time. Thanks you. Robert A. Goldstein Corporate Strategy Compaq Computer Corporation Robert.Goldstein@Compaq.Com (T) 281-518-2280 ------------------------------ Date: Tue, 8 Jul 1997 11:33:00 -0400 From: matthew halsey Subject: re: Questions This is the preamble of a multipart MIME formatted message. If you are reading this text your mail system is most likely not capable of properly decoding MIME messages. To extract the contents of this message, save it to a file and then use an external MIME decoding utility. - --mime-boundary-iaanaabhmb-008B242D Content-type: text/plain; charset="ISO-8859" Content-disposition: inline Content-transfer-encoding: 7bit Form: Reply Text: (25 lines follow) Robert, I would like to address the last of your questions. At INTELSAT, we have had customers using our satellites for Internet traffic for quite some time and they are, by and large, happy with it. The issue is what you are trying to do. The satellite delay does impose a throughput limitation on TCP traffic. If this throughput limitation (typically a few 100 kbps) is the most restrictive on any link, then it will be noticed. However, in the public internet there are many tighter throughput limitations (such as modems, and overloaded fibers). Consequently the throughput limitation of regular satellite TCP is never normally reached. Therefore, satellite throughput capability is normally OK. The only result is then the delay. This is satellite's burden but, with the congested Internet causing so much delay anyway, it does not appear to be the big issue that it might be. There are plans to include accepted RFCs into TCP to give it a boost. These include SACK, Fast Recovery/Retransmit and Large Windows. The TCPSAT group is drafting an informational RFC to this end. Hope this helps a little. Matt Halsey Senior Engineer INTELSAT Original text: (66 lines follow) From ROBERT@SMTPGATE (Goldstein, Robert) {Robert.Goldstein%COMPAQ.com.@intelsat1.intelsat.int}, on 8/7/97 11:05 AM: To: TCP-OVER@SMTPGATE ("'TCP/IP SAT (POSTING)'") {tcp-over-satellite@achtung.sp.trw.com} Hello, I am new to this thread and have been monitoring it lately. I am extremely interested in your efforts and would like to understand more about the issues you are trying to solve. My interest stems from an article that Karen Hansen wrote on TCP/IP over satellites and what efforts are currently underway on improving it. I am no EE or IM professional by any stretch of the word, so please accept my apology now if I make light of any assumptions, make incorrect statements or leave important information out. I am currently working on a project that is evaluating broadband communications and one specific technology we are looking into is satellite. From what I have read and seen, the new Ka-Band satellite systems that will be launched over the next three to four years may be ideal for broadband data communications and could provide robust bi-directional capabilities, including, obviously, access to the internet. Hughes, Lockheed Martin, Loral etc.. are banking on this idea. However, my conversations with many network managers have given me an impression that satellite is not looked as a viable way to send large streams of data through versus frame relay and ATM over terrestrial lines. Unless the TCP/IP issue is resolved, satellite may not prove to be as accepted as once believed. Also, many software applications and O/S have buffers set at certain lengths for specific reasons. Just modifying kernels, buffers and windows may not be the right answer. You lose bandwidth in the network and can slow it down tremendously in order to utilize the satellite connection. Moreover, asking network managers to make such modifications is something I personally do not think any vendor, software company or service integrator wants to do. That is why I believe that global modifications to TCP/IP and standards is the only answer. So, here are a few questions I would like your feedback on: Do any "white papers" exist that sum up the issues and state the cost and benefits of TCP/IP over satellite in an easily understandable form? I read a very compelling and somewhat biased article on latency over satellites by people at TELEDESIC (http://www.teledesic.com/overview/latency.html). While I understand the motivation behind the paper, I must say that it lays out the problem and has a compelling argument for LEO satellites. Has anyone else read this? What are you feelings? Finally, I have been told that TCP/IP over satellites works just fine and the military has been using it for quite some time. If this is true, why is it in an issue? I appreciate your feedback and time. Thanks you. Robert A. Goldstein Corporate Strategy Compaq Computer Corporation Robert.Goldstein@Compaq.Com (T) 281-518-2280 Use Proportional Font: true Previous From: ROBERT@SMTPGATE (Goldstein, Robert) {Robert.Goldstein%COMPAQ.com.@intelsat1.intelsat.int} Previous To: TCP-OVER@SMTPGATE ("'TCP/IP SAT (POSTING)'") {tcp-over-satellite@achtung.sp.trw.com} Original to: TCP-OVER@SMTPGATE ("'TCP/IP SAT (POSTING)'") {tcp-over-satellite@achtung.sp.trw.com} Attachment Count: 0 - --mime-boundary-iaanaabhmb-008B242D Content-type: application/octet-stream; name="ATTRIBS.BND" Content-disposition: attachment; filename="ATTRIBS.BND" Content-transfer-encoding: base64 QmV5b25kIFBhY2tlZCBBdHRyaWJ1dGVzAACGjdc5CgAAAAAAQmV5b25kIFByb3ByaWV0YXJ5 IERhdGEaAAAAABEAAAAAAAQADQCWAgAAAAAAAAAAAAAAAAAAAAAAAAAAT3JpZ2luYWwgdGV4 dIYLRgoAAAAAAAAAAAAAhAIDAIYLfgICAAIAAABAAAIAAQABAM4AAAAAAAAAAgDPALgKAAAA AAAAOP8AAAAAAACQAQAAAAAAAAAATVMgU2FucyBTZXJpZgAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAOP8AAAAAAACQAQAAAAAAAAAAQ291cmllcgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AIAAAQABAHUAAQB1AM4AAQDOAM8AAgDPAPd/AgDWAPd/AgDXAPd/AgAZAfd/AgBgAfd/AgCl Afd/AgDoAfd/AgAqAvd/AgBzAvd/AgC6Avd/AgDXAvd/AgDYAvd/AgAZA/d/AgBbA/d/AgCh A/d/AgDoA/d/AgApBPd/AgBqBPd/AgCuBPd/AgC0BPd/AgC1BPd/AgD7BPd/AgBBBfd/AgCF Bfd/AgDNBfd/AgDuBfd/AgDvBfd/AgA0Bvd/AgB3Bvd/AgC/Bvd/AgADB/d/AgA/B/d/AgCA B/d/AgDDB/d/AgAJCPd/AgARCPd/AgASCPd/AgBOCPd/AgBPCPd/AgCVCPd/AgDdCPd/AgDe CPd/AgAjCfd/AgBFCfd/AgCLCfd/AgDUCfd/AgAcCvd/AgA6Cvd/AgA7Cvd/AgCBCvd/AgDF Cvd/AgDiCvd/AgDjCvd/AgAVC/d/AgAWC/d/AgAqC/d/AgA9C/d/AgBZC/d/AgB1C/d/AgCG C/d/AgCHC/d/AAAAAAAAAAAAAAAAZAABgAEBAAMBgAQBAAYBgAcBAAkBgAoBAAwBgA0BAA8B gBABABIBgBMBABUAAAAAAAAAAAAAAABkAAGwAQFgAwEQBQHABgFwCAEgCgHQCwGADQEwDwHg EAGQEgFAFAHwFQGgFwAA - --mime-boundary-iaanaabhmb-008B242D-- ------------------------------ Date: Tue, 08 Jul 1997 12:28:35 -0700 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Re: Questions > Do any "white papers" exist that sum up the issues and state the cost > and benefits of TCP/IP over satellite in an easily understandable form? We published a some-what introductory paper on Internet over Satellite in INET'97. The on-line version: http://www.wins.hrl.com/people/ygz/papers/inet97.html It is inappropriate to make simple statement on whether satellites make better Internet access media. We argued that many TCP/IP applications perform to user expectation while others are hindered by latencies. The later can be improved in several ways: 1. adopt a set of TCP options, such as ones being worked on in this working group, 2. use protocol-boosting network middleware at the boundaries of satellite networks and terrestrial networks, such as TCP-spoofing or split-TCP, and 3. use TCP wisely in applications, such as HTTP 1.1 instead of HTTP 1.0 in web browsing. Also, TCP is not the only game in the Internet. For multicast applications (MBONE, "push", webcast, whatever it is), few can beat the benefit of satellite. We have compared MBONE over terrestrial Internet v.s. MBONE over satellite Internet (using NASA ACTS). Although the latency is larger, the delay seen by vic/vat is actually smaller over satellite than over terrestrial, because of less jitter and fewer losses (going through fewer routers). Yongguang Zhang ============================================================================== Yongguang Zhang, Ph.D., Research Staff Member (Computer Science) Hughes Research Laboratories, RL-96, 3011 Malibu Canyon Road, Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com phone: 310-317-5147 URL: http://www.wins.hrl.com/people/ygz fax: 310-317-5695 ------------------------------ Date: Wed, 09 Jul 1997 13:42:42 -0700 From: ygz@isl.hrl.hac.com (Yongguang Zhang) Subject: Reminder & New Deadline (satellite workshop at MobiCom'97) I appologize if you have received this more than once ... This is a CFP reminder for the 2nd Int'l workshop on Satellite-based Information Services (WOSBIS), in connection with MobiCom'97. The deadline for extended abstracts or full papers has been extended to July 25. URL: http://www.wins.hrl.com/conferences/WOSBIS97/ =========================================================================== 2nd Int'l Workshop on Satellite-based Information Services (WOSBIS'97) CALL FOR PAPERS October 1, 1997, Budapest, Hungary (In connection with ACM MobiCom'97) ----------------------------------- Sponsored by ACM Sigmobile (pending) and NASA Lewis Research Center Satellite communications will play an increasingly important role in our future information-based society. This trend is evidenced by the large number of systems planned or in operation (e.g. GPS, ACTS, DirecTV(TM), Iridium, DirecPC, SpaceWay, Teledesic). As its popular predecessor held in conjunction with MobiCom'96, the objective of this workshop is to provide a forum for exploratory research contributions on satellite applications and services. The services are characterized by direct or global broadcast capabilities of LEO, MEO, GEO satellites, low setup costs, high and possibly asymmetric bandwidth, and unconventional network routing. Applications of such services are often real-time, mobile, high bandwidth, and they include telemedicine, public information services, education, entertainment, Internet access, digital battlefield, emergency and disaster response. Topics of contributed papers will include, but are not limited to: * Applications using satellite services * Software and architectures for integration with terrestrial networks * Data management and file systems in satellite applications * Distributed computing using satellite communication * Software techniques for reducing latency in satellite applications * Satellite network protocols and routing * ATM over satellites and quality of services * TCP/IP for satellite communications * Internet applications, WWW access and cache through satellites * Direct broadcast and multicast applications * Asymmetric communication services * Security issues in satellite communications and applications * Reliability and scalability in satellite applications * Software for satellite switch control * Mobile computing and location tracking in satellite applications * Inter-satellite communication and network management * Mobile satellite services * Middleware for satellite-based information services development The setting of the workshop will be informal, and will encourage discussion and interaction. We solicit the submission of research papers, position papers, experience papers, and panel proposals. SUBMISSIONS We invite papers for presentation and discussion at the workshop. Send a postscript copy of the paper by email to the address below. The page limit on all submissions is 5-10 pages, double spaced, 10 pt minimum (approximately 1500-2500 words). Yongguang Zhang Hughes Research Labs, RL-96 Malibu, CA 90265 E-mail: ygz@isl.hrl.hac.com PROCEEDINGS: An informal proceedings will be published and distributed at the workshop. The format will be similiar to standard ACM proceedings: double column, single spaced and the page limit on each paper is 12. Authors of accepted papers are encouraged to submit their final papers in HTML (in addition to Postscript). The HTML versions can be slightly different than the Postscript versions to add hyperlinks to relevant work, such as to executable demos available at their own sites. An electronic proceeding linking together the HTML versions will be made available to major Web search engines and mirrored at the sites of the organizers. IMPORTANT DATES: * Papers Due: July 25, 1997 * Acceptance Notification: Aug. 15, 1997 * Final papers due: Sep. 15, 1997 * Workshop: Oct. 1, 1997 WEB SITE: http://www.wins.hrl.com/conferences/WOSBIS97/ GENERAL CHAIRS: Son K. Dao, Hughes Research Labs. Ouri Wolfson, EECS Dept, Univ. of Illinois, Chicago TECHNICAL PROGRAM CO-CHAIRS: John S. Baras, University of Maryland Gary Minden, Univ. of Kansas. Yongguang Zhang, Hughes Research Labs. PUBLICITY: Walid Dabbous, INRIA, France Nghia Pham, Eutelsat TECHNICAL PROGRAM COMMITTEE: Rafael Alonso, David Sarnoff Research Center John Baras, Univ. of Maryland Kul Bhasin, NASA Lewis Research Center Tzi-cker Chiueh, SUNY, Stony Brook Imrich Chlamtac, Boston Univ. Walid Dabbous, INRIA, France Son Dao, Hughes Research Labs. Raj Jain, Ohio State Univ. Randy Katz, UC Berkeley Henry Korth, Bell Labs Gary Minden, Univ. of Kansas Shawn Ostermann, Ohio Univ. Csaba Szabo, Technical University of Budapest Ouri Wolfson, Univ. of Illinois, Chicago Yechiam Yemini, Columbia Univ. Yongguang Zhang, Hughes Research Labs. ------------------------------ Date: Fri, 11 Jul 1997 07:56:02 -0700 From: Aaron Falk Subject: TCPSAT charter has been approved! TCPSAT charter has been approved by the IESG. We're a working group now, it's official! aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division Aaron.Falk@trw.com ------------------------------ Date: Fri, 11 Jul 1997 13:05:42 +0000 From: hdhillon@attmail.com (Harry S Dhillon) Subject: RE: TCPSAT charter has been approved! Aaron...is their an initial draft, framework document or presentations that will be reviewed in Munich? Are there any plans or Web location(s) for this? Thanks. Harry Dhillon Lucent Technologies - ---------- From: Aaron Falk Sent: Friday, July 11, 1997 3:56 AM To: tcp-over-satellite Subject: TCPSAT charter has been approved! TCPSAT charter has been approved by the IESG. We're a working group now, it's official! aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division Aaron.Falk@trw.com ------------------------------ Date: Fri, 11 Jul 1997 15:09:12 -0400 (EDT) From: Deborah Bakin Subject: Response to Booster Questions Hello, Sorry it's taken me so long to reply to Eric's mail, I got caught up in other projects (imagine that ;-)) I'd like to start out by thanking Eric for all the good questions. It's really helpful to get a new perspective on our work. I hope this response will open a continuing dialog. o Are there (will there be) soft-copies available for the papers identified on your Web site? Unfortunately many of our papers are not yet published, and as such we feel it might not be appropriate to put them out on the net. However, we are happy to send copies of our papers (electronic or paper) to anyone who is interested. We have a paper which was just accepted to JSAC which is an overview of the protocol boosters, and another paper which was just presented at IMSC'97 which discusses the FZC booster. Just send me an e-mail at dbakin@bellcore.com. In addition, we are in the process of updating our web site (gump.bellcore.com:8000/boosters) to include overview slides on boosters which we presented to DARPA (our current funding source for the project). Stay tuned! o This might be covered in one of the papers; Why, in particular, do you feel boosters are particularly suited for networks with satellite links. Boosters are protocol adaptors. Adaptors are most useful when the connection consists of (at least) two networks which have very different link characteristics. For example, satellite systems usually have very different characteristics then wireline networks (some combination of increased latency, noise, bandwidth scarcity, and bandwidth asymmetry). [More detail on this can be found in the JSAC paper.) One might ask why we need protocol adaptation in the first place. Ideally, the satellite community would get a satellite friendly TCP deployed everywhere in the Internet. Unlike applications, which can spread rapidly, protocol infrastructure must be done incrementally. Boosters (and other adaptation techniques) help protocols evolve (and allow a survival of the fittest!) by allowing small incremental changes where conditions are suitable. Does this still hold true if the satellite links are not lossy? Protocol boosters are a general category of solution. The boosters we have worked on to-date are part of the error control family of boosters. If the link is not lossy, then the error control boosters will not provide any benefit. However, other booster families could still be useful. For example, a booster which improves latency/QoS control could be very useful for links which have increased latency, asymmetry, or bandwidth scarcity. o Have any other "boosters" been designed and tested for environmental effects other than channel errors? Aye, there's the rub. It is true that the only boosters that have been designed and tested to-date are those in the error control family. But this is a moving target, the latency control boosters are under study. o How are protocol boosters different than other things that would fall into the "spoofing" bucket, and why are they better than using full-fledged application proxies to separate significantly different subnetwork environments? Aren't most (if not all) spoofing techniques parasitic (and use information from other protocols)? I find the definition of spoofing to be a bit murky, but the definition I'm using is that spoofing implies fooling the end system in some way. Boosters differ from spoofing in two (related) ways: 1) we can delete a booster and everything still works fine and 2) we're not changing the meaning of the end-to-end protocol messages. (e.g., if I get an ACK, it should mean the system I sent it to has received the packet). Robustness is the key issue. A secondary issue is reducing processing. Application proxies must read/write data into memory. Boosters may need to read all the data as it passes by (and may delay or modify the messages); but they do so as little as possible. The Protocol Booster architectures that we have implemented do not affect the "fast path processing" in network routers (or any other entity). Boosters are only called into play for flows in which they are designated. o How do you foresee protocol boosters *transparently* mitigating the satellite environments problems such as long-delay paths and asymmetry? How about non-transparently but without violating TCP's message semantics or syntax? Does the definition of transparency you are using also include not having a negative impact on other flows in the network (are you fair, or does the parasitic nature include eating the bandwidth of other flows)? Transparency only refers to the fact that the application need not know the booster is there (except, hopefully, for the performance improvement). Boosters are designed to be applied selectively - which in fact makes them fair, rather than unfair, to non-boosted flows. Techniques such as link adaptation or creating new protocols treat all traffic alike, and in so doing they have the potential to waste precious resources e.g., link capacity or processing capacity. o How do boosters interact with TCP's congestion avoidance and control mechanisms? For example, say I've got a path that is experiencing congestion, what is to stop a FEC booster pair (running somewhere in the path) from doing it's best to correct for the *congestion* based losses and preventing a traffic source from reducing it's transmission rate? Can boosters distinguish losses due to corruption (erasure seems tougher) and congestion? While I can conceptualize how boosters can improve the performance of a particular connection, do they "work and play well with others"? The easy answer is that FEC should only be applied to recover from non-congestion errors. The (current) FEC control assumes all loss is due to error; therefore it increases redundancy when errors increase (up to some limit). This is undesirable positive feedback if errors are due to congestion. It is an interesting question (meaning we don't know the answer) whether we can distinguish losses and act appropriately. Presently we are working under the premise that most errors in satellite networks come from corruption, not from congestion. Thus, we feel confident in the use of the FEC booster over the satellite portion of the connection. The issue of how well boosters "work and play with others" is an important one. We believe that if designed correctly, they can help each other out. At the same time, however, we realize that it is entirely possible to hang yourself. Clearly much testing will be needed to ensure proper booster interaction. o How much a priori information about a network topology (or a particular connections path) must be available prior to using boosters? It is dependent on which boosters you are using. No booster should be dependent on the topology, meaning they will not fail as the topology changes. However, some boosters may become less effective. o You note that boosters can "be used with IP-layer authentication and security methods", but if the transport header and payload were encrypted (say, I trust you to route my packets, but I don't trust you to look at the contents), can boosters still boost performance? We believe IPSEC will become widely used in the next few years. Therefore, the error control booster family is being designed to use whatever information is available. The FZC booster works transparently with IPSEC. In fact, only one of our boosters (ARQ1), so far, cannot be used with IPSEC. Clearly application spoofing does not work with end-to-end IPSEC. More information translates into greater efficiency. An FEC booster, for example, may use TCP sequence number information to perform "code combining." Without the TCP header, it defaults to using just IP information (e.g., it knows packet destination/source and packet boundaries and fragment id). > You state that *any* solution for the satellite environment *must* > include UDP and RTP; > > What special problems do UDP and RTP have that need to be addressed? Applications which rely on UDP (instead of TCP) are becoming more and more popular. One example of this is multicast, which is a natural application for satellites. Thus we need to include UDP and RTP in any solitions for the satellite environment. The issues with UDP and RTP are reliability and bandwidth efficiency. The FEC booster can improve (though not guarantee) reliability. We are also looking to experiment with other error control boosters for UDP and RTP (ARQ2 in particular). We are currently looking at other Booster families (other than the error control family) for helping with long-delay paths and asymmetry. However, we really haven't thought these through too well yet - if anyone is interested, we could get together and talk over the ideas sometime! Looking forward to more great discussions. --Deb, Tony, and Bill ------------------------------ Date: Fri, 11 Jul 1997 16:45:26 -0700 From: Aaron Falk Subject: Re: TCPSAT charter has been approved! Harry S Dhillon wrote: > Aaron...is their an initial draft, framework document or > presentations that will be reviewed in Munich? Eric is still working on the draft and should post it RSN (real soon now). I expect that a summary of the document and comments/issues relating to it will be presented in Munich. We'll see what develops. > Are there any plans or Web location(s) for this? The draft will be posted on the wg web site http://www.isr.umd.edu/CSHCN/Links/IPoS/ aaron - -- Aaron Falk (310) 814-4932 TRW, Inc Electronics Systems & Technology Division Aaron.Falk@trw.com ------------------------------ End of tcp-over-satellite-digest V1 #31 *************************************** tcp-over-satellite-digest Saturday, July 19 1997 Volume 01 : Number 032 In this issues: IEEE Communications Suscribe me please See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Mon, 14 Jul 1997 10:34:44 -0400 From: "Eric A. Bobinsky" Subject: IEEE Communications No doubt almost everyone on these lists get IEEE Communicatoins Magazine, but for those who might not, the July 1997 issue contains a number of very good articles on broadband satellite communications. >--------------------------------------------------------------------------< Eric A. Bobinsky Terasphere Corp. Telephone (216) 243-2992 343 W. Bagley Road, Suite 405 Facsimile/Telecopie (216) 243-2934 Post Office Box 10 International +1 216 243 2992/2934 Berea, Ohio 44017 main@terasphere.com USA http://www.terasphere.com >--------------------------------------------------------------------------< ------------------------------ Date: Fri, 18 Jul 1997 19:02:20 +0200 From: Ahmed Serhrouchni Subject: Suscribe me please Suscribe me please ------------------------------ End of tcp-over-satellite-digest V1 #32 *************************************** tcp-over-satellite-digest Friday, July 25 1997 Volume 01 : Number 033 In this issues: ACM/IEEE MobiCom'97 Advance Program FYI-- FW: Draft IPPM Charter See the end of the digest for information about tcp-over-satellite-digest. ---------------------------------------------------------------------- Date: Thu, 24 Jul 97 22:32:36 -0400 From: Dave Johnson Subject: ACM/IEEE MobiCom'97 Advance Program MobiCom'97 is being held September 26 through 30 in Budapest, and on October 1, we are holding the Second International Workshop on Satellite-based Information Services (WOSBIS'97) in the same location, associated with the conference. Both of these events may be of interest to readers of this list. I have included a copy of the MobiCom'97 Advance Program below. For more information on WOSBIS'97, visit the WOSBIS'97 homepage at http://www.wins.hrl.com/conferences/WOSBIS97/. Dave Johnson Program Co-Chair, MobiCom'97 --------------------------------------------------------------------------- MobiCom'97 Advance Program and Call for Participation THE THIRD ANNUAL ACM/IEEE INTERNATIONAL CONFERENCE ON MOBILE COMPUTING AND NETWORKING The Palace of the Hungarian Academy of Sciences Budapest, Hungary Tutorials and Conference: September 26-30, 1997 Workshops: October 1, 1997 The wireless communication revolution is bringing fundamental changes to telecommunication and computing. Wide-area cellular systems and wireless LANs promise to make integrated networks a reality and provide fully distributed and ubiquitous mobile computing and communications, thus bringing an end to the tyranny of geography. Furthermore, services for the mobile user are maturing and are poised to change the nature and scope of communication. This conference, the third of an annual series, serves as the premier international forum addressing networks, systems, algorithms, and applications that support the symbiosis of mobile computers and wireless networks. The MobiCom'97 technical program features the presentation of 26 excellent papers, selected after detailed review from over 100 submissions received this year. In addition, the program will include 4 panel discussion sessions and 2 invited keynote speakers, plus a number of tutorials before the conference and 2 workshops after the conference. All together, MobiCom'97 offers an outstanding technical program and promises to be an exciting conference on the cutting edge of mobile computing and networking. We invite you to join us for MobiCom'97 and hope to see you in beautiful and historic Budapest! Important Dates Hotel Reservation Deadline: August 15, 1997 Early Registration Deadline: August 29, 1997 For more information, please contact either of the Program Co-Chairs: David B. Johnson, Carnegie Mellon University, dbj@cs.cmu.edu, Telephone: +1 412 268 7399, Fax: +1 412 268 5576; or Christopher Rose, Rutgers University, crose@ece.rutgers.edu, Telephone: +1 908 445 5250, Fax: +1 908 445 2820. For complete details and all the latest MobiCom'97 information, visit the MobiCom'97 Home Page on the World Wide Web at: http://www.monarch.cs.cmu.edu/~mobicom97/ --------------------------------------------------------------------------- The location for MobiCom'97 is the Palace of the Hungarian Academy of Sciences. The Academy is the most prestigious scientific institution in Hungary, and the Palace, the central building of the Academy, is one of the most beautiful buildings in Budapest. It is located on the Pest side of Budapest, east of the Danube River, near the Chain Bridge. The Palace overlooks the northern end of Roosevelt Square, along the embankment of the Danube River. Built between 1861 and 1865, this neo-Renaissance building is decorated, both inside and outside, with sculptures and wall-paintings by the most outstanding Hungarian artists of the age. The conference hotels are all located nearby, and many of the sights and shops of Budapest are within easy walking distance. Budapest is easily reached by air, rail, road, or river. Budapest's airport is served by a number of major world airlines, with nonstop flights from many foreign cities. For transportation from the airport to your hotel, we suggest the LRI MINIBUS Service. The information desk for this shuttle service is located in the center of the airport arrival lobby. You can order this service inside the baggage claim area as well. The price of the shuttle is 1200 Hungarian Forint (HUF) per person (approximately US $7-8) one way. Citizens of some countries may require entry visas to visit Hungary. No visa is needed for citizens of the USA, Canada, or any of the European countries, except Albania, Turkey, Kazakhstan, and Uzbekistan. A visa is needed for citizens of Japan and Australia. If in doubt, please check this with the Hungarian Embassy in your country. They will help you to complete the necessary entry formalities. --------------------------------------------------------------------------- FRIDAY, SEPTEMBER 26 -------------------- 8:30am - 5:00pm Tutorial 1 (Full-Day) * Wireless ATM: Standards, Architectures, Protocols & Implementation, Lou Dellaverson (Motorola, USA), C.-K. Toh (Hughes Research Laboratories, USA), and Arup Acharya (NEC, USA) 8:30am - 5:00pm Tutorial 2 (Full-Day) * Mobile IP: Adding Mobility to the Internet, Charles E. Perkins (Sun Microsystems, USA) SATURDAY, SEPTEMBER 27 ---------------------- 8:30am - 5:00pm Tutorial 3 (Full-Day) * Simulation of Large Mobile Wireless Networks, Rajive Bagrodia and Mario Gerla (University of California at Los Angeles, USA) 8:30am - 12:00pm Tutorial 4 (Half-Day, Morning) * Cellular Wireless Networks: Principles and Operation, Zygmunt J. Haas (Cornell University, USA) 1:30pm - 5:00pm Tutorial 5 (Half-Day, Afternoon) * Disconnected and Weakly Connected Access to the World Wide Web: Issues and Techniques, Murray S. Mazer (Open Group Research Institute, USA) 7:00pm - 9:00pm Welcome Reception SUNDAY, SEPTEMBER 28 -------------------- 1:00pm - 2:00pm Registration 2:00pm - 3:30pm Opening Session * Welcome and opening remarks * Awards presentation * Opening Keynote Speaker 3:30pm - 4:00pm Break 4:00pm - 5:30pm Session 1: Reconfiguration and Adaptation * Composable Ad-hoc Mobile Services for Universal Interaction, Todd Hodes, Randy H. Katz, Edouard Servan-Schreiber, and Lawrence Rowe (University of California at Berkeley, USA): BEST STUDENT PAPER AWARD * Dynamic Network Configuration Support for Mobile Computers, Jon Inouye (Oregon Graduate Institute, USA) * Location-Aware Mobile Applications based on Directory Services, Henning Maass (Philips Research Laboratories Aachen, Germany) Evening Conference Dinner Banquet MONDAY, SEPTEMBER 29 -------------------- 8:30am - 10:00am Session 2: Wireless Network Architectures * Reliable Broadcast in Mobile Multihop Networks, Elena Pagani and Gian Paolo Rossi (Universita degli Studi di Milano, Italy) * Route Optimization in Mobile ATM Networks, Gopal Dommety (Ohio State University, USA), Malathi Veeraraghavan (Bell Laboratories, USA), and Mukesh Singhal (Ohio State University, USA) * Wireless Andrew: Experience Building a High Speed, Campus-Wide Wireless Data Network, Bernard J. Bennington and Charles R. Bartel (Carnegie Mellon University, USA) 10:00am - 10:30am Break 10:30am - 12:00pm Concurrent Sessions Session 3A: Mobile and Wireless Data Delivery * Geographic Addressing and Routing, Julio C. Navas and Tomasz Imielinski (Rutgers, USA) * The Effects of Asymmetry on TCP Performance over Wide-Area Wireless Networks, Hari Balakrishnan, Venkata N. Padmanabhan, and Randy H. Katz (University of California at Berkeley, USA) * Log-time Algorithms for Scheduling Single and Multiple Channel Data Broadcast, Sohail Hameed and Nitin H. Vaidya (Texas A&M University, USA) Session 3B: PANEL 1 * Building and Managing Large Wireless LANs: Real-World Experiences, Moderator: Victor Bahl (Microsoft, USA) 12:00pm - 2:00pm Conference Lunch * Luncheon Keynote Speaker 2:00pm - 3:30pm Concurrent Sessions Session 4A: Multimedia and QoS Issues * Multimedia Communication in Cellular PACS Network, Yukio Hashimoto and Behcet Sarikaya (University of Aizu, Japan); and Mehmet Ulema (DaeWoo Telecom, USA) * Delivering Diverse Delay/Dropping QoS Requirements in a TDMA Environment, Jeffrey M. Capone and Ioannis Stavrakakis (Northeastern University, USA) * Uplink CDMA Systems with Diverse QoS Guarantees for Heterogeneous Traffic, Sunghyun Choi and Kang G. Shin (University of Michigan, USA) Session 4B: PANEL 2 * Commercial Applications of Mobile Ad Hoc Networking: Are We Kidding Ourselves?, Moderator: M. Scott Corson (University of Maryland, USA) 3:30pm - 4:00pm Break 4:00pm - 5:00pm Session 5: Wireless Error Control * An Adaptive Hybrid ARQ Scheme with Concatenated FEC Codes for Wireless ATM, Inwhee Joe (Georgia Institute of Technology, USA) * Low Power Error Control for Wireless Links, Paul Lettieri, Christina Fragouli, and Mani B. Srivastava (University of California at Los Angeles, USA) Evening Dinner Cruise (optional) TUESDAY, SEPTEMBER 30 --------------------- 8:30am - 10:00am Session 6: Mobile IP * Mobile Multicast (MoM) Protocol: Multicast Support for Mobile Hosts, Tim Harrison, Carey L. Williamson, Wayne Mackrell, and Richard B. Bunt (University of Saskatchewan, Canada) * A New Multicasting-based Architecture for Internet Host Mobility, Jayanth P. Mysore and Vaduvur Bharghavan (University of Illinois at Urbana-Champaign, USA) * A Public-Key Based Secure Mobile IP, John Zao, Stephen Kent, Joshua Gahm, Gregory Troxel, Matt Condell, Pam Helinek, Nina Yuan, and Isidro Castineyra (BBN, USA) 10:00am - 10:30am Break 10:30am - 12:00pm Concurrent Sessions Session 7A: Location Management and Handover * A New Location Update Strategy for Cellular Networks and its Implementation using a Genetic Algorithm, Sajal K. Das and Sanjoy K. Sen (University of North Texas, USA) * A Dynamic Paging Scheme for Wireless Communication Systems, Guang Wan and Eric C. Lin (Southern Methodist University, USA) * A Connection Handover Protocol for LEO Satellite ATM Networks, Huseyin Uzunalioglu, Wei Yen, and Ian F. Akyildiz (Georgia Institute of Technology, USA) Session 7B: PANEL 3 * Integration of Wireless and Wired Networks: Visions and Reality, Moderator: Mooi Choo Chuah (Lucent, USA) 12:00pm - 1:30pm Lunch 1:30pm - 3:00pm Concurrent Sessions Session 8A: Protection in Mobile Computing * A Protection Scheme for Mobile Agents on Java, Daniel Hagimont and Leila Ismail (INRIA, France) * Ticket Based Service Access for the Mobile User, Bhrat Patel and Jon Crowcroft (University College London, UK) * Dealing with Server Corruption in Weakly Consistent, Replicated Data Systems, Mike Spreitzer, Marvin Theimer, and Karin Petersen (Xerox PARC, USA); Alan Demers (Oracle Corporation, USA); and Doug Terry (Xerox PARC, USA) Session 8B: PANEL 4 * QoS in the Next Generation Mobile Internet: What is Feasible?, Moderator: Andrew T. Campbell (Columbia University, USA) 3:00pm - 3:30pm Break 3:30pm - 5:00pm Session 9: Proxy-Based Architectures * Support for Mobile Pen-Based Applications, Wayne Citrin, Paul Hamill, Mark D. Gross, and Adrienne Warmack (University of Colorado at Boulder, USA) * A General Purpose Proxy Filtering Mechanism Applied to the Mobile Environment, Bruce Zenel (Columbia University, USA) and Dan Duchamp (AT&T Labs - Research, USA) * Web Browsing in a Wireless Environment: Disconnected and Asynchronous Operation in ARTour Web Express, Henry Chang, Carl Tait, Norman Cohen, Moshe Shapiro, and Steve Mastrianni (IBM T.J. Watson Research Center, USA); and Rick Floyd, Barron Housel, and David Lindquist (IBM, USA) 5:00pm Conference Adjourns WEDNESDAY, OCTOBER 1 -------------------- All day Workshops * The Second International Workshop on Satellite-based Information Services (WOSBIS'97). For more information, visit the WOSBIS'97 homepage at http://www.wins.hrl.com/conferences/WOSBIS97/. * The First International Workshop on Discrete Algorithms and Methods for Mobile Computing and Communications (DIAL-M). Visit the DIAL-M homepage at http://www.polytechnique.fr/poly/~derepas/dialm/ for more information. --------------------------------------------------------------------------- TUTORIAL 1 ---------- Wireless ATM: Standards, Architectures, Protocols & Implementation Dr. Lou Dellaverson (Motorola, USA), Dr. C.-K. Toh (Hughes Research Laboratories, USA), and Dr. Arup Acharya (NEC, USA) Friday, September 26 8:30am - 5:00pm ATM is currently viewed as the next high speed integrated network paradigm, supporting different classes of traffic and providing quality of service. Mobile communications have evolved and created a significant impact on the way we work and communicate. The convergence of mobile communications, computing, and ATM gives rise to Wireles ATM networks. While ATM helps to bring multimedia to the desktop, Wireless ATM provides similar services to mobile computers and devices. In addition, Wireless ATM networks provide seamless integration with ATM-based B-ISDN networks. This tutorial will cover system-level architectures for mobile/wireless ATM with necessary radio protocols for wireless ATM access and networking protocols to support mobility management. Standardization activity within the ATM Forum's WATM group will be presented along with implementation experience from research prototypes of mobile and wireless ATM. This tutorial will not only benefit researchers, professors, students, but also consultants, network engineers and managers who wish to acquire the knowledge and practical know-how on Wireless ATM. TUTORIAL 2 ---------- Mobile IP: Adding Mobility to the Internet Charles E. Perkins (Sun Microsystems, USA) Friday, September 26 8:30am - 5:00pm The Internet is growing by leaps and bounds, and likewise mobile computers are becoming more and more popular. When mobile computers move and attach themselves to new networks within the Internet, they can use Mobile IP as a means to achieve seamless roaming transparently to application software. In this situation, transparent means that the applications work just as before and don't need to be recompiled or reconfigured. Seamless means that roaming from one place to another occurs without inconvenience to the user. As long as a physical communication path exists, the user might not even be aware when movement has happened. The objective of this tutorial is to lay out all the necessary protocol technology to allow mobile computers to use Mobile IP, and to describe the relevant operation of other protocols which can be used to aid mobility, such as DHCP and Service Location Protocol. Topics that will be covered include Agent Advertisements, registration procedures, tunneling mechanisms, the role of security, and home agents and foreign agents. We will also cover how to set up a home network, getting care-of addresses via DHCP, Route Optimization, smooth handoffs, IPv6 mobility support, and the Service Location Protocol. In addition, we will look at an architectural model for supporting nomadic users under development within the Cross-Industry Working Team (XIWT) in the "Nomadicity" group. TUTORIAL 3 ---------- Simulation of Large Mobile Wireless Networks Prof. Rajive Bagrodia and Prof. Mario Gerla (University of California at Los Angeles, USA) Saturday, September 27 8:30am - 5:00pm Protocols for wireless networks are complex to design, evaluate and implement. Their performance depends on a combination of factors that include multimedia traffic patterns, mobility models, application objectives, processor characteristics, and radio characteristics. Evaluation of a protocol as a function of these diverse parameters is analytically intractable. Given the complexity of the radio environment, sequential simulation of networks with thousands of nodes requires several days, and perhaps, even weeks. To make the design more interactive, it is imperative to reduce the turnaround time for the models. The goal of this tutorial is to describe efficient simulation techniques for very large mobile wireless networks and to present some representative case studies. The environment has been built using the Maisie simulation language at UCLA. A number of approaches to reducing the simulation time for such models will be presented including parallel simulation, hierarchical modeling, and multi-paradigm models. The tutorial will begin with an overview of existing simulators, including OPNET, Bones, and other commercial products. The primary emphasis of the tutorial is on presenting the use of Maisie for parallel simulation of network models and their subsequent porting into physical implementation. The sources of overhead in the parallel execution of network models will be discussed together with methods to reduce their impact. Common pitfalls encountered in the design of parallel simulation models will be discussed. We will also describe techniques to port simulation models to protocol implementations. Finally, a number of case studies will be presented to highlight the lessons that have been learned in the design, simulation, and implementation of wireless network protocols. TUTORIAL 4 ---------- Cellular Wireless Networks: Principles and Operation Prof. Zygmunt J. Haas (Cornell University, USA) Saturday, September 27 8:30am - 12:00pm This tutorial addresses the basic networking concepts of mobile cellular and wireless networks, exposing both the theoretical and practical aspects of mobile communication. As an introduction, basic enabling technology will be presented, such as the cellular principle and multiple access technologies (e.g., CDMA). Following this introduction to mobile radio, we will investigate the underlying techniques used in design and operation of cellular networks, including handoff schemes, channel assignment and power control algorithms, common-air protocols (e.g., IS-54/136, IS-95, GSM, etc.), and microcellular architectures. Some more advanced concepts, such as macrodiversity and multi-tier wireless networks, will be briefly discussed. Next, we will address the subject of user mobility support in the wireless environment. In particular, call processing functions, which include roaming, routing, and registration, will be explained. The differences between mobility management in data networks and in voice networks will be clarified. As an example, a comparison of the Cellular Digital Packet Data (CDPD) and Internet mobility support through Mobile IP will be presented. The tutorial will be augmented with abundance of examples from existing and proposed future wireless networks. The tutorial is targeted towards broad audience, both from the academic and the industrial environments. TUTORIAL 5 ---------- Disconnected and Weakly Connected Access to the World Wide Web: Issues and Techniques Dr. Murray S. Mazer (Open Group Research Institute, USA) Saturday, September 27 1:30pm - 5:00pm This tutorial addresses the concepts, issues, and techniques involved in supporting weakly connected and disconnected access to Web-based information resources. ``Access'' includes both reading and writing - in addition to continuing to browse under diminished bandwidth conditions, the user may wish to create or change content, having it integrated back into the Web when the connectivity is sufficient. As background, we will review techniques used for disconnected and weakly connected access to network-based file systems. We will compare and contrast file systems and the World Wide Web, pointing out numerous ways in which the two types of information systems differ (and how those differences affect the adaptation of techniques from the file system space to the Web space). The tutorial will include a review of systems for "off-line browsing" (a.k.a. disconnected reading) and for filtering requests and responses to adapt to changing bandwidth conditions. We will address issues, techniques, and limitations regarding architectural choices, meta-data requirements, data management, "weblet" management, consistency, pre-fetching policies, change staging and integration, content transformation, security, user expectations, and other relevant topics. We will also discuss the impact of HTTP1.1 and other topical standards. --------------------------------------------------------------------------- MobiCom'97 Registration Form ---------------------------- Last Name (Surname): _____________________________________________________ _____________________________________ [ ] Prof. [ ] Dr. [ ] Mr. [ ] Ms. First Name: ______________________________________________________________ Title/Position: __________________________________________________________ Company/Organization: ____________________________________________________ Address: _________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ Telephone: _____________________________ Fax: ____________________________ E-mail address: __________________________________________________________ WWW homepage URL: ________________________________________________________ Name on badge: ___________________________________________________________ [ ] ACM or [ ] IEEE membership #: ________________________________________ Special needs (please describe): _________________________________________ _________________________________________ Vegetarian Meals: [ ] Yes [ ] No TUTORIAL SELECTIONS: Please select the tutorials you would like to attend: Friday, September 26: [ ] T1 (Full-Day) Wireless ATM: Standards, Architectures, Protocols & Implementation, Lou Dellaverson (Motorola, USA), C.-K. Toh (Hughes Research Laboratories, USA), and Arup Acharya (NEC, USA) [ ] T2 (Full-Day) Mobile IP: Adding Mobility to the Internet, Charles E. Perkins (Sun Microsystems, USA) Saturday, September 27: [ ] T3 (Full-Day) Simulation of Large Mobile Wireless Networks, Rajive Bagrodia and Mario Gerla (University of California at Los Angeles, USA) [ ] T4 (Half-Day, Cellular Wireless Networks: Principles and Morning) Operation, Zygmunt J. Haas (Cornell University, USA) [ ] T5 (Half-Day, Disconnected and Weakly Connected Access to the Afternoon) World Wide Web: Issues and Techniques, Murray S. Mazer (Open Group Research Institute, USA) WORKSHOP SELECTION: If you would like to attend one of the two workshops, please make your selection below (both workshops will be held on October 1, immediately following the conference technical program): [ ] WOSBIS The Second International Workshop on Satellite-based Information Services [ ] DIAL-M The First International Workshop on Discrete Algorithms and Methods for Mobile Computing and Communications REGISTRATION FEES: Early Registration Late Registration (Through August 29) (After August 29) Fee for each half-day tutorial: ACM/IEEE Members: [ ] $150 [ ] $200 Non-members: [ ] $200 [ ] $250 Full-time Students: [ ] $50 [ ] $70 Fee for each full-day tutorial: ACM/IEEE Members: [ ] $200 [ ] $250 Non-members: [ ] $250 [ ] $300 Full-time Students: [ ] $75 [ ] $95 Conference registration fee: ACM/IEEE Members: [ ] $400 [ ] $450 Non-members: [ ] $450 [ ] $500 Full-time Students: [ ] $100 [ ] $120 Workshop registration fee: ACM/IEEE Members: [ ] $100 [ ] $150 Non-members: [ ] $120 [ ] $170 Full-time Students: [ ] $50 [ ] $70 Total registration fees: Half-day Tutorials (____ half-day tutorials * $______) $_______ Full-day Tutorials (____ full-day tutorials * $______) $_______ Conference Registration $_______ Workshop Registration $_______ Optional Dinner Cruise (____ tickets * $45) $_______ Total $_______ PAYMENT INFORMATION: [ ] I have enclosed a check or money order in USD payable to MobiCom'97 Please charge the Total above to my: [ ] VISA [ ] MasterCard [ ] American Express Credit Card #: _____________________________ Expiration Date: _________ Name as it appears on card: ___________________________________________ Signature: ____________________________________________________________ SEND PAYMENT TO: To register for MobiCom'97, print this form, fill it out, and mail or fax it to: ACM/IEEE MobiCom'97 c/o Ms. Nadine Hunley Lucent Technologies Bell Laboratories, Room 3K-331 101 Crawfords Corner Rd. Holmdel, NJ 07733 USA Telephone: +1 732 949-0819 Fax: +1 732 834-5906 E-mail: nhunley@lucent.com You may also register by e-mail by completing and returning this plain text copy of the registration form. Please note that your credit card number is not secure when transmitted through e-mail. Payment by check, money order, or credit card must accompany your registration form. Purchase orders cannot be accepted. All fax and e-mail registrations must be paid by credit card. All registration fees above are in U.S. Dollars (USD) and must be paid in U.S. Dollars. A credit card signature will be required at the conference for e-mail registrations. Note: Written requests for refunds must be postmarked no later than September 12, 1997. Refunds are subject to a US $50 service charge. Participants with confirmed registration who fail to attend or notify MobiCom registration of cancellation before the refund date are subject to the full fee. Substitutions are allowed at any time. Registrations received after September 12, 1997 will be processed on-site only. All conference registrations include attendance at conference sessions, a copy of the conference proceedings, the Welcome Reception on September 27, the Conference Lunch and Conference Dinner Banquet on September 28, and coffee breaks. Breakfast is included in all hotels offered for the conference. Additional tickets to the Conference Dinner Banquet and additional copies of the conference proceedings will be available for additional cost. Please inquire when you register if you are interested in additional banquet tickets. Tutorial registration includes attendance at the tutorial, tutorial notes, and coffee breaks; full-day tutorials also include lunch. Workshop registration includes attendance at the workshop, workshop proceedings, lunch, and coffee breaks. An optional dinner cruise on the Danube River is being arranged for Monday evening, September 29. The price for this cruise is $45 per person and is not included in your conference registration fee. If you would like to join us for this dinner cruise, please mark the number of tickets desired on your registration form and add the appropriate amount to your total registration. --------------------------------------------------------------------------- MobiCom'97 Hotel Reservation Form --------------------------------- Please return to CONTOURS by August 15, 1997 Last Name (Surname): _____________________________________________________ _____________________________________ [ ] Prof. [ ] Dr. [ ] Mr. [ ] Ms. First Name: ______________________________________________________________ Company/Organization: ____________________________________________________ Address: _________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ Telephone: _____________________________ Fax: ____________________________ E-mail address: __________________________________________________________ Arrival Date: ______________________ Departure Date: _____________________ Sharing room with: _______________________________________________________ Special needs (please describe): _________________________________________ _________________________________________ Vegetarian Meals: [ ] Yes [ ] No HOTEL SELECTION: Please select the hotel and type of room you would like to reserve: Single Room Double Room Hotel ATRIUM HYATT: Room with Danube-view: USD 277 USD 294 Room without Danube-view: USD 230 USD 246 Hotel TAVERNA: USD 98 USD 127 Hotel GELLERT: USD 89 USD 150 City Panzio MATYAS: USD 66 USD 84 City Panzio PILVAX: USD 66 USD 84 Hotel VENTURA: USD 51 USD 60 All hotel rates are per night and include breakfast and VAT. All prices are in U.S. Dollars (USD). For more information on the available hotels, see the complete Advance Program or visit the MobiCom'97 Home Page at http://www.monarch.cs.cmu.edu/~mobicom97/. RESERVATION DEPOSIT: Deposit equal to one night in the chosen hotel: USD ______ Bank commission and handling fee: USD ___ 12 Total: USD ______ DEPOSIT PAYMENT INFORMATION: [ ] Bank cheque or money order in U.S. Dollars payable to CONTOURS. Private cheques cannot be accepted. [ ] Eurocheque in Hungarian Forint (HUF) payable to CONTOURS. The limit of one cheque is HUF 30000. [ ] Bank transfer to account number 10200885-32613003-00000000 to Hungarian Credit Bank (H-1539 Budapest 114, P.O. Box 624), made out to the order of CONTOURS. Your bank transfer must indicate your name and "MobiCom'97". [ ] VISA [ ] Eurocard/MasterCard [ ] American Express [ ] Diners Credit Card #: _____________________________ Expiration Date: _________ Name as it appears on card: ___________________________________________ Billing address: ______________________________________________________ _______________________________________________________________________ Date: _________________________________________________________________ Signature: ____________________________________________________________ SEND RESERVATION TO: Please print this form, fill it out, and return a copy not later than August 15, 1997 to: CONTOURS Congress Bureau Alkotas u. 47 H-1123 Budapest Hungary Telephone: +36-1-2122239 or 2122240 Telephone/Fax: +36-1-1566712 E-mail: contours@contours.ind.eunet.hu WWW: http://contours.aux.net/index2.htm Your reservation form should be accompanied by a deposit equal to one night at the chosen hotel. CONTOURS can confirm your reservation only once the deposit arrives. All payments should be directed to CONTOURS Congress Bureau at the address above. An additional USD 12 should be sent with your payment to cover the bank commission and the handling fee. A limited number of rooms has been reserved at each hotel. Hotel reservation will be made on a first-come first-served basis. If the hotel requested is fully booked, CONTOURS will suggest another hotel. All requests, changes, or cancellations in hotel reservations should be directed to CONTOURS. A confirmation will be sent by CONTOURS showing your request and the money received, and including detailed information on the hotel reserved. The deposit will be deducted from the whole amount of the accommodation. The balance must be paid upon arrival at the CONTOURS Registration Desk at the conference (not to the hotel). At the CONTOURS Registration Desk, you can arrange the rest of your payment by credit card, traveler's cheque, or cash. Only extras are paid directly to the hotel. Note: Cancellations of hotel reservation must be sent in writing to CONTOURS. If your cancellation is received before September 1, 1997 the deposit minus USD 20 will be refunded. Cancellations received after this date are not entitled to any refund. All refunds will be made after the close of the conference. --------------------------------------------------------------------------- ------------------------------ Date: Fri, 25 Jul 1997 14:11:03 -0400 From: "Eric A. Bobinsky" Subject: FYI-- FW: Draft IPPM Charter >From: randy@its.bldrdoc.gov >Date: Fri, 25 Jul 97 10:39:35 PDT >Subject: FW: Draft IPPM Charter >To: t1a13@t1.org >Cc: t1a1-internet@t1.org >Sender: owner-t1a13@t1.org >Reply-To: randy@its.bldrdoc.gov > >For the information of T1A1.3 participants... > >--- On Fri, 25 Jul 1997 12:13:30 -0400 Guy T Almes wrote: >Received: from betelgeuse.advanced.org by ntia.its.bldrdoc.gov with SMTP > (1.38.193.4/16.2) id AA10937; Fri, 25 Jul 1997 10:32:40 -0600 >Return-Path: >Received: from bianca by betelgeuse.advanced.org via SMTP (951211.SGI.8.6.12.PATCH1502/940406.SGI) > id MAA19556; Fri, 25 Jul 1997 12:19:32 -0400 >Message-Id: <3.0.1.32.19970725121330.015eae08@betelgeuse.advanced.org> >X-Sender: almes@betelgeuse.advanced.org >X-Mailer: Windows Eudora Pro Version 3.0.1 (32) >Date: Fri, 25 Jul 1997 12:13:30 -0400 >To: ippm@advanced.org >From: Guy T Almes >Subject: Draft IPPM Charter >Cc: almes@advanced.org, Scott Bradner , > allyn@eng.sun.com >Mime-Version: 1.0 >Content-Type: text/plain; charset="us-ascii" > >Dear Colleagues, > Please read and review the enclosed draft charter for the new >IPPM working group. > -- Guy >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >IP Performance Metrics (IPPM) > >The IPPM WG will develop a set of standard metrics that can be applied >to the end-to-end quality, performance and reliability of an IP >datagram service. These metrics will be designed such that they can be >performed by the providers themselves, customers, potential customers or >independent testing groups. It is important that the metrics not represent >a value judgement (i.e. define "good" and "bad"), but rather provide >unbiased quantitative measures of performance. > >The areas covered will include: > I) Path Performance > A) Delay and packet loss of datagrams > B) Bulk data transfer performance of large TCP flows > II) Availability > A) End to end availability > B) Time to recover (e.g., switch to backup paths) > C) Route stability (e.g., Spurious route transitions) > III) Multicast stability and performance > IV) others TBD > >Services peripheral to layer 3 IP service, such as NOC/NIS services, >DNS, etc. are beyond the scope of this working group. The IPPM WG >will define specific metrics, cultivate technology for the accurate >measurement and documentation of these metrics, and promote the sharing >of effective tools and procedures for measuring these metrics. It will >also offer a forum for sharing information about the implementation and >application of these metrics, but actual implementations and applications >are understood to be beyond the scope of this working group. > > >Goals and Milestones > >August 1997 Submit, as an informational RFC, a framework document > describing the creation of metrics in the working group. > >August 1997 Create drafts of standard metrics for connectivity > and treno-bulk-throughput. > >August 1997 Share information on implementations of delay and > loss. Update metric specifications based on > implementation experience. > >December 1997 Share information on implementations of connectivity > and treno-bulk-throughput. Update metric specifications > for connectivity, and treno-bulk-throughput based on > implementation experience. > >December 1997 Submit documents on delay and loss as RFCs. > >March 1998 Create drafts of standard metrics in the area of routing > stability. > >March 1998 Submit documents on connectivity and treno-bulk-throughput > as RFCs. > >August 1998 Share information on implementations of routing stability. > Update documents on routing stability based on > implementation experience. > >December 1998 Create drafts of standard metrics for multicast > performance and stability. > >December 1998 Submit documents on routing stability as RFCs. > >March 1999 Share information on implementations of the metrics for > multicast performance and stability. > >August 1999 Update the metrics for multicast performance and stability > based on implementation experience. > >December 1999 Submit documents on multicast performance and stability as > RFCs. > >Jan 1, 2000 Discover, in hindsight, that a Y2K reliability metric > would have been a good idea. :-) > > > > >-----------------End of Original Message----------------- > > > ------------------------------ End of tcp-over-satellite-digest V1 #33 ***************************************