Adrian,
Thanks for the input. It has been received in sufficient time to get into
the final report.
We will consider you input. I do have some questions, however, regarding
some of the suggestions.
We tested and set the optimum receiver buffer size starting a twice the
Bandwidth-Delay product (BDP) and found for our test setup with the OS used
(which appeared to be the most stable OS) that the optimal setting was
slightly less the 1 BDP. So, we could say that "in theory" the receive
buffer should be 2 x BDP, but the data did not support that of Solaris.
IMHO, protocol tuning doesn't scale well - particularly in non-gateway
implementations and for systems such as NASA's planned sensorWeb. I would
really like to see better auto-tuning features developed.
The paper on performance for "SCPS-TP assumed corruption", is for a gateway
implementations. All data is received at the gateway and buffered. Thus,
all data is buffered at one source and ECN type mechanisms can be
implemented on the point-to-point link. This is fine, but is a very
limited case. I would not be comfortable extrapolating such performance to
general deployments of "SCPS-TP assumed corruption" over fully meshed
networks. Am I missing something? Gateway testing is great, but assuming
everything will use a gateway is ..... maybe not to good. If one encrypts
prior to the gateway (very common in a military environment), your limited
as to what you can do. I'm always a bit fearful when I see testing
performed using gateways in that I fear decision makers may not understand
that those results may only be valid when used in an architecture that
allows for deployment of a gateway.
Assuming ECN in a fully meshed network may be wishful thinking. Although
ECN concepts have been around and experimentally show to be advantageous in
an ideal world, I believe, to date, deployment of ECN is limited at
best. I'm may be incorrect here as I do not have the compete knowledge
of what is deployed in the backbone. A quick breeze through the QoS
commands on Cisco routers did not show ECN settings, but that was a very
quick search.
Best Regards,
Will
At 11:07 AM 9/20/02 -0700, Adrian J. Hooke wrote:
>Hi William:
>
>The SCPS team has been alerted that you are asking for comments on your
>SCPS-TP testing report by 21 September:
>
>http://roland.grc.nasa.gov/~ivancic/papers_presentations/SCPS_TCP_RATE_FINA
>L_0903.pdf
>
>Unfortunately we don't have the resources to do an in-depth technical
>evaluation of your work. However, we have taken a quick glance at the
>results and have two recommended changes and some comments. In particular,
>we would strongly suggest that (for consistency with the rest of the
>technical data) you should re-write the Abstract, Executive Summary
>(section 1) and Conclusions (Section 11) along the lines suggested below.
>
>Best regards
>Adrian Hooke, NASA-JPL
>
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>RECOMMENDED CHANGE:
>
>1 ABSTRACT
>
>Tests were performed at Glenn Research Center (GRC) to compare the
>performance of the Space Communications Protocol Standard - Transport
>Protocol (SCPS-TP,otherwise known as "TCP Tranquillity") relative to other
>variants of TCP, and to
>determine the implementation maturity level of these protocols -
>particularly for higher speeds. The testing was performed over reasonably
>high data rates of up to 100 Mbps with delays that are characteristic of
>near-planetary environments. The tests were run for a fixed packet size,
>but for variously errored environments. This report documents the testing
>performed to date.
>
>2 EXECUTIVE SUMMARY
>
>There have been numerous discussions relative to the role of the SCPS-TP
>in the ever-evolving TCP family, particularly with respect to other known
>TCP features that accommodate long bandwidth-delay products (e.g., large
>windows, selective acknowledgements). In order to gain a better
>understanding of the performance of the SCPS-TP relative to other TCP
>variants and to determine the maturity of the various options for use on
>higher-rate space links, the NASA Space and Data Communications Systems
>(SCDS) Office requested Glenn Research Center to perform a comprehensive
>set of tests. Tests were
>performed at Glenn Research Center to validate the operation of the
>Reference Implementation of the SCPS-TP relative to its controlling CCSDS
>specification, and to perform a comprehensive comparison of SCPS-TP
>protocol options relative to other TCP options.
>
>We have studied the effect of delay and BER on the performance of
>congestion-friendly and rate-based protocols in emulated space links, both
>under uncongested conditions and with limited congestion. The results
>correlate well with previous testing of TCP options, including the SCPS-TP:
> * The single stream and multi-stream test results clearly illustrate
> that the TCP-Vegas enhancements to TCP (that are implemented by the
> SCPS-TP) provide measurable performance improvements over a TCP SACK
> implementation tested in high bandwidth-delay product environments.
> Since performance considerations are subjective, the operational value of
> these performance increases can best be assessed by the users of specific
> applications hosted within those environments.
> * Very small transactions such as command and control will probably
> see little difference in performance for any variant of TCP.
> * In extremely error-prone environments with high RTT latencies, use
> of a rate-based TCP variant such as that provided by SCPS-TP is
> advisable, assuming that the network implementation is properly
> engineered. Users are cautioned against using rate-based protocols on
> networks where resource allocation policies are based on contention.
> * The tested SCPS-TP implementation was the protocol Reference
> Implementation, which exists as a user application rather than as a more
> generalized protocol service provider. For some deployments, an in-kernel
> protocol service provider may prove more desirable than the deployment of
> application level protocol service providers. The tradeoff is between the
> more efficient use of resources and possibly higher performance for
> in-kernel services versus the ease of maintainability and deployment of
> application level implementations. Commercial in-kernel and application
> level implementations of SCPS-TP exist, but have not been tested as part
> of this effort.
> * Even with equal performance, the SCPS-TP version of a rate-based
> protocol may be more desirable to implement than other rate-based
> protocols (such as the Multicast Dissemination Protocol) since the
> SCPS-TP is capable of requiring only sending-side modification. This
> feature of the SCPS-TP eliminates many of the risks associated with
> requiring universal deployment of new software.
> * Existing TCP implementations (drawn from a variety of communities,
> and including the SCPS-TP) appear to satisfy all currently-known space
> mission needs; however, the space mission community should maintain an
> awareness of current and future TCP research that is being performed by
> other communities.
>11 CONCLUSIONS
>
>(Repeat the same changes that are noted above in the Executive Summary.)
>
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>RECOMMENDED CHANGE:
>
>Sec 4.2.2, "Modified Slow Start", there should be a bullet that reads:
>
> * The Vegas Fast Retransmit algorithm is *not* part of SCPS-TP.
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>COMMENT:
>
>Section 9.1.1.3: the "optimum receiving buffer size" is mentioned, which
>is defined to be in a fairly narrow range centered around one Bandwidth
>Delay Product (BDP) of the long-delay test case. With SCPS's
>Vegas-Corruption and Rate configurations, this is not optimal. Your
>Solaris implementation might not have shown it due to OS-imposed penalties
>on outside-the-kernel protocols and buffer management, but in high-error
>conditions, the SCPS protocols are best configured with greater than one
>BDP (e.g., two) of storage, so that the time spent waiting for a
>retransmission to be received does not cause the sender to become window
>limited. This isn't typically an issue with native TCP, as the error
>recovery regime in the congestion control mechanisms limits the protocol's
>ability to consume buffer space.
>
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>COMMENT:
>
>You appear to have missed the most important "Multiple Flow" test to
>conduct. In Section 9.3, you state that "In addition,
>SCPS-Vegas-Corruption was not tested in this environment, as it would have
>been a misapplication of the protocol." This is seriously incorrect. The
>attached MILCOM paper specifically examines the issue of use of the
>Vegas-Corruption configuration for congestion control in the presence of
>errors. It's absolutely the *RIGHT* application of the SCPS-TP protocol,
>and is a significant omission from your paper.
>
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>COMMENT:
>
>Also, in that same section 9.3, you seem to have missed the opportunity to
>draw some relatively important conclusions:
>
>a) SCPS Vegas Congestion proved to be *more fair* than TCP with the Van
>Jacobson congestion control algorithm.
>
>b) Because there were fewer retransmissions required, the SCPS Vegas
>algorithm was also *more efficient* than TCP with the Van Jacobson
>congestion control algorithm.
>
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>COMMENT:
>
>In Section 10, you cite some problems with TCP Vegas in general (the last
>three bullets before Section 11). It should be noted that the issue cited
>in the second bullet is easily addressed and anticipates the deployment of
>Explicit Congestion Notification (ECN). The other two bullets cite issues
>in which Vegas algorithms are less aggressive than the Van Jacobson
>algorithm, and will, no doubt, receive attention should these become real
>problems.
>
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>-end-
>
This archive was generated by hypermail 2b29 : Mon Sep 23 2002 - 09:01:27 EDT