Hi William:
The SCPS team has been alerted that you are asking for comments on your
SCPS-TP testing report by 21 September:
http://roland.grc.nasa.gov/~ivancic/papers_presentations/SCPS_TCP_RATE_FINAL_0903.pdf
Unfortunately we don't have the resources to do an in-depth technical
evaluation of your work. However, we have taken a quick glance at the
results and have two recommended changes and some comments. In particular,
we would strongly suggest that (for consistency with the rest of the
technical data) you should re-write the Abstract, Executive Summary
(section 1) and Conclusions (Section 11) along the lines suggested below.
Best regards
Adrian Hooke, NASA-JPL
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
RECOMMENDED CHANGE:
1 ABSTRACT
Tests were performed at Glenn Research Center (GRC) to compare the
performance of the Space Communications Protocol Standard - Transport
Protocol (SCPS-TP,otherwise known as "TCP Tranquillity") relative to other
variants of TCP, and to
determine the implementation maturity level of these protocols -
particularly for higher speeds. The testing was performed over reasonably
high data rates of up to 100 Mbps with delays that are characteristic of
near-planetary environments. The tests were run for a fixed packet size,
but for variously errored environments. This report documents the testing
performed to date.
2 EXECUTIVE SUMMARY
There have been numerous discussions relative to the role of the SCPS-TP
in the ever-evolving TCP family, particularly with respect to other known
TCP features that accommodate long bandwidth-delay products (e.g., large
windows, selective acknowledgements). In order to gain a better
understanding of the performance of the SCPS-TP relative to other TCP
variants and to determine the maturity of the various options for use on
higher-rate space links, the NASA Space and Data Communications Systems
(SCDS) Office requested Glenn Research Center to perform a comprehensive
set of tests. Tests were
performed at Glenn Research Center to validate the operation of the
Reference Implementation of the SCPS-TP relative to its controlling CCSDS
specification, and to perform a comprehensive comparison of SCPS-TP
protocol options relative to other TCP options.
We have studied the effect of delay and BER on the performance of
congestion-friendly and rate-based protocols in emulated space links, both
under uncongested conditions and with limited congestion. The results
correlate well with previous testing of TCP options, including the SCPS-TP:
* The single stream and multi-stream test results clearly illustrate
that the TCP-Vegas enhancements to TCP (that are implemented by the
SCPS-TP) provide measurable performance improvements over a TCP SACK
implementation tested in high bandwidth-delay product environments. Since
performance considerations are subjective, the operational value of these
performance increases can best be assessed by the users of specific
applications hosted within those environments.
* Very small transactions such as command and control will probably see
little difference in performance for any variant of TCP.
* In extremely error-prone environments with high RTT latencies, use of
a rate-based TCP variant such as that provided by SCPS-TP is advisable,
assuming that the network implementation is properly engineered. Users are
cautioned against using rate-based protocols on networks where resource
allocation policies are based on contention.
* The tested SCPS-TP implementation was the protocol Reference
Implementation, which exists as a user application rather than as a more
generalized protocol service provider. For some deployments, an in-kernel
protocol service provider may prove more desirable than the deployment of
application level protocol service providers. The tradeoff is between the
more efficient use of resources and possibly higher performance for
in-kernel services versus the ease of maintainability and deployment of
application level implementations. Commercial in-kernel and application
level implementations of SCPS-TP exist, but have not been tested as part of
this effort.
* Even with equal performance, the SCPS-TP version of a rate-based
protocol may be more desirable to implement than other rate-based protocols
(such as the Multicast Dissemination Protocol) since the SCPS-TP is capable
of requiring only sending-side modification. This feature of the SCPS-TP
eliminates many of the risks associated with requiring universal deployment
of new software.
* Existing TCP implementations (drawn from a variety of communities,
and including the SCPS-TP) appear to satisfy all currently-known space
mission needs; however, the space mission community should maintain an
awareness of current and future TCP research that is being performed by
other communities.
11 CONCLUSIONS
(Repeat the same changes that are noted above in the Executive Summary.)
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
RECOMMENDED CHANGE:
Sec 4.2.2, "Modified Slow Start", there should be a bullet that reads:
* The Vegas Fast Retransmit algorithm is *not* part of SCPS-TP.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
COMMENT:
Section 9.1.1.3: the "optimum receiving buffer size" is mentioned, which
is defined to be in a fairly narrow range centered around one Bandwidth
Delay Product (BDP) of the long-delay test case. With SCPS's
Vegas-Corruption and Rate configurations, this is not optimal. Your
Solaris implementation might not have shown it due to OS-imposed penalties
on outside-the-kernel protocols and buffer management, but in high-error
conditions, the SCPS protocols are best configured with greater than one
BDP (e.g., two) of storage, so that the time spent waiting for a
retransmission to be received does not cause the sender to become window
limited. This isn't typically an issue with native TCP, as the error
recovery regime in the congestion control mechanisms limits the protocol's
ability to consume buffer space.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
COMMENT:
You appear to have missed the most important "Multiple Flow" test to
conduct. In Section 9.3, you state that "In addition,
SCPS-Vegas-Corruption was not tested in this environment, as it would have
been a misapplication of the protocol." This is seriously incorrect. The
attached MILCOM paper specifically examines the issue of use of the
Vegas-Corruption configuration for congestion control in the presence of
errors. It's absolutely the *RIGHT* application of the SCPS-TP protocol,
and is a significant omission from your paper.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
COMMENT:
Also, in that same section 9.3, you seem to have missed the opportunity to
draw some relatively important conclusions:
a) SCPS Vegas Congestion proved to be *more fair* than TCP with the Van
Jacobson congestion control algorithm.
b) Because there were fewer retransmissions required, the SCPS Vegas
algorithm was also *more efficient* than TCP with the Van Jacobson
congestion control algorithm.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
COMMENT:
In Section 10, you cite some problems with TCP Vegas in general (the last
three bullets before Section 11). It should be noted that the issue cited
in the second bullet is easily addressed and anticipates the deployment of
Explicit Congestion Notification (ECN). The other two bullets cite issues
in which Vegas algorithms are less aggressive than the Van Jacobson
algorithm, and will, no doubt, receive attention should these become real
problems.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-end-
This archive was generated by hypermail 2b29 : Fri Sep 20 2002 - 14:09:01 EDT