Problems similar to the short-initial-segment problem and the odd/short-final-segment problem have been encountered by Moldeklev and Gunningberg [14] and Crowcroft, Wakeman, Wang, and Strovica [5].
Moldeklev and Gunningberg describe how MTU affects TCP transfer efficiency [14]. They find that interactions between sender and receiver window sizes, Nagle's algorithm, delayed acknowledgements, and BSD socket buffering code can result in a number of conditions where TCP data transfer is tied to delayed acknowledgements (a ``throughput deadlock'' in their terminology). Their observations focus on a large-MTU networks (such as ATM) where in some circumstances all data transfer becomes deadlocked. This problem is similar to our short-initial-segment problem, however in our case the problem occurred due to application level behavior (flushing outgoing data) rather than due to MTU and buffer size interactions. Most current HTTP systems have combinations of buffering and MTU values which avoid the deadlocks observed Moldeklev and Gunningberg, however these problems could arise as MTU-discovery and large-MTU networks become more widely deployed [12].
Crowcroft, Wakeman, Wang, and Strovica experimented with SunRPC traffic over TCP. SunRPC calls have very similar behavior with HTTP requests and responses in P-HTTP. They found that mismatches between user- and TCP-level buffering caused a problem similar to the odd/short-final-segment problem we describe. Their conclusion is that more integrated approaches must be taken to multi-layer processing to avoid these kinds of performance problems. To this statement we would add that, when the actual implementations of different layers cannot be integrated, careful documentation and understanding of the buffering strategies at each layer is very important.
Finally, while the problems described in Section 2.5 (avoiding data copies and tuning socket buffers to the network bandwidth-delay) are well known, we are not aware that others have encountered the slow-start re-start problem described in Section 2.4. Hoe has addressed the problem of excessive packet loss due to aggressive slow-start rates by limiting the aggressive phase of slow-start [10]. This work is complementary to our approach where we pace packets instead of slow-starting after an idle connection. Hoe also has suggested (independent of our work) rate-based pacing as a potential future alternative to slow-start [9]. We are currently examining the effects of augmenting slow-start with rate-based pacing during re-starts; we expect to have a performance evaluation of our implementation shortly.