Re: 512 parallel FTP sessions very slow?

[Maarten Litmaath <litmaath@fnal.gov>]

Dear NS experts,
recently I asked you for clues that would explain the slowness of the
following supposedly simple simulation of 512 parallel FTP sessions
(to estimate the maximum performance for a switch simulation we have
in mind):

#----------------------------------------------------------------------------
set ns [new Simulator]

$ns set-address-format expanded

set maxnode 512
#set maxnode 511

set f [open out.tr w]
$ns trace-all $f
set nf [open out.nam w]
$ns namtrace-all $nf

for {set i 0} {$i < $maxnode} {incr i} {
    set node($i) [$ns node]
    set k [expr $i + $maxnode]
    set node($k) [$ns node]
    $ns duplex-link $node($i) $node($k) 10Mb 10ms DropTail
    $ns queue-limit $node($i) $node($k) 100
    set tcp [$ns create-connection TCP $node($i) TCPSink $node($k) 0]
    set ftp [$tcp attach-source FTP]
    $ns at 0.000 "puts \"\[$ns now] start $i\"; $ftp start"
    $ns at 0.100 "$ftp stop; puts \"\[$ns now] stop $i\""
    puts "node $i defined"
}

$ns at 0.200 "close $f; close $nf; $ns halt"
$ns run
#----------------------------------------------------------------------------

Compared to the original code we made a few modifications that do not change
the behavior: we tried using 10Mb and 10ms for the duplex link, we set the
queue-limit to 100, and we added logging to see what is going on.

We find that this code takes 1.5 hours on a Sparc-20, which is much too
much and inconsistent with other people being able to simulate thousands
of nodes at a time.

Guillermo Rodriguez Garcia <guille@iies.es> suggested that a good fraction
of this time was due to the topology generation, but according to Mark Allman
<mallman@grc.nasa.gov> NS is spending "the vast majority of its time running
the simulation, not setting it up" (he dumped debug info periodically).

Experimenting further I found that up to 511 (!) nodes the simulation seems
to take a reasonable time: 38 minutes for 0.1 (!) second of simulated time.
When I add one more node it becomes 88 minutes!

Somehow NS behaves differently when the number of node address bits goes
from 9 (511) to 10 (512): does that ring a bell?

I also noticed that during this "simple" simulation NS grows to 76 MB,
and that the logfiles are NOT being written as the simulation proceeds,
but only at the end!  What kind of buffering is going on here?!
The logfile sizes are 5 and 11 MB, respectively.

Thanks for any further clues.  I will inform you of any progress.
Greetings,
	Maarten