Re: a validation question

[Sally Floyd <floyd@aciri.org>]

Lloyd -

>Okay, just built the latest ns snapshot under Solaris (and copied the
>newcast/ contents over, since they've been atticed). I'm slightly
>puzzled by the zillion warnings of:
>
>using backward compatibility mode
>
>on stderr 

Well, that one is because we had validation tests written in the
format accepted for NS1 but not for NS2, and in the change from
NS1 to NS2 I wanted these validation tests to run, with no change,
on NS2.  A second reason for NS1 scripts to be able to run,
unmodified, on NS2 is that a number of people (including me) had
NS1 scripts that they used and had made publically available, that
we did not want to have to rework.  

A third, not-irrelevant factor is that the validation scripts for
NS1 came from validation scripts that I used in 1990 for the
predecessor of NS1, a simulator derived from Keshav's REAL, and
maintained by Steve McCanne and myself, that went through several
incarnations before reaching NS1.  Each new incarnation of the
simulator (from C to C++, adding Tcl, etc.) required rewriting an
entire set of validation scripts because backward compatibality
was lost each time.  So the backward compatibility with NS1 scripts
is a key issue for me.

On the other hand, we don't want to encourage users to use these
maintained-only-for-backwards-compatibility formats in their
simulation scripts, because they are not the efficient way to do
things.  Therefore, the warning message about backward compatibility
mode.

If you had suggestions for rephrasing, that would be fine.  It
would also be possible to remove those messages from at least some
of the validation tests, but I don't think it would be particurly
pretty. (E.g., if the validation tests ends in *-v1.tcl, then it
is written in NS1 format, and we could tell the backward compatibility
code in "tcl/lib/ns-compat.tcl" to be quiet in that case...

(I don't believe that warning:
>Route to base_stn not known: dropping pkt
>should be on stderr; 

I don't know about this one - it is not my area.

>[..]
>../../ns test-suite-cbq-v1a.tcl cbqForOld QUIET
>WARNING:  This test was not run.
>This functionality is not implemented in ns-2.
>Test output agrees with reference output
>
>strikes me as odd.
>
>If the test was not run and not implemented, how can the test output
>possibly agree with the reference ouput? I'm now wondering about all
>the other 'Test output agrees with reference output' messages that I'd
>been happily taking for granted.

Ah yes.  A small bit of functionality in NS1 was never implemented
in NS2, such as an obscure variant of CBQ; or the old way of
specifying something in the script was not made backward compatible.
In cases where the simulation does not in fact run, the validation
test code in "tcl/test/test-all-template1" does not explicitly
detect this fact, but simply uses the file produced by the previous
validation test.  That means that the responsibility lies with the
person checking in the validation output (in this case, in
"tcl/test/test-output-cbq-v1") to know what is going on.

For these cases, I could change the script so that it doesn't make
the test, and doesn't say "Test output agrees with reference output",
but instead does nothing.  It is not at the top of my to-do list,
however, so possibly someone else will do it...

But the validation tests in fact work, and in fact do compare files.
To verify, just make a small change to tcp.cc, or to a default
variable in "tcl/lib/ns-default.tcl", and recompile, and run
validate, and it will be quite clear...

- Sally