benchmarking and performance tuning

[Robert G. Brown]

On Wed, 22 Jul 1998, Peter Stein wrote:

>  
> First, thank you Mr. Becker for providing the tulip driver.
> Thanks to this driver I have 2 Linux systems connected in
> a very cost effective way (2 Netgear FA310TX NICs connected
> via a category 5 crossover cable).
>  
> Can someone on this mailing list suggest where I might find
> information on:
>  
> 1. Determining if my ethernet configuration is performing at
>    the optimum level.

See the (tulip, actually) network performance chart on:

www.phy.duke.edu/brahma/

and the associated links to the home of the netperf program suite.
There are a couple other network performance tools, but netperf is
quite general and powerful.

> 2. Tweaking and tuning the setup to improve performance.

There isn't a lot do to do tweak or tune at the kernel level.  Most of
the remaining tweaking and tuning involves arranging your code or
whatever to use the channels efficiently.  This involves a combination
of load management/synchronization, ensuring that message sizes are
generally larger than around 1 KB (some of the actual optimum points
are discussed on the web page), and selection of protocol (i.e. UDP vs
TCP vs RPC).

> Right now I'm seeing a transfer rate of 18-19 Mbps on a ftp of
> a large file to /dev/null. It seems like the driver is causing
> the NICs to operate at their best setting (which is 100baseTX-FD).
> When I force the driver to 10baseT-FD the transfer rate seems to
> get cut in half. But 18-19 Mbps doesn't seem very good for
> 100baseTX-FD.
>  
> The driver revision I'm using is 89H. Any suggestions would be
> greatly appreciated.

The biggest problem you will encounter is that your optimum netperf
benchmarks (which SHOULD show that you can easily achieve roughly 95-97
Mbps UDP throughput at optimum packet sizes on quiescent hosts -- if
they are PPros or better, a hair less for TCP) do not reflect transfer
times on a busy network via real software operating a layer or two
down.  FTP, NFS and even rcp all make lousy benchmarks of whether or
not your network is performing, and yet they are often the things that
you want to perform well.  To my direct experience (on a switched
100BT network capable, as my benchmarks clearly show, of yielding
nearly 99% of the theoretically attainable bandwidth, once packet
header and the like are accounted for) NFS can EASILY be a factor of
ten or more slower than it "should" be.  FTP and rcp tend to run at
around 20%-50% of available bandwidth (sorry for the big range, but it
all depends on what is going on at the time), consistent with your
measurements.  I would have expected a bit better for your ftp
measurements -- perhaps you have just Pentiums?  or are running other
things in the background?  Notice that a large part of real network
communications is CPU bound, so having a busy CPU really does make a
big difference, especially on the receiver (which has to
asynchronously stop what it is doing to process an incoming packet --
very inefficient).

  Hope this helps,

    rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb@phy.duke.edu