3c59x: recurring transmit errors

[Roderick Johnstone]

Ravi Swamy wrote:
> 
> On Tue, 28 Jul 1998, Dan Stromberg wrote:
> 
> > Ravi Swamy wrote:
> >
> > > Your friend is somewhat correct on Linux NFS writes.  Linux NFS writes
> > > to another Linux box are quite fast.  I've gotten over 2.5MB/s which I'm
> > > quite happy with.  Linux NFS writes to a Solaris server are horrendous.
> > > I can't even get 300kB/s.  No I'm not assigning blame, for all I know it
> > > could be Solaris but I've seen at least 3 people verify the same behavior
> > > and none knew how to "fix" it.  No, changing the rsize and wsize did not
> > > help at all.  Maybe 5% but that's it.  I've tried 1k, 2k, 4k, 8k, 16k, 32k.
> >
> > In my case, using an 8k rsize and wsize makes a VERY LARGE difference -
> > it yields a very nice performance improvement.
> 
> Even with 8k rsize,wsize I only get around 300kB/s.  This is on a PPro 200 w/
> a 3c905 on a 100Mb hub.  There is very little traffic on the hub, I can
> run the test at 1am and see very little difference.
> 
> In contrast I can take a 4 year old SPARCstation 5 running Solaris 2.5.1 w/
> a 10Mb link going to the same remote Solaris NFS server and get 1.03MB/s.
> This is copying the exact same file from the Linux box to Solaris server
> and a lowly 10Mb Solaris box to the Solaris server.
> 
> > In fact, I'd say the current default block size is a poor choice, and
> > should be raised to 8k, like on other unixes.
> 
> I just don't see it helping much.  It does seem to speed things up by
> maybe 25% but I need a 10X  increase.  My 300KB/s should be 3MB/s.
> 
> The Suns on the 100Mb hub can easily get 5 or 6 MB/s.
> 
> > IMO, the slowness problem is in the linux NFS code, because solaris,
> > irix and osf/1 (and possibly others) can read/write to/from each other
> > quickly.
> 
> I've never seen it explained.  I always hear people say "change the rsize/wsize"but it is just a >small improvement.  I can ftp files to and from Linux<->Solaris
> and get almost 8MB/s but NFS writes crawl...

Hi


We (me and my computing service support guys) looked into this nfs slow
write problem a while back. My understanding (I'm not the expert in
this...they are) is that the nfs2 protocol which is all that linux 2.0.x
can do requires that when writing a file across nfs the whole file is
sync'ed on the server after each nfs write. wsize can be 8192 bytes max,
I think, so increasing to this limit can help a bit, particularly for
small files,
although I think modern versions of redhat default to this value.
For larger files this re-writing (sync'ing) becomes catastrophically
long.

Let me say now that my experience with this is writing from linux
(redhat 5.0) to Digital unix 4.0B. A fellow sysadmin has confirmed that
he has similar problems from linux to Solaris 2.5 or maybe 2.6. If you
look at the activity light on the disk you are doing big file nfs writes
to you will see it stays on solid for the duration of the (long) write,
in some sense confirming the diagnosis that the files is being written
many many times.
This particular problem doesn't seem to occur linux-linux, so I guess
the nfs server in linux somehow doesn't obey the letter of the protocol
and doesn't keep re-syncing the file.

My hope is that this problem is solved in the 2.2.x kernels, but I'm not
sure if they do nfs3, although I think the nfs has been considerably
updated. As a quick test of what things might be like, my fellow
sysadmin put in a 2.1.1xx kernel and indeed the nfs write speed from it
to DUnix/Solaris
seemed
to be rather faster, but not as fast as solaris-solaris or dunix-dunix.
In the test done the linux 2.0.x to Solaris write was 25 times slower
than
soalris-solaris and linux 2.1.x to Solaris was 5 times slower than
Solaris-solaris.
I'm not sure if these factors scale with file size.

Hope this explains things a bit.

Roderick




--
Roderick Johnstone                          Email: rmj@ast.cam.ac.uk
X-ray Astronomy Group                       Phone: +44 1223 337510
Institute of Astronomy                      Fax:   +44 1223 337523
Madingley Road, Cambridge CB3 0HA, UK