epic100 hangs on SMP-Machine
Terry Barnaby
terry@beam.demon.co.uk
Tue May 11 05:38:38 1999
Christian,
It looks like the same problem we were having with the SMC card. I
have put our version of the device driver on our website in:
http://www.beam.demon.co.uk/pub/epic100.c
The problems were:
Subject: Network locking up with SMC EPIC/100 Ethernet cards
We are experiencing a problem with the SMC 83C171 chip on SMC EtherPower II
PCI Ethernet cards on the Linux operating system. In general the driver works
well and can transfer
at a bi-directional rate of around 6MBytes per second between nodes. However,
in particular circumstances, one of which is where a TCP/IP process is
ping-ponging a lump of data the network can lock up for a period.
On in depth driver testing it was found that the 83C171 stops running
the transmit ring queue. There are two entries in the transmit queue with the
ring status fields set to: 0x6003 and 0x8000. The chip will restart on
setting the TXQUEUED bit high. Quite often the fault will not occur for some
time (2mins) and then repeatibly fail every second for a period.
I believe we have tracked down this problem. It appears to be a race hazard in
setting the transmit ring buffers status register to setting the TXQUEUED bit.
In certain situations, for example when of two nodes ping ponging network
packets, then the situation can arise, on a fast machine, where a packet is
placed into the ring and the TXQUEUED bit is set just as the EPIC chip has
come to the end of the ring and is clearing the TXQUEUED bit. This can leave
two packets in the ring awaiting transmission. This is only cleared on some
other packets requiring transmission setting the TXQUEUED bit.
This problem may occur in other network drivers including the Tulip driver.
I am not sure of the best way to cure this. One way that appears to work is
to check when the TxEmpty interrupt occurs if there are more than one packet
in the ring. If so then set the TXQUEUED bit. Another way is a delay between
the setting of the status ring entry and setting the TXQUEUED bit.
In looking at the driver there appears to be another possible problem:
1. In the epic_interrupt routine the variable dirty_tx should be
an unsigned int. After 2G packets the driver may fall over
(10 hours at peak rate). Actualy I beleive the driver fall
over after 4G packets anyway ? Should'nt the cur_tx/dirty_tx
and apporiate RX pointers be reset when they get larger.
This appears to be in the Tulip driver as well.
The transmit code is basically:
/* Set up TX Ring entry */
ep->tx_ring[entry].txlength = (skb->len >= ETH_ZLEN ? skb->len : ETH_ZLEN);
ep->tx_ring[entry].bufaddr = virt_to_bus(skb->data);
ep->tx_ring[entry].buflength = skb->len;
ep->tx_ring[entry].control = 0x14;
entry++;
/* Inserting a delay here seems to cure the problem */
/* udelay(1); */
outl(TxQueued, dev->base_addr + COMMAND);
The TxQueued is only set here in the driver code. The Interrupt routine
gathers the status on transmitted packets.
Any ideas would be gratefully received.
Terry
THE PROBLEM
==============================================
We have a problem with networking under Linux. We believe it is associated with
the EPIC/100 Ethernet card driver, but it could be the Linux networking system.
The problem is that the Ethernet interface seems to lock up occasionally. It
appears that the system will accept no more packets until a packet is
transmitted. It appears to be timing or packet size related.
We have noticed this problem for around 6 months now with other motherboards,
processors, hubs and switches, kernel versions and EPIC driver versions,
but have finally got around to trying to track down the problem.
We have a simple test program that demonstrates the problem.
The test application ping-pongs a 2568 byte lump of data between two
processes running on two nodes, using the write(), read() system calls. The
TCP/IP socket has been set to TCP_NODELAY. When not locked up the test program
reports a bi-directional data rate of 5.708708 MBytes per second. Buffer
transmit sizes of 2568 seems to cause this to happen more frequently.
With this saturated test the system will lock up about once per 10 seconds.
Pinging from the remote machine unlocks the network.
The current systems are:
Motherboard: Supermicro P6DBU
Processors: Pentium II 450Mhz
Ethernet: SMC Etherpower II (Chip: 83C171A2QF P, A29840, BH6482.1)
Linux: RedHat 5.2
Kernel: 2.0.36
Driver: EPIC/100 v1.03, 1.06, 1.07
Hub: Allied Telesyn MR904TX
We have noticed this problem for around 6 months now with other motherboards,
processors, hubs and switches, kernel versions and EPIC driver versions,
but have finally got around to trying to track down the problem.
Setting the EPIC100.c drivers debug level to 4 does not provide and messages.
There are no errors reported in /proc/net/dev
The /proc/tcp file has the following entry when locked up
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt
uid timeout inode
0: 0200A8C0:13D8 0100A8C0:041B 01 00000454:00000000 01:0000037C 00000006
1002 0 1663
1: 0200A8C0:13D8 00000000:0000 0A 00000000:00000000 00:FFFF405E 00000000
1002 0 1660
When Running normally it has the following
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt
uid timeout inode
0: 0200A8C0:13D8 0100A8C0:041B 01 00000A08:00000000 01:00000014 00000000
1002 0 1663
1: 0200A8C0:13D8 00000000:0000 0A 00000000:00000000 00:FFFEFC69 00000000
1002 0 1660
We have swapped the SMC EPIC/100 board for an older SMC Tulip based card
and all works fine. So this points to the EPIC/100 driver.
It appears that an older version of the SMC Chip 83C170 may not have this
problem or it might be different.
> Hello,
>
> I have quiet some problems with a SMB 9432BTX, (epic driver 1.06, 10Mbit/s,
> twisted pair) running under Linux 2.2.2 on a SMP machine. After some time of
> working the network card suddenly hangs up, without further error messages.
> Vice versa, after several hours of being deaf and blind, it starts working
> again -- sometimes when you sort of "force" this by unplugging and
> replugging the cable, shutting down and bringing up the network-level etc.
>
> ifconfig returns reasonable values, the pci and irq seems to be setup
> correctly.
>
> eth0 Link encap:Ethernet HWaddr 00:E0:29:37:65:03
> inet addr:161.42.65.23 Bcast:161.42.65.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0
> Interrupt:10 Base address:0xa800
>
>
> I've already read about similar problems in this mailing list. Is there some
> "definitive" fix for these kinds of problems?
>
> Greetings
> Christian
>
> E-Mail: Christian_Ohr@med.uni-heidelberg.de
> | To unsubscribe, send mail to Majordomo@cesdis.gsfc.nasa.gov, and within the
> | body of the mail, include only the text:
> | unsubscribe this-list-name youraddress@wherever.org
> | You will be unsubscribed as speedily as possible.
>
--
Dr Terry Barnaby BEAM Ltd
Phone: +44 1454 324512 Northavon Business Center, Dean Rd
Fax: +44 1454 313172 Yate, Bristol, BS17 5NH, UK
Email: terry@beam.demon.co.uk Web: www.beam.demon.co.uk
BEAM for: Visually Impaired X-Terminals, Parallel Processing, Software Dev
"Tandems are twice the fun !"
| To unsubscribe, send mail to Majordomo@cesdis.gsfc.nasa.gov, and within the
| body of the mail, include only the text:
| unsubscribe this-list-name youraddress@wherever.org
| You will be unsubscribed as speedily as possible.