[Beowulf] new release of GAMMA: x86-64 + PRO/1000

[Giuseppe Ciaccio]

On Sat, 4 Feb 2006, Tim Mattox wrote:

> Hello,
> I have been hoping to find the time to see what it would take to use both
> the GAMMA and my FNN stuff together.

Support for FNN with GAMMA is almost done.  I had a student working at this,
but while he was working I forked GAMMA towards the x86-64 porting, and now
we have to converge for a new release.  Unfortunately, it is always NP-hard
to converge two people's job schedules :-(

It was kinda trivial, BTW.

> BTW- Anyone have good or bad reports on GigE 48-port switch performance?

I have a bad behaviour report with some low-cost switches.  There is a
feature commonly implemented -- so called broadcast storm limitation.
If that feature is enabled, GAMMA seems to run into troubles (it uses bcast
packets for barrier, pkt loss detection, and other things).
Low-cost switches may not allow the admin to switch the bcast limitation off.
This is the case, for instance, with the 3COM 2824 (an unmanaged switch).

BTW:  Anybody who knows how to hack the 3COM 2824 so as to suppress the
broadcast limitation...?

> As others have posted, it would be cool if there was some
> form of GAMMA-lite that would work without needing a custom ethernet driver
> for each kind of NIC.  The e1000 is very common, but you don't tend to find
> Intel NIC parts built into motherboards for AMD Athlon64s or Opterons. ;-)

Such a GAMMA-lite approach would not perform much different from current TCP.
GAMMA achieves its performance thanks to a number of tricks, like: IRQ
suppression in the NIC, avoidance of skbuffs, monolithic code (no procedure
calls across different layers).  With the current Linux network architecture,
all of the above can only be achieved by happily hacking the network driver.

On Sat, 4 Feb 2006, Mark Hahn wrote:

> further, would Van Jacobson's "channels" concept help out here?
> http://www.lemis.com/grog/Documentation/vj/lca06vj.pdf
>
> channels are a "get the kernel out of the way" approach, which I think
> makes huge amounts of sense.

That would be a different, and better, architecture for the Linux
networking code.  Nothing really new -- it resembles U-Net, and VIA.
But, if I understand well, such an architecture implies an additional memcpy
on receive compared to current in-kernel TCP, unless the NIC is "intelligent"
(programmable) enough.

giuseppe