[Beowulf] Cheap SDR IB

[Kozin, I (Igor)]

I thought the thrust of the original post was that you can build now a cheap IB cluster with up to 24 nodes. The subsequent discussion was around questioning whether you need IB for up to 16-24 nodes. 
The advantage you point to is for 32 nodes. There is no question that IB is much better at this scaling point for many codes, not just WRF.
I used to think that the typical break point is about 16 nodes. We have a lot of app data on our web site which confirm this and will be looking into how (if at all) quad cores change this.

As far as the games are concerned never underestimate what people can do if sufficient and affordable resources are provided. If the opposite were true we'd be still staring at black and white/green screens and playing packman. Game developers have to offload a lot (too much in fact) to a client out of necessity, not because they want to (BTW, there are amazing exploits because of the lags and the way the clients are built). Having said that there is no question that IB is hardly applicable to MMORPGs on the scale they are, probably not even for LAN parties. However a small cluster for real time ray tracing might be a good proposition.

Igor


-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Håkon Bugge
Sent: 31 January 2008 09:51
To: Mark Hahn
Cc: Beowulf Mailing list
Subject: Re: [Beowulf] Cheap SDR IB

At 21:01 30.01.2008, Mark Hahn <hahn at mcmaster.ca> wrote:
>whenever I ask about IB bandwidth, people always point fingers
>at weather codes, which apparently are fond of doing the transpose
>in multi-dimension FFT's using all-to-all.  while convenient, this
>seems a bit silly, since transpose is O(N) communications, not O(N^2).


Mark,

interconnect does matter. Here is a solid benchmark using WRF, 128 
cores, Woodcrest 3.00GHz. System spec can be found at 
http://www.spec.org/mpi2007/results/res2007q4/mpi2007-20071013-00029.html

Scali MPI Connect 5.6.2 using IB (IB as specified in the link above):
  Success 127.wrf2 base mref ratio 21.03, runtime 370.653066

Scali MPI Connect 5.6.2 using Gbe (Broadcom NetXtreme II BCM5708 
1000Base-T (B2), Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 
v1.3.29, 9k MTU, Switch? Jeff, can you fill in here, the system 
should be familiar to you):
  Success 127.wrf2 base mref ratio=4.82, runtime=1618.248048

That's a pretty decent advantage to IB, isn't?



Thanks, Hakon


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf