[Beowulf] SiCortex experience anyone?

[Lawrence Stewart]

I guess I am the token SiCortex person who posts here sometimes.   
There are probably other
lurkers as well.  Hopefully this is generic enough to not trigger a  
commercial
allergic response from the moderator...

The philosophical idea was something like noticing that HPC codes tend  
to
miss in the cache, with a "flops per miss" figure of merit.

So we tried to build a system with relatively less CPU but just as  
much memory
and very good communications.

As a consequence, we tend to do very well on communications intensive  
codes,
and relatively less well on codes that smell like DGEMM or which have an
unusually large amount of integer stuff.

As a general matter, your code will be limited by something (memory,  
comm, cpu,
I/O are the usual suspects), so a balanced system is one is which  
everything
breaks at once.

We're probably unbalanced towards comms, whereas a commodity cluster is
(usually) unbalanced towards CPU.

Porting is pretty easy, in the sense that "configure" and "make" tend  
to just work,
but it isn't unusual to have to tune to get the best performance.   
We've worked
pretty hard to make the performance tools play well together.

Is it a beowulf?  The software philosophy is about the same, but the  
hardware isn't
really commodity, except the DIMMs.

Regarding scalability of the systems, they range from 72 to 5832  
cores, and are mostly
assembled from the same boards with different backplanes.  We think we  
know how to
cluster systems as well, should anyone turn up with a suitable  
suitcase of cash.

-Larry