[Beowulf] some thoughts on thermal design, liquid cooling, etc.

[Jim Lux]

It occurs to me that the real limiting factor in producing "cluster 
oriented thermal design" is the volume of sales.

Say you want to design a custom motherboard/package for use in 
clusters.  This is, at a guess, probably a 3-5 million dollar project 
(maybe down around a million if it's real close to an existing design).

Say the cost of a node is around a kilobuck or 2 (in plain, non-custom, 
commodity trim).

If you had a cluster with 1000 of those custom mobos, you're looking at 
adding $3K/node to the cluster.  That's a bit punitive...  You could buy a 
lot of machine room and cooling for that $3 mil.

Now, on the other hand, if you had 100 people willing to each buy a cluster 
of this scale, then it's only adding $30-50/node, which is a lot more 
reasonable.

Compare this to the consumer motherboard market (which, after all, is what 
we are really using here...) A production run of several million mobos 
isn't all that huge, so a Dell or HP can and do create customized 
motherboard designs to meet some peculiar requirement (on-board 
peripherals, etc.).  Such customization only adds a buck to the mobo cost, 
and presumably, that buck is made up in cheaper packaging, shorter cables, 
one less manufacturing step, or somewhere.

Somehow, I doubt that the total sales of ALL motherboards for clusters, of 
a given instance of motherboard design, exceeds a million units.  Cluster 
buyers tend to want different processors, different peripherals, etc., and 
each configuration change would drive a whole new design cycle.


There is hope on the horizon.  The increasing drive to "media computers" is 
creating a demand for PCs that have high performance, but are quiet and 
have good cooling.  I have a Motorola Moxi BMC9012 "set top box" at home 
from the cable company, and it is basically a Linux computer with an 80GB 
drive and a some custom video hardware.  It's also hideously noisy (for 
something designed to sit in your living room) and dissipates >100W (all 
the time.. there's no on-off button).  There WILL be consumer pressure to 
make it silent and to do better thermal management.




James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875