[Beowulf] dollars-per-teraflop : any lists like the Top500?

[Gus Correa]

Hi Rahul, list

So far we have three data points, if I counted right:

1) Yours: US$35k/TFlop, 100-node Beowulf cluster.
2) Dmitry Chubarov's US$158k/TFlop, SKIF/Cyberia cluster,
Feb/2007, Russia
3) Mark Hahn's $CAD 30k/TFlop (approx US$28/Tflop), in Canada,
a Beowulf cluster, I presume, size not specified.

Not sure if all these numbers correspond to nominal Tflops (Rmax),
or, to actual HPL benchmark (Rpeak),
or to some estimate, say,(85% Rpeak/Rmax HPL)*(nominal Tflops),
or to some other performance metric.

I can add one point to the data table.
19 months ago we paid about US$37k/Tflop (nominal),
or US$43k/Tflop (actual HPL).
(Small Beowulf cluster w/ IB, IPMI, GigE,
not counting UPS, storage, and head node.)

I guess if you buy the latest greatest fastest processors they
add a quite a bit on the $ side.

The prices seem to be significantly higher if you buy
outside the USA, Canada, maybe the EU also,
as Dmitry's number suggests.

Cray recently sold a 244Tflop XT6
to Brazil for weather forecast for US$20M (US$82k/Tflop).
A bigger Petaflop version was sold to
Los Alamos Natl. Lab for US$45M (US$45k/Tflop):

http://www.theregister.co.uk/2010/04/29/brazil_buys_cray_xt6/
http://insidehpc.com/2010/04/21/cray-wins-big-deal-in-brazil/
http://www.hpcwire.com/offthewire/Cray-Wins-20M-Contract-with-Brazils-National-Institute-for-Space-Research-91710269.html
http://www.networkworld.com/news/2010/040210-cray-supercomputer.html
http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1415533&highlight=
http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1409130&highlight

Interesting that the $/Tflop ratio seems to (still)
be significantly lower in the Beowulfs,
even though commodity processors, RAM, etc,
are used more and more in the brand name machines.

Mark got the best deal.
Maybe we should go buy in Canada, as people do with medicine.  :)

Cheers,
Gus Correa

Rahul Nabar wrote:
> On Thu, Jul 1, 2010 at 11:41 AM, Gus Correa <gus at ldeo.columbia.edu> wrote:
>> Hey Rahul:
>>
>> Your original question went a long way, didn't it?  :)
>> Somehow you always ask questions that trigger these
>> interesting debates.
> 
> Yes it did. I am not sure if that is good or bad though. :) I hope it
> doesn't make me look like a "troll". I just had a genuine curiosity to
> know more about the economic side of HPC, economies of scale etc.;
> this is the stuff often in the shadows and I don't see much coverage
> in the typical material I read.
> 
> Best,
>