[Beowulf] how Google warps your brain

[Bill Rankin]

Good points by Jim, and while I generally try and avoid "me too" posts, I just wanted to add my two cents.


In my previous life I worked on building a central HPC cluster facility at Duke.  The single biggest impediment to creating this resource was actually trying to justify its expense and put a actual number on the cost savings of having a centrally managed system.  This was extremely difficult to do given the way the university tracked its infrastructure and IT costs.

If a research group bought a rack or two of nodes then they were usually hosted in the local school/department facilities and supported by local IT staff.  The cost of power/cooling and staff time became part of a larger departmental budget and effectively disappeared from the financial radar.  They were not tracked at that level of granularity.  They were effectively invisible.

Put all those systems together into a shared facility and all of a sudden those costs become very visible.  You can track the power and cooling costs.  You now have salaries for dedicated IT/HPC staff.  And ultimately you have one person having to cut some very large checks.  And because of the university funding model and the associated politics it is extremely difficult, if not impossible, to actually recoup funds from the departments or even the research groups who would be saving money.

In order to make it work, you really need the senior leadership of the university to commit to making central HPC infrastructure an absolute requirement, and sticking to that commitment when it comes budget time and the politics are running hot and heavy over who gets how much.

Now to most of us this is a rehash of a conversation that we have had often before.  And with clusters and HPC pretty much established as a necessity for any major research university, the development of central facilities would seem to be the obvious solution.  I find it somewhat concerning that institutions like Harvard are apparently still dealing with this issue.


-bill