[Beowulf] Gentoo in the HPC environment

[Matt Wallis]

On 2 Jul 2014, at 4:19 pm, Jonathan Aquilina <jaquilina at eagleeyet.net> wrote:

> How would the same arguments apply if you are just dealing with dns
> servers web servers databases etc.

If you're just dealing with standard services then you're not really doing high performance clustering, but fault tolerance.

In a typical enterprise data centre you're also more likely to have one server per application, or just a small handful of applications, and so you can configure each server appropriately.

In our environment we're specifically dealing with the case that not only are we running a handful of commercial packages, but also a lot of in house home grown code, often bought in from research centres around the world. They're written by many different people with access to a lot of different software and hardware, and we do our best to accommodate them all, all at once, in a single shared application space.

Beyond that, researchers also have to shoot for reproducibility. If they produce a set of results, then they should, and often do, release the exact details of the environment under which their results were obtained. This way if someone else gets different results to their research, using the same set of inputs, they can go back and rerun their simulations under the same code versions as last time, and compare to try and eliminate the possibility that their results were fraudulently generated (ie, they faked their research), they may have just been running on a buggy version of software.

Enterprises do often keep older versions of compilers and Operating Systems around for reasons of backwards compatibility. When Windows Vista came out, a Dev I was working with at the time, insisted on upgrading to Vista and the latest version of Visual Studio, despite his customer still being on XP. He later downgraded everything back to XP when he found he couldn't compile the clients application anymore, and his versions wouldn't work on their XP systems.

Compilers change, CPUs change, operating systems change, applications change, usually at different rates, and frequently in separate directions. How much you need to support all at once will be a big factor in how you go about setting up your environment.