[Beowulf] Building new cluster - estimate

[Eric Thibodeau]

Joe Landman wrote:
>
>
> Eric Thibodeau wrote:
>
>> And select the main stuff (HDD drivers) as built in and don't fsck 
>> around with the initrd stuff, that's only usefull for kernels that 
>> need to be generic and adapt to all hardware (ie: install 
>> CDs)...other than that, monolithic a kernel works fine ;)
>
> Advantage of modules is you can upgrade them without upgrading the 
> kernel.  Go ahead, build in that e1000 driver.  I dare yah... :(
Ok...I didn't put enought emphasis on "main" stuff....as in, _all you 
need to get the system booted, which essentially means HDD chipset 
drivers, the rest I do build as a module (NIC, video and such).
>
> More to the point it does give some good flexibility for end users 
> with a need to keep the core "separate" from the drivers for maintenance.
>
> Initrd is subtle and quick to anger.  One must use burnt offerings to 
> placate the spirits of initrd.
LOL!
>
> Well, it would be a heck of a lot nicer if the tools were a little 
> more forgiving ... Oh you don't have this driver in your initrd ... ok 
> ... PANIC (mwahahahaha)
Pahahahahah... Point in case, I am building a CD-only cluster system 
(based on Gentoo) and I am currently _NOT_ using initrd because all that 
really needs to be built in is NFSroot support an all NICs I care to put 
in. Obviously this is a deprecated approach but it's proven to be the 
most effective and easy to maintain in my case.
>>
>> <rant>
>> ...and such. I'd tell you to use the Gentoo Clustering LiveCD but 
>> that's work in progress...you could still build the cluster using 
>> Gentoo...if you're performance savvy...and want things like OpenMP 
>> capable compiler 
>
> I have been hearing claims like this for a long time.  I have not seen 
> any real tests that back these claims up.  Do you have any?
I'm actually working on such benchmarks. Did you know that compiling 
with the default ICC optimization will cause your bridge to crumble due 
to floating point assumptions?...

Ok, so my computation have diverged horribly mostly because I am 
computing 47(vector size)*5000(K-Means clusters)*6,787,955(learning 
dataset)*5(iterations to convergence) for a total of 7,975,847,125,000 
FLOPS (or about 8Tera FLOPS) as part of an iterative learning process, 
the error adds up. So performance is very sensitive to what your 
intended goal is too ;)

>   Most of the arguments I have heard are "oh but its compiled with 
> -O3" or whatever. Any decent HPC code person will tell you that that 
> is most definitely not a guaranteed way to a faster system ...
Hey...as I stated above, one would have to be quite silly to claim -O3 
as the all well and all good optimization solution. At least you can 
rest assured your solutions will add up correctly with GCC. To get a 
"faster" system, you really have to look at your app, use strace, ltrace 
and gprof, then you can play with that. What I _am_ saying though is 
that Gentoo _does_ empower the administrator by giving him the ability 
to customize the OS if a bottleneck is to be identified.
>
>> (gcc-4.3.1, or ICC ;) ) _integrated_ into your system (not a hackish 
>
> Er... We often use several different compilers in several different 
> trees.  Several gccs, pgi, icc, eieio ... you name it.  All are 
> integrated.
Are-you currently able to run GCC-4.3.x versions on your current setup, 
I'm actually eager to know. I'm still living under the ASSumption od 
binary distributions not coping too well with multi-library 
environments. Point in case, one of my colleagues _really_ wanted 
firefox 3 on his ubuntu system. The installer trickled down to having to 
uninstall glibc...and he forced it to YES (and this is just a browser, 
not something that is used to _make_ code and would be tied to glibc)
>
>> afterthought of an RPM that pulls in a new glibc that breaks the install 
>
> Er ... not the slightest clue as to what you are talking about.  I 
> haven't seen gcc, icc, pgi, ... touch our glibc.
>
> Maybe I am missing the fun.  Which ICC version is this?  Which gcc is 
> this, which glibc is this?
>
Sorry about that I might have been misleading, GCC is generally the one 
most sensitive to glibc, not the other ones although the latest ICC 
(10.1.x series) do claim compatibility with the GNU environment so it 
might get a little more dependency there.

Cheers!

Eric