Fedora cluster project? (was Re: [Beowulf] Opteron/Athlon Clustering)

[Mitchell Skinner]

On Wed, 2004-06-09 at 18:14, Jeffrey B. Layton wrote:
>     I've been using FC1 since 2 days after it came out. It's OK,
> but I've had a few problems with it (it locks up about once a
> week but I'm not sure that's the fault of FC1). I've found it to
> be decent overall. However, I think it's a bit too cutting edge
> for production clusters. The target for FC is more toward the
> desktop IMHO. That's the wrong target for clusters. At the
> ClusterWorld conference the panel on cluster distros was pretty
> much in agreement that a cluster distro needs to be lighter
> weight than a typical distro (I'll mention a few things below
> on that).

My experience with fedora has been that it's pretty solid, although I
haven't used it on a cluster.  I seem to recall seeing somewhere that
RGB has, and I'd be curious to know about that experience.

The main objection that I buy so far to Fedora on clusters relates to
ISV distro support; obviously if you're in that boat then Fedora is
probably more cutting edge than your ISV would like to deal with.  Linux
in general is getting more of a desktop focus lately and Fedora is right
there because it's closer to what's happening, but I don't get the
feeling that the distro itself is de-emphasizing non-desktop use.

In the absence of the ISV app thing, there's some sentiment that the
software versions in Fedora are not fully tested enough for "production"
use.  Obviously, Fedora is not as well tested as RHEL; however, I'm not
convinced that for clusters the extra functionality you get with newer
software isn't worth trading away some testing.

I don't see how the "lighter weight" issue relates to Fedora; obviously
your compute nodes are not going to have the full complement of packages
regardless.

>    Since I use cAos quite a bit let me make a few comments about
> it. cAos-1 is something like a "proof of concept" in that the developers
> wanted to get the build process down before moving on. cAos-2, which
> should be out very soon will be much more advanced. cAos is not a
> straight RHEL rebuild. I like to think of it as a RHEL rebuild as a
> starting point, but with more thought into the other components. Also,
> many of the cAos developers are cluster people so everything they do
> is focused on clusters to some degree. So it's a well thought out distro
> that works great on the desktop and works very well on clusters.

I certainly respect the effort that people are making with cAos.  On the
other hand, following the fedora list and peeking at the cAos lists
gives me the impression that fedora, as a release, gets more testing
from end users than cAos will for a while.  Obviously, cAos benefits
from the testing that the packages in RHEL have gone through; however,
the fact that it isn't a straight rebuild means that you can't rely on
that quite as much.

As an example, just the other day on the rocks list someone posted that
samba on 3.2.0 is broken because there's some version skew between the
different samba packages.  I know cAos is not Rocks, however I think
people underestimate the amount of work involved in coordinating a
release.

> Up to date is not always good (I've been there before :)

Agreed; on the other hand, if cAos-2 will be "much more advanced", then
aren't you/they moving a little in the more up-to-date direction as
well?

> I really don't like anaconda. Have you looked at the code - yuck! The
> installed in cAos (cinch) is a much better idea. It's not GUI, but that
> is a good thing IMHO. You can even hack it since it's just a bash script!

IMHO, anaconda can be as ugly as it wants, as long as red hat keeps
maintaining it and making sure it runs on a ridiculous variety of
hardware, which they appear to be doing.  Was running clean code the
principal reason for developing cinch, or was there functionality that
didn't exist in anaconda that the cAos developers wanted?


I think there's a lot to like about the warewulf approach, if you have a
homogenous hardware environment; I seem to recall seeing heterogenous
support being one of the goals for ww2 and I'm looking forward to that. 
We're not heterogenous but I think there's a decent chance that we'll
end up that way.  In any case, if I do end up participating in a fedora
clustering project I'd certainly want to package up the warewulf vnfs
and tools as an option there.

Mitch