[Beowulf] What services do you run on your cluster nodes?

[Perry E. Metzger]

Greg Lindahl <lindahl at pbm.com> writes:
> On Tue, Sep 23, 2008 at 07:43:19PM -0400, Perry E. Metzger wrote:
>> As for the daemons, remember that with a proper scheduler, you will
>> switch straight from an incoming network interrupt to a high priority
>> process that is expecting the incoming packet, and that even works
>> correctly on some (but not all) Linux kernels. A user process cannot
>> take priority over other tasks, at least not without someone being
>> quite deliberate about it.
>
> You realize that most big HPC systems are using interconnects that
> don't generate many or any interrupts, right?

Of course. Usually one even uses interrupt pacing/mitigation even in
gig ethernet on a modern machine -- otherwise you're not going to get
reasonable performance. (For 10Gig, you have to do even uglier
tricks.)

However, my argument still holds without any change. Until you
actually process the packet, which happens in the kernel, userland
won't see it anyway, and when the kernel processes it, it is free to
switch to whatever userland process it wishes, and (under normal
circumstances) it will do the right thing.

So, unless you're going to do the computation in the kernel in a tight
loop polling driver, it is six of one, half a dozen or the other.

The truth is, of course, that if your userland code is so tightly
dependent on real time scheduling that any of this makes a difference,
you're just not going to get anything remotely like decent performance
on Linux on a PC because between scheduling coughs, SMM, processing
I/O interrupts, non-deterministic waits for disk I/O, etc, you would
lose -- which is why, of course, no real code is that insanely timing
critical.

Getting back to the original trigger of the thread, in practice, if
Postfix is hanging out and wakes up three times at 2am to send a
status message, no one is going to notice, not compared to all the all
the rest of the things that are going on.

> It doesn't seem that you've bothered to read the papers that several
> people have pointed you at.

Of course! I'm an ignoramus! Utterly foolish, utterly uninformed, not
worth listening to at all. I've also got "no experience in the field",
as you already explained so gracefully in an earlier message. In fact,
I can't even read or write!


Perry
-- 
Perry E. Metzger		perry at piermont.com